Centrum bone cement leakage early warning method and system based on magnetic resonance imaging

Abstract

According to the method of the embodiment of the invention, a centrum bone cement leakage early warning method and system based on magnetic resonance imaging are provided, and the method comprises the following steps: firstly, obtaining MRI image data of a centrum, and preprocessing the MRI image data to obtain standard image data; then, carrying out segmentation processing on the standard image data through Attention U-Net so as to extract centrum bone and soft tissue parts, and generating a three-dimensional geometric model; then, on the basis of bone cement injection parameters and the three-dimensional geometric model, a fluid-structure coupling finite element model is constructed for simulation calculation; and finally, inputting a simulation result into the multi-branch fusion network model for processing, and outputting a vertebral bone cement leakage result. According to the technical scheme, based on the multi-branch fusion deep network, the multi-moment finite element simulation and the medical image, more comprehensive, accurate and extensible prediction analysis of bone cement leakage is achieved, and the method has remarkable beneficial effects in clinical operation planning and risk prevention.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and particularly relates to a method and system for early warning of vertebral bone cement leakage based on magnetic resonance imaging.

Background Art

[0002] Bone cement is a material widely used in spinal surgery in recent years, mainly used in surgeries such as fracture repair, vertebroplasty, and spinal orthopedics, especially in the treatment of kyphosis deformity and vertebral compression fractures. As a functional material, the mechanism of action of bone cement is based on its adhesiveness, mechanical strength, and rapid hardening characteristics. In vertebroplasty, the injection of cement not only fills the fracture site but also restores the damaged bone structure by enhancing the rigidity of the fracture area. Cement leakage is a common complication in vertebroplasty. When bone cement is injected into the vertebra, if the cement fails to fully stabilize in the target area, it may leak into adjacent soft tissues, nerve roots, or even around the spinal cord, resulting in clinical symptoms such as pain, nerve compression, and even nerve damage. Therefore, inventing a method for early warning of vertebral bone cement leakage based on magnetic resonance imaging is a technical problem that needs to be urgently solved by those skilled in the art.

Summary of the Invention

[0003] In view of this, an embodiment of the present invention provides a method and system for early warning of vertebral bone cement leakage based on magnetic resonance imaging.

[0004] In a first aspect, an embodiment of the present invention provides a method for early warning of vertebral bone cement leakage based on magnetic resonance imaging, the method comprising:

[0005] S1. Obtain MRI image data of the vertebra, and obtain standard image data after preprocessing the MRI image data;

[0006] S2. Perform segmentation processing on the standard image data through Attention U-Net to extract the vertebral bone and soft tissue parts, and generate a three-dimensional geometric model;

[0007] S3. Based on the bone cement injection parameters and the three-dimensional geometric model, construct a fluid-structure coupling finite element model for simulation calculation;

[0008] S4. Input the simulation results into a multi-branch fusion network model for processing, and output the vertebral bone cement leakage results.

[0009] As described above in the aspect and any possible implementation manner, a further implementation manner is provided, where S1 includes:

[0010] S11. Use the MRI image data as the source image for denoising and artifact suppression, and then perform normalization of resolution and grayscale;

[0011] S12. Construct a thin plate spline mapping function:

[0012] where (x, y) are the coordinates of the point to be transformed in the source image, (x i , y i ) are the coordinates of the i-th feature point in the source image, U(r) = r 2 log(r) is the thin plate spline kernel function, ||·|| is the Euclidean distance;

[0013] S13. Calculate the deformation energy through the formula where d j is the position vector of the corresponding feature point in the target image, (x j , y j ) are the coordinates of the j-th feature point in the source image, λ is the regularization coefficient, and E bend (f) is the bending energy of the deformation function f;

[0014] S14. By minimizing the deformation energy, establish a system of linear equations to solve for a1, a2, a3, and ω i Solve for the mapping function f(x, y), and map all pixel points of the source image to the target image coordinate system through the thin plate spline mapping function and align them to obtain the standard image data.

[0015] In the aspects and any possible implementation manners described above, a further implementation manner is provided. The S2 specifically includes:

[0016] S21. Extract the multi-layer encoder features and decoder up-sampling features of the standard image through the U-Net architecture, and perform 1×1 convolution on each layer of features to map them to the same dimension;

[0017] S22. After fusing the mapped features, perform non-linear activation, and then calculate the attention map through the Sigmoid layer;

[0018] S23. Perform pixel-wise weighting on the multi-layer encoder feature machine based on the attention map, and fuse the weighted encoder features with the sampling features of the corresponding layer of the decoder;

[0019] S24. Transmit the fusion result to the decoder to restore the spatial resolution of the feature map, and map the restoration result to three categories: background, vertebral bone, and soft tissue;

[0020] S25. After converting the output of each pixel into a probability distribution belonging to each category through the Softmax activation function, obtain the segmentation result;

[0021] S26. Process the segmentation result through the Marching Cubes algorithm to obtain a mesh, and perform smoothing and repair on the mesh to obtain a three-dimensional geometric model.

[0022] For the aspects and any possible implementation manners described above, a further implementation manner is provided. Specifically, S3 includes:

[0023] S31: Process the three-dimensional geometric model of the vertebral body bone and soft tissue through the Marching Cubes algorithm to obtain a triangular mesh, while retaining the information of bone pores and cracks. Input the bone cement injection parameters and physical parameters. The bone cement injection parameters include: injection pressure, injection rate, initial viscosity, and hardening parameters. The physical parameters include: elastic modulus of bone, Poisson's ratio, cement density, and external force information.

[0024] S32: At the current time step, after the fluid-structure coupling finite element model performs solid solution and fluid solution, perform coupling iteration until convergence, and use the output simulation calculation result as the initial value of the next time step to iteratively simulate the subsequent bone cement injection process.

[0025] S33: Output the simulation results of the cement flow field and bone mechanics field including multiple time steps.

[0026] For the aspects and any possible implementation manners described above, a further implementation manner is provided. Specifically, S4 includes:

[0027] S41: Obtain the data of the cement flow field and bone mechanics field at multiple time steps {t1, t2,..., t N}, and process them into time series field data {Φ 1 , Φ 2 ,..., Φ N} in the same coordinate system.

[0028] S42: Use the time series field data as the first input branch of the multi-branch fusion network model, and extract time series features in ways such as temporal convolution, temporal graph neural network, or temporal attention.

[0029] S43: Use the segmented MRI image data after segmentation as the second input branch of the multi-branch fusion network model, and extract anatomical structure features through a convolutional neural network.

[0030] S44: Use the bone cement injection parameters as the third input branch of the multi-branch fusion network model, and extract embedded vector features using a fully connected layer.

[0031] S45: After fusing the extracted time series features, anatomical structure features, and embedded vector features in the intermediate layer of the multi-branch fusion network model, construct a spatio-temporal representation vector.

[0032] After the output head of the multi-branch fusion network model performs regression prediction classification on the spatio-temporal representation vector, the bone cement leakage result is obtained.

[0033] For the aspects and any possible implementation manners described above, a further implementation manner is provided. Specifically, S21 includes:

[0034] The U-Net architecture includes an encoder and a decoder with L layers. The encoder feature at the l-th layer of the encoder is F l e , and the sampled feature at the corresponding layer of the decoder is F l d ; perform 1×1 convolution mapping on the above features: Let φ(F l e ) = W e F l e , φ(F l d ) = W d F l d , where W e and W d are both learnable convolution kernels;

[0035] Specifically, S22 includes:

[0036] After fusing the features of φ(F l e ) and φ(F l d ), activate through ReLU to obtain the intermediate fusion feature F l e , F l fuse = RELU(φ(F l e ) + φ(F l d ) + b), and calculate the intermediate fusion feature through the Sigmoid layer to obtain the attention map α l = Sigmoid(W f * F l fuse + b f ), where b and b f are both biases;

[0037] Specifically, S23 includes:

[0038] Based on the attention map α l perform pixel-wise weighting on the encoder feature F l e : The weighted encoder feature The sampled feature F corresponding to the corresponding layer of the decoder l d After performing addition fusion, F is obtained l fuse2 , that is

[0039] The specific content of S24 includes:

[0040] Input the fusion result F l fuse2 into the upsampling module corresponding to the l-th layer of the decoder. After performing bilinear interpolation, it is gradually restored to the same spatial size as the original image; starting from the bottom layer, upsampling and merging are performed in reverse until the original resolution is restored, and then a feature map matching the resolution of the input image is output; use a 1×1 convolution in the last layer to map the number of channels to the number of categories. The number of categories is three, corresponding to the background, vertebral bone, and soft tissue respectively, and the result vector Z(x,y) corresponding to the three categories is output;

[0041] The specific content of S25 includes:

[0042] Perform Softmax on Z(x,y) in the channel dimension to obtain the probability distribution of each category:

[0043] c = 0, 1, 2, p0(x,y) + p1(x,y) + p2(x,y) = 1; find

[0044] the channel with the maximum probability for each pixel, and construct a mapping label as the segmentation result;

[0045] S26. Stack the segmentation results to form three-dimensional volume data, and select the mask corresponding to the target structure category; use the Marching Cubes algorithm to traverse the three-dimensional voxels and judge the vertex states of the voxel cubes according to the threshold, perform interpolation and topological look-up table on the detected isosurfaces, and splice and generate triangular meshes; perform bilateral filtering update on the vertices of the triangular meshes; remove the fragments in the mesh based on connected component analysis, fill the holes or openings, and verify the integrity of the mesh topology; output a smooth and closed three-dimensional geometric model.

[0046] For the aspects and any possible implementation manners described above, a further implementation manner is provided. The specific content of S32 includes:

[0047] S321. Initialize the interface pressure;

[0048] After receiving the input data, the fluid-structure coupling finite element model uses the interface pressure obtained in the previous time step as the normal load on the solid surface, and assembles the solid finite element equation:

[0049] After solving the solid finite element equations by the nonlinear discrete method, new solid displacements are output, and the stress field in the bone is updated. The solid interface displacements are transferred to the fluid domain to update the boundary geometry of the mesh;

[0050] S323. Read the new coordinates after the boundary geometry is updated, reconstruct the fluid mesh, and then assemble the fluid equation: A f (v, p)x = b f (u), solve the fluid equation to obtain the new fluid velocity and interface pressure, and transfer the interface pressure to the solid equation for the next iteration;

[0051] S324. Calculate the interface residual If ||Δσ|| is lower than the set tolerance, the coupled solution for this time step converges, go to the next time step, and enter S325; if ||Δσ|| is not lower than the set tolerance, the coupled solution for this time step does not converge, return to S322;

[0052] S325. Use the new fluid velocity, interface pressure, and solid displacements solved in the above steps as the initial values for the next time step, and iteratively simulate the subsequent bone cement injection process;

[0053] where, M s is the mass matrix, C s is the damping matrix, K s is the global stiffness matrix, u is the solid displacement, and are the first-order and second-order time derivatives of the solid displacement respectively, F ext is the external force vector, F fluid (p) is the load vector for the fluid pressure transferred to the solid interface, A f (v, p)x represents the fluid discrete system, A f represents that the system coefficient matrix depends on the current solution (v, p), x represents the fluid unknowns, b f (u) represents the right-hand vector of the fluid system.

[0054] For the aspects and any possible implementation manners as described above, a further implementation manner is provided. Specifically, S45 includes: Stack the time series feature z time , the anatomical structure feature z img and the embedded vector feature z param in the vector dimension to form the comprehensive feature z fuse = Concat(z time , z img , z param ), input z fuse into the fuse layer and map it through MLP to obtain the spatio-temporal representation vector z final = MLPfuse (z fuse ).

[0055] For the aspects and any possible implementation manners described above, a further implementation manner is provided. Specifically, S46 includes:

[0056] Input z final into the FC layer and ReLU layer of the multi-branch fusion network for processing and then map it to the target dimension to obtain z out , and use the output head of the multi-branch fusion network to perform Softma processing on z out to obtain the leakage type;

[0057] The multi-branch fusion network is trained in the following way:

[0058] Input the training samples into the model to be trained in the forms of three branches: a time series branch, an image branch, and a parameter branch. After being processed by the LSTM layer, CNN layer, and MLP layer respectively, obtain the time series feature z time , the anatomical structure feature z img and the embedded vector feature z param , then fuse them in the middle layer to generate z fuse , z fuse is input into the fuse layer and mapped through the MLP to obtain the spatio-temporal representation vector z final , input z final into the FC layer and ReLU layer for processing and then map it to the target dimension to obtain z out , and perform Softmax processing on z out by the output head to obtain the leakage type;

[0059] The loss function is: B is the total number of training samples, and C is the total number of categories; y i,c is the true label, indicating whether the sample i belongs to the category c; is the probability predicted by the model output, representing the prediction probability that the model predicts that the sample i belongs to the category c. Continuously train until L CE is lower than the preset threshold, then the final multi-branch fusion network model is obtained.

[0060] In a second aspect, an embodiment of the present invention provides a vertebral body bone cement leakage warning system based on magnetic resonance imaging using the method according to any one of claims 1-9. The system includes:

[0061] A preprocessing module, configured to obtain the MRI image data of the vertebral body, and preprocess the MRI image data to obtain standard image data;

[0062] A segmentation module, configured to segment standard image data through Attention U-Net to extract vertebral bone and soft tissue parts, and generate a three-dimensional geometric model;

[0063] A calculation module, configured to construct a fluid-structure coupling finite element model based on bone cement injection parameters and the three-dimensional geometric model for simulation calculation;

[0064] A processing module, configured to input the simulation results into a multi-branch fusion network model for processing, and then output the vertebral bone cement leakage result.

[0065] One of the above technical solutions has the following beneficial effects:

[0066] In the method of the embodiment of the present invention, a method and system for early warning of vertebral bone cement leakage based on magnetic resonance imaging are proposed. The method first obtains MRI image data of the vertebra, and after preprocessing the MRI image data, standard image data is obtained; then the standard image data is segmented through Attention U-Net to extract vertebral bone and soft tissue parts, and a three-dimensional geometric model is generated; subsequently, based on bone cement injection parameters and the three-dimensional geometric model, a fluid-structure coupling finite element model is constructed for simulation calculation; finally, the simulation results are input into a multi-branch fusion network model for processing, and the vertebral bone cement leakage result is output. The method of the present invention is based on multi-moment cement flow field and bone mechanics field data, models the diffusion of bone cement in the vertebra and the evolution of bone stress in the time dimension, captures the dynamic changes of fluid-structure interaction over time, and can more accurately discover the high-risk leakage periods and characteristics during the injection process compared with single-moment or static analysis. The multi-branch model simultaneously integrates the fluid-structure finite element time series field, MRI images and injection parameters to provide richer and complementary inputs for leakage prediction. In summary, the technical solution of the present invention is based on a multi-branch fusion deep network, multi-moment finite element simulation and medical images, realizing a more comprehensive, accurate and scalable prediction analysis of bone cement leakage, and having significant beneficial effects in clinical surgical planning and risk prevention.

Description of the Drawings

[0067] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0068] Figure 1 It is a schematic flowchart of a method for early warning of vertebral bone cement leakage based on magnetic resonance imaging provided by an embodiment of the present invention;

[0069] Figure 2Schematic block diagram of a vertebral bone cement leakage warning system based on magnetic resonance imaging provided by an embodiment of the present invention;

[0070] Figure 3 Hardware structure schematic diagram of a vertebral bone cement leakage warning system based on magnetic resonance imaging provided by an embodiment of the present invention.

Specific implementation manners

[0071] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0072] Please refer to Figure 1 , which is a schematic flowchart of a vertebral bone cement leakage warning method based on magnetic resonance imaging provided by an embodiment of the present invention. As shown in the figure, the method includes the following steps:

[0073] S1. Obtain MRI image data of the vertebral body, and obtain standard image data after preprocessing the MRI image data;

[0074] S2. Perform segmentation processing on the standard image data through Attention U-Net to extract the vertebral bone and soft tissue parts, and generate a three-dimensional geometric model;

[0075] S3. Based on the bone cement injection parameters and the three-dimensional geometric model, construct a fluid-structure coupling finite element model for simulation calculation;

[0076] S4. After inputting the simulation results into a multi-branch fusion network model for processing, output the vertebral bone cement leakage results.

[0077] Specifically, the S1 includes:

[0078] S11. After denoising and artifact suppression using the MRI image data as the source image, perform normalization of the resolution and grayscale;

[0079] S12. Construct a thin plate spline mapping function:

[0080] where (x, y) is the coordinate of the point to be transformed in the source image, (x i , y i ) is the coordinate of the i-th feature point in the source image, U(r) = r 2 log(r) is the thin plate spline kernel function, and ||·|| is the Euclidean distance;

[0081] S13. Calculate the deformation energy through a formula where d j is the position vector of the corresponding feature point of the target image, (x j , y j ) is the coordinate of the j-th feature point in the source image, λ is the regularization coefficient, and E bend (f) is the bending energy of the deformation function f;

[0082] S14. By minimizing the deformation energy, establish a linear equation system to solve for a1, a2, a3, and ω i Solve for the mapping function f(x, y), and map all pixel points of the source image to the target image coordinate system through the thin plate spline mapping function for alignment to obtain the standard image data.

[0083] Specifically, S2 specifically includes:

[0084] S21. Extract the multi-layer encoder features and decoder upsampling features of the standard image through the U-Net architecture, and perform 1×1 convolution on each layer of features to map them to the same dimension;

[0085] S22. After fusing the mapped features, perform non-linear activation, and then calculate the attention map through the Sigmoid layer;

[0086] S23. Perform pixel-by-pixel weighting on the multi-layer encoder features based on the attention map, and fuse the weighted encoder features with the sampling features of the corresponding layers of the decoder;

[0087] S24. Transmit the fusion result to the decoder to restore the spatial resolution of the feature map, and map the restored result to three categories: background, vertebral bone, and soft tissue;

[0088] S25. After converting the output of each pixel into a probability distribution belonging to each category through the Softmax activation function, obtain the segmentation result;

[0089] S26. Process the segmentation result through the Marching Cubes algorithm to obtain a mesh, and after smoothing and repairing the mesh, obtain a three-dimensional geometric model.

[0090] Specifically, S3 specifically includes:

[0091] S31. Process the three-dimensional geometric models of vertebral bone and soft tissue through the Marching Cubes algorithm to obtain triangular meshes, while retaining the information of bone internal pores and cracks; input the bone cement injection parameters and physical parameters, where the bone cement injection parameters include: injection pressure, injection rate, initial viscosity, and hardening parameters, and the physical parameters include: elastic modulus of bone, Poisson's ratio, cement density, and external force information;

[0092] S32. At the current time step, after the fluid-structure coupling finite element model performs solid and fluid solutions, coupling iteration is carried out until convergence, and the output simulation calculation results are used as the initial values for the next time step, and the subsequent bone cement injection process is iteratively simulated;

[0093] S33. Output the simulation results of the cement flow field and bone mechanics field including multiple time steps.

[0094] Specifically, the S4 specifically includes:

[0095] S41. Obtain the cement flow field and bone mechanics field data at multiple time steps {t1, t2,..., t N}, and process them into time series field data {Φ 1 , Φ 2 ,..., Φ N} in the same coordinate system;

[0096] S42. Take the time series field data as the first input branch of the multi-branch fusion network model, and extract time series features in the way of temporal convolution, temporal graph neural network or temporal attention, etc.;

[0097] S43. Take the segmented MRI image data after segmentation as the second input branch of the multi-branch fusion network model, and extract anatomical structure features through a convolutional neural network;

[0098] S44. Take the bone cement injection parameters as the third input branch of the multi-branch fusion network model, and extract embedded vector features using a fully connected layer;

[0099] S45. After fusing the extracted time series features, anatomical structure features and embedded vector features in the middle layer of the multi-branch fusion network model, construct a spatio-temporal representation vector;

[0100] S46. After the output head of the multi-branch fusion network model performs regression prediction classification on the spatio-temporal representation vector, obtain the bone cement leakage result.

[0101] Specifically, the S21 specifically includes:

[0102] The U-Net architecture includes an encoder and a decoder with L layers. The encoder feature at the l-th layer of the encoder is F l e , and the sampling feature at the corresponding layer of the decoder is F l d ; perform 1×1 convolutional mapping on the above features: Let φ(F l e ) = W e F le , φ(F l d ) = W d F l d , where W e and W d are both learnable convolutional kernels;

[0103] Specifically, S22 includes:

[0104] Perform feature fusion on φ(F l e ) and φ(F l d ), and then obtain the intermediate fusion feature F l e through ReLU activation, l fuse F l e = RELU(φ(F l d ) + φ(F l f ) + b), and calculate the attention map α l fuse f f l ) by passing the intermediate fusion feature through the Sigmoid layer, where b and b l are both biases;

[0105] Specifically, S23 includes:

[0106] Perform pixel-by-pixel weighting on the encoder feature F l l e based on the attention map α: Additively fuse the weighted encoder feature with the sampled feature F l d of the corresponding layer of the decoder to obtain F l fuse2 , that is,

[0107] Specifically, S24 includes:

[0108] The fusion result F l fuse2Input it into the upsampling module corresponding to the l-th layer of the decoder. After performing bilinear interpolation, it is gradually restored to the same spatial size as the original image. Starting from the bottom layer, perform upsampling and merging in reverse until the original resolution is restored, and then output a feature map that matches the resolution of the input image. Use a 1×1 convolution in the last layer to map the number of channels to the number of categories. The number of categories is three, corresponding to the background, vertebral bone, and soft tissue respectively, and output the result vector Z(x, y) corresponding to the three categories.

[0109] Specifically, S25 includes:

[0110] Perform Softmax on Z(x, y) in the channel dimension to obtain the probability distribution of each category:

[0111] c = 0, 1, 2, p0(x, y) + p1(x, y) + p2(x, y) = 1; Find

[0112] the channel with the maximum probability for each pixel, and construct a mapping label as the segmentation result;

[0113] S26. Stack the segmentation results to form three-dimensional volume data, and select the mask corresponding to the target structure category. Use the Marching Cubes algorithm to traverse the three-dimensional voxels and judge the vertex states of the voxel cubes according to the threshold. Interpolate and perform topological look-up on the detected isosurfaces, and splice and generate a triangular mesh. Perform bilateral filtering update on the vertices of the triangular mesh. Remove the fragments in the mesh based on connected component analysis, fill the holes or openings, and verify the integrity of the mesh topology. Output a smooth and closed three-dimensional geometric model.

[0114] Specifically, S32 includes:

[0115] S321. Initialize the interface pressure;

[0116] After receiving the input data, the fluid-structure coupling finite element model uses the interface pressure obtained in the previous time step as the normal load on the solid surface, and assembles the solid finite element equation:

[0117] Solve the solid finite element equation through the nonlinear discretization method, output the new solid displacement, update the stress field in the bone, and transfer the solid interface displacement to the fluid domain to update the boundary geometry of the mesh;

[0118] S323. Read the new coordinates after the boundary geometry update, reconstruct the fluid mesh, and then assemble the fluid equation: A f (v, p)x = b f (u), solve the fluid equation to obtain the new fluid velocity and interface pressure, and transfer the interface pressure to the solid equation for the next iteration;

[0119] S324. Calculate the interface residuals If ||Δσ|| is lower than the set tolerance, the coupled solution for this time step converges. Proceed to the next time step and enter S325. If ||Δσ|| is not lower than the set tolerance, the coupled solution for this time step does not converge. Return to S322;

[0120] S325. Use the newly obtained fluid velocity, interface pressure, and solid displacement from the above steps as the initial values for the next time step, and iteratively simulate the subsequent bone cement injection process;

[0121] where M s is the mass matrix, C s is the damping matrix, K s is the global stiffness matrix, u is the solid displacement, and are the first and second time derivatives of the solid displacement respectively, F ext is the external force vector, F fluid (p) is the load vector transmitted by the fluid pressure to the solid interface, A f (v,p)x represents the fluid discrete system, A f represents that the system coefficient matrix depends on the current solution (v,p), x represents the fluid unknowns, b f (u) represents the right - hand side vector of the fluid system.

[0122] Specifically, S45 specifically includes: Stack the time - series feature z time , the anatomical structure feature z img and the embedding vector feature z param in the vector dimension to form the comprehensive feature z fuse =Concat(z time ,z img ,z param ). Input z fuse into the fuse layer and map it through MLP to obtain the spatio - temporal representation vector z final =MLP fuse (z fuse ).

[0123] Specifically, S46 specifically includes:

[0124] Input z final into the FC layer and ReLU layer of the multi - branch fusion network for processing and then map it to the target dimension to obtain z out . Use the output head of the multi - branch fusion network to perform Softma processing on z out to obtain the leakage type;

[0125] The multi - branch fusion network is trained in the following way:

[0126] The training samples are input into the model to be trained in the form of three branches: a time series branch, an image branch, and a parameter branch. After being processed by an LSTM layer, a CNN layer, and an MLP layer respectively, the time series feature z time , the anatomical structure feature z img and the embedded vector feature z param are obtained. Then, they are fused in the middle layer to generate z fuse , z fuse is input into the fuse layer and mapped through an MLP to obtain the spatio-temporal representation vector z final . z final is input into the FC layer and the ReLU layer for processing and then mapped to the target dimension to obtain z out . The output head performs Softmax processing on z out to obtain the leakage type;

[0127] The loss function is: B is the total number of training samples, and C is the total number of categories; y i,c is the true label, indicating whether the sample i belongs to the category c; is the probability predicted by the model output, representing the probability that the model predicts that the sample i belongs to the category c. Keep training until L CE is lower than the preset threshold, then the final multi-branch fusion network model is obtained.

[0128] Through the above steps, the present invention achieves the following technical effects:

[0129] In the method of the embodiment of the present invention, a vertebral bone cement leakage warning method and system based on magnetic resonance imaging are proposed. The method first obtains the MRI image data of the vertebra, and obtains the standard image data after preprocessing the MRI image data; then performs segmentation processing on the standard image data through Attention U-Net to extract the vertebral bone and soft tissue parts, and generates a three-dimensional geometric model; subsequently, based on the bone cement injection parameters and the three-dimensional geometric model, a fluid-structure coupling finite element model is constructed for simulation calculation; finally, the simulation result is input into the multi-branch fusion network model for processing, and the vertebral bone cement leakage result is output. The method of the present invention is based on the multi-moment cement flow field and bone mechanics field data, models the diffusion of bone cement in the vertebra and the evolution of bone stress in the time dimension, captures the dynamic changes of fluid-structure interaction over time, and can more accurately discover the high-risk leakage periods and characteristics during the injection process compared with single-moment or static analysis. The multi-branch model simultaneously integrates the fluid-structure finite element time series field, MRI images and injection parameters to provide richer and complementary inputs for leakage prediction. In summary, the technical solution of the present invention is based on a multi-branch fusion deep network, multi-moment finite element simulation and medical images, realizing a more comprehensive, accurate and scalable prediction analysis of bone cement leakage, and having significant beneficial effects in clinical surgical planning and risk prevention.

[0130] The embodiment of the present invention further provides an apparatus embodiment for implementing each step and method in the above method embodiment.

[0131] Please refer to Figure 2 , which is Figure 2 A schematic block diagram of a vertebral bone cement leakage warning system based on magnetic resonance imaging provided by an embodiment of the present invention. The system includes:

[0132] A preprocessing module 210, configured to obtain the MRI image data of the vertebra, and obtain the standard image data after preprocessing the MRI image data;

[0133] A segmentation module 220, configured to perform segmentation processing on the standard image data through Attention U-Net to extract the vertebral bone and soft tissue parts, and generate a three-dimensional geometric model;

[0134] A calculation module 230, configured to construct a fluid-structure coupling finite element model for simulation calculation based on the bone cement injection parameters and the three-dimensional geometric model;

[0135] A processing module 240, configured to input the simulation result into the multi-branch fusion network model for processing, and output the vertebral bone cement leakage result.

[0136] Since each unit module in this embodiment can execute Figure 1For the method shown, for parts not described in detail in this embodiment, reference may be made to the relevant description of Figure 1 .

[0137] Please refer to Figure 3 , which is a schematic diagram of the hardware structure of a vertebral bone cement leakage warning system based on magnetic resonance imaging provided by an embodiment of the present invention. The data prediction device includes at least one processor and a memory. The at least one processor is coupled to the memory and is configured to read and execute instructions in the memory to perform the method for warning of vertebral bone cement leakage based on magnetic resonance imaging provided by the embodiment of the present invention.

[0138] In a third aspect, an embodiment of the present invention provides a computer-readable medium. The computer-readable medium stores program code. When the computer program code runs on a computer, the computer is caused to execute the method for warning of vertebral bone cement leakage based on magnetic resonance imaging provided by the embodiment of the present invention.

[0139] At the hardware level, the device may include a processor, and optionally also an internal bus, a network interface, and a memory. Among them, the memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory, etc. Of course, the device may also include other hardware required for other services.

[0140] The processor, network interface, and memory may be interconnected through an internal bus. The internal bus may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc.

[0141] The memory is used to store a program. Specifically, the program may include program code, and the program code includes computer operation instructions. The memory may include a memory and a non-volatile memory, and provide instructions and data to the processor.

[0142] The steps of the method disclosed in the embodiments of the present invention can be directly embodied as being executed and completed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory, and the processor reads the information in the memory and combines its hardware to complete the steps of the above method.

[0143] The systems, devices, modules or units illustrated in the above embodiments can be specifically implemented by a computer chip or an entity, or by a product with certain functions. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0144] For the convenience of description, when describing the above devices, they are divided into various units or modules according to functions and described separately. Of course, when implementing the present invention, the functions of the various units or modules can be implemented in the same or multiple software and / or hardware.

[0145] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program codes.

[0146] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of the present invention. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams, can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for realizing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0147] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to operate in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means that implement the functions specified in the process Figure 1 one process or multiple processes and / or blocks Figure 1 specified in one block or multiple blocks.

[0148] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, such that a series of operational steps are performed on the computer or other programmable apparatus to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable apparatus provide steps for implementing the functions specified in the process Figure 1 one process or multiple processes and / or blocks Figure 1 specified in one block or multiple blocks.

[0149] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0150] The memory may include non-permanent memory in the form of computer-readable media, random access memory (RAM) and / or non-volatile memory such as read only memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.

[0151] Computer-readable media includes both permanent and non-permanent, removable and non-removable media implemented by any method or technology for storage of information. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technologies, compact disc read only memory (CD-ROM), digital versatile discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media such as modulated data signals and carrier waves.

[0152] It should also be noted that the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, commodity or device comprising a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, commodity or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, commodity or device comprising said element.

[0153] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, system or computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) containing computer-usable program code.

[0154] The present invention can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention can also be practiced in a distributed computing environment where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0155] Each embodiment in the present invention is described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the description of the method embodiment.

[0156] The above description is only for the embodiments of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the scope of the claims of the present invention.

Claims

1. A method for warning of vertebral bone cement leakage based on magnetic resonance imaging, characterized in that, The method includes: S1. Obtain the MRI image data of the vertebral body, and perform preprocessing on the MRI image data to obtain standard image data; S2. Perform segmentation processing on the standard image data through Attention U-Net to extract the vertebral bone and soft tissue parts, and generate a three-dimensional geometric model; S3. Based on the bone cement injection parameters and the three-dimensional geometric model, construct a fluid-structure coupling finite element model for simulation calculation; S4. Input the simulation results into a multi-branch fusion network model for processing, and output the vertebral bone cement leakage results.

2. The method for warning of vertebral bone cement leakage in magnetic resonance imaging according to claim 1, wherein, The S1 includes: S11. Take the MRI image data as the source image, perform denoising and artifact suppression, and then perform normalization of resolution and grayscale; S12. Construct a thin plate spline mapping function: where (x, y) are the coordinates of the point to be transformed in the source image, and (x i , y i ) are the coordinates of the i-th feature point in the source image, U(r) = r 2 log(r) is the thin plate spline kernel function, and ||·|| is the Euclidean distance; S13. Calculate the deformation energy through formulas where d j is the position vector of the corresponding feature point of the target image, (x j , y j ) is the coordinate of the j-th feature point in the source image, λ is the regularization coefficient, and E bend (f) is the bending energy of the deformation function f; S14. By minimizing the deformation energy, establish a system of linear equations to solve for a1, a2, a3, and ω i Solve for the mapping function f(x, y), and map all pixel points of the source image to the target image coordinate system through the thin plate spline mapping function for alignment to obtain the standard image data.

3. The method for warning of vertebral bone cement leakage based on magnetic resonance imaging according to claim 2, wherein The S2 specifically includes: S21. Extract the multi-layer encoder features and decoder upsampling features of the standard image through the U-Net architecture, and perform 1×1 convolution on each layer of features to map them to the same dimension; S22. After fusing the mapped features, perform non-linear activation, and then calculate the attention map through the Sigmoid layer; S23. Perform pixel-by-pixel weighting on the multi-layer encoder feature machine based on the attention map, and fuse the weighted encoder features with the sampling features of the corresponding layer of the decoder; S24. Transmit the fusion result to the decoder to restore the spatial resolution of the feature map, and map the restored result into three categories: background, vertebral bone, and soft tissue; S25. Obtain the segmentation result by converting the output of each pixel into a probability distribution belonging to each category through the Softmax activation function; S26. Process the segmentation result through the Marching Cubes algorithm to obtain a mesh, and perform smoothing and repair on the mesh to obtain a three-dimensional geometric model.

4. The method for warning of vertebral bone cement leakage based on magnetic resonance imaging according to claim 3, wherein The S3 specifically includes: S31. Process the three-dimensional geometric model of the vertebral bone and soft tissue through the Marching Cubes algorithm to obtain a triangular mesh, and at the same time retain the bone internal pore and crack information; input the bone cement injection parameters and physical parameters, where the bone cement injection parameters include: injection pressure, injection rate, initial viscosity, and hardening parameters, and the physical parameters include: elastic modulus of bone, Poisson's ratio, cement density, and external force information; S32. At the current time step, after the fluid-structure coupling finite element model performs solid solution and fluid solution, perform coupling iteration until convergence, and use the output simulation calculation result as the initial value of the next time step to iteratively simulate the subsequent bone cement injection process; S33. Output the simulation results including the cement flow field and bone mechanics field of multiple time steps.

5. The method for warning of vertebral bone cement leakage based on magnetic resonance imaging according to claim 4, wherein The S4 specifically includes: S41. Obtain the cement flow field and bone mechanics field data at multiple time steps {t1, t2,..., t N}, and process them into time series field data {Φ 1 , Φ 2 ,..., Φ N} in the same coordinate system; S42. Take the time series field data as the first input branch of the multi-branch fusion network model, and extract time series features in ways such as temporal convolution, temporal graph neural network, or temporal attention; S43. Take the segmented MRI image data after segmentation processing as the second input branch of the multi-branch fusion network model, and extract anatomical structure features through a convolutional neural network; S44. Take the bone cement injection parameters as the third input branch of the multi-branch fusion network model, and use a fully connected layer to extract the embedded vector features; S45. After fusing the extracted time series features, anatomical structure features, and embedded vector features in the middle layer of the multi-branch fusion network model, construct a spatio-temporal representation vector; S46. After the output head of the multi-branch fusion network model performs regression prediction classification on the spatio-temporal representation vector, obtain the bone cement leakage result.

6. The method for warning of vertebral bone cement leakage based on magnetic resonance imaging according to claim 3, wherein The S21 specifically includes: The U-Net architecture includes an encoder and a decoder with L layers. The encoder feature at the l-th layer of the encoder is F l e , and the sampled feature at the corresponding layer of the decoder is F l d ; perform 1×1 convolutional mapping on the above features: Let φ(F l e ) = W e F l e , φ(F l d ) = W d F l d , where both W e and W d are learnable convolutional kernels; The S22 specifically includes: For φ(F l e ), and φ(F l d ), after feature fusion, the intermediate fusion feature F l e is obtained through ReLU activation, F l fuse = ReLU(φ(F l e ) + φ(F l d ) + b). The intermediate fusion feature is calculated through the Sigmoid layer to obtain the attention map α l = Sigmoid(W f * F l fuse + b f ), where b and b f are both biases; The S23 specifically includes: Based on the attention map α l Perform pixel-by-pixel weighting on the encoder feature F l e : The weighted encoder feature Is additively fused with the sampled feature F of the corresponding layer of the decoder l d To obtain F l fuse2 , that is The S24 specifically includes: Input the fusion result F l fuse2 into the upsampling module corresponding to the l-th layer of the decoder. After performing bilinear interpolation, restore it layer by layer to the same spatial size as the original image. Perform upsampling and merging in reverse from the bottom layer until the original resolution is restored, and then output a feature map that matches the resolution of the input image. Use a 1×1 convolution in the last layer to map the number of channels to the number of classes. The number of classes is three, corresponding to the background, vertebral bone, and soft tissue respectively, and output the result vector Z(x, y) corresponding to the three classes; The S25 specifically includes: Perform Softmax on Z(x, y) in the channel dimension to obtain the probability distribution of each class: c = 0, 1, 2, p0(x, y) + p1(x, y) + p2(x, y) = 1; Search Find the channel with the maximum probability for each pixel, and construct a mapping label as the segmentation result; S26. Stack the segmentation results to form three-dimensional volume data, and select the mask corresponding to the target structure category; Use the Marching Cubes algorithm to traverse the three-dimensional voxels and judge the vertex state of the voxel cube according to the threshold, perform interpolation and topological look-up table on the detected isosurface, and splice to generate a triangular mesh; Perform bilateral filtering update on the triangular mesh vertices; Remove the fragments in the mesh based on connected component analysis, fill the holes or openings, and verify the integrity of the mesh topology; Output a smooth and closed three-dimensional geometric model.

7. The method for warning of vertebral bone cement leakage based on magnetic resonance imaging according to claim 4, wherein The S32 specifically includes: S321. Initialize the interface pressure; S322. After receiving the input data, the fluid-structure coupling finite element model takes the interface pressure obtained in the previous time step as the normal load on the solid surface, and assembles the solid finite element equation: After solving the solid finite element equation by the non-linear discrete method, new solid displacements are output, and the stress field within the bone is updated. The solid interface displacements are transmitted to the fluid domain to update the boundary geometry of the mesh. S323. After reading the new coordinates updated by the boundary geometry and reconstructing the fluid mesh, assemble the fluid equation: A f (v, p)x = b f (u), solve the fluid equation to obtain the new fluid velocity and interface pressure, and transfer the interface pressure to the solid equation for the next iteration; S324. Calculate the interface residuals If ||Δσ|| is lower than the set tolerance, the coupled solution for this time step converges. Proceed to the next time step and go to S325. If ||Δσ|| is not lower than the set tolerance, the coupled solution for this time step does not converge. Return to S322. S325. Take the newly obtained fluid velocity, interface pressure, and solid displacement solved in the above steps as the initial values for the next time step, and iteratively simulate the subsequent bone cement injection process; Among them, M s is the mass matrix, C s is the damping matrix, K s is the global stiffness matrix, u is the solid displacement, and ü are the first and second time derivatives of the solid displacement respectively, F ext is the external force vector, F fluid (p) is the load vector transmitted by the fluid pressure to the solid interface, A f (v,p)x represents the fluid discrete system, A f represents that the system coefficient matrix depends on the current solution (v,p), x represents the fluid unknowns, b f (u) represents the right-hand side vector of the fluid system.

8. The method for warning of vertebral bone cement leakage based on magnetic resonance imaging according to claim 5, wherein, The S45 specifically includes: stacking the time series feature z time , the anatomical structure feature z img and the embedded vector feature z param on the vector dimension to form a comprehensive feature z fuse = Concat(z time , z img , z param ), and inputting z fuse into the fuse layer to obtain the spatio-temporal representation vector z final = MLP fuse (z fuse ).

9. The method for warning of vertebral bone cement leakage based on magnetic resonance imaging according to claim 8, characterized in that, The S46 specifically includes: Input z final into the FC layer and ReLU layer of the multi-branch fusion network for processing, and then map it to the target dimension to obtain z out , and use the output head of the multi-branch fusion network to process z out through Softmax to obtain the leakage type; The multi-branch fusion network is trained in the following way: The training samples are input into the model to be trained in the form of three branches: a time series branch, an image branch, and a parameter branch. After being processed by an LSTM layer, a CNN layer, and an MLP layer respectively, the time series feature z time , the anatomical structure feature z img , and the embedded vector feature z param are obtained. Then, they are fused in the middle layer to generate z fuse . z fuse is input into the fuse layer and mapped through an MLP to obtain the spatio-temporal representation vector z final . z final is input into the FC layer and the ReLU layer for processing and then mapped to the target dimension to obtain z out . The output head performs Softmax processing on z out to obtain the leakage type; The loss function is as follows: B is the total number of training samples, and C is the total number of categories; y i,c is the true label, indicating whether sample i belongs to category c; is the probability predicted by the model, representing the prediction probability of the model that sample i belongs to category c. Keep training until L CE is lower than the preset threshold, then the final multi-branch fusion network model is obtained.

10. A magnetic resonance imaging-based vertebral bone cement leakage warning system using the method according to any one of claims 1-9, characterized in that, The system includes: A preprocessing module, which is used to obtain the MRI image data of the vertebral body, and preprocess the MRI image data to obtain standard image data; A segmentation module, which is used to perform segmentation processing on the standard image data through Attention U-Net to extract the vertebral bone and soft tissue parts, and generate a three-dimensional geometric model; A calculation module, which is used to construct a fluid-structure coupling finite element model based on the bone cement injection parameters and the three-dimensional geometric model for simulation calculation; A processing module, which is used to input the simulation results into the multi-branch fusion network model for processing, and output the vertebral bone cement leakage result.

Images