Bone biomechanical modeling method and system based on multi-modal image

Abstract

The invention relates to the technical field of bone biomechanical simulation, and discloses a bone biomechanical modeling method and system based on a multi-modal image, and the method comprises the steps: S1, collecting a CT image and an MRI image of a to-be-detected bone structure, and generating a multi-modal image; s2, adopting a depth variation inference algorithm to extract common features of the multi-modal images, adopting a generative adversarial network algorithm to optimize the common features, and performing information fusion on the multi-modal images to generate comprehensive image data; s3, reconstructing a three-dimensional bone structure model based on the comprehensive image data; s4, simulating mechanical property parameters of the three-dimensional bone structure model under different external loads and physiological states by adopting a nonlinear finite element method based on the three-dimensional bone structure model, and outputting mechanical properties of bone tissues; and S5, based on the three-dimensional bone structure model and the mechanical characteristics of the bone tissue, generating a personalized bone biomechanical model by using a depth variation inference algorithm. According to the method, an accurate personalized bone biomechanical modeling scheme can be provided.

Description

Technical Field

[0001] The present invention relates to the technical field of bone biomechanics simulation, and in particular to a bone biomechanics modeling method and system based on multimodal images. Background Art

[0002] The assessment of fracture risk or the formulation of clinical treatment plans after fracture and the treatment effect mainly depend on the biomechanical properties of bone structure. Due to the small number of human bone samples available for experiments, it is unrealistic or limited to use in vitro experimental methods to study the biomechanical properties of bone structure. The biomechanical model based on medical imaging can simulate various complex stress conditions of bone structure and can repeat the experiment indefinitely. Therefore, it has become a simple and quick method to analyze bone mechanical properties.

[0003] Traditional bone structure reconstruction methods mostly rely on two-dimensional imaging data or single modality images for bone modeling, lacking accurate restoration of the fine features of bone structure. Traditional mechanical property simulation methods are usually based on idealized bone tissue properties and fail to fully consider individual differences, resulting in large deviations in the generated bone biomechanical models in clinical applications. The existing three-dimensional modeling and mechanical property simulation processes lack the support of advanced technologies such as deep learning, and cannot dynamically optimize and adjust the bone structure model, thus affecting the accuracy and reliability of the simulation results. The existing technology faces the following main problems.

[0004] (1) Insufficient accuracy: Existing 3D bone structure reconstruction methods fail to fully extract subtle bone structure features, resulting in insufficient spatial resolution and detail of the bone model.

[0005] (2) Lack of personalization: Existing mechanical property simulations often ignore individual differences, making it difficult to generate a personalized bone biomechanical model that conforms to the physiological characteristics of each patient.

[0006] (3) Lack of effective nonlinear simulation: Most existing mechanical property simulation methods rely on linear assumptions and fail to accurately simulate the nonlinear deformation and stress distribution of bone structures under complex physiological conditions. Summary of the invention

[0007] In order to solve the above technical problems, the purpose of the present invention is to provide a bone biomechanical modeling method based on multimodal images, based on multimodal image reconstruction, deep learning algorithm and nonlinear finite element analysis method, to provide an accurate personalized bone biomechanical modeling solution, and provide more efficient support for clinical applications.

[0008] In order to solve the above technical problems, the present invention proposes the following technical solutions: The present application relates to a bone biomechanical modeling method based on multimodal images, comprising: S1: Acquire CT images and MRI images of the bone structure to be tested and generate multimodal images; S2: extracting common features of the multimodal images using a deep variational inference algorithm, optimizing the common features using a generative adversarial network algorithm, performing information fusion on the multimodal images, and generating comprehensive image data; S3: reconstructing a three-dimensional bone structure model based on the comprehensive image data; S4: Based on the three-dimensional bone structure model, a nonlinear finite element method is used to simulate the mechanical characteristic parameters of the three-dimensional bone structure model under different external loads and physiological states, and the mechanical characteristics of the bone tissue are output; S5: Based on the three-dimensional bone structure model and the mechanical properties of bone tissue, a personalized bone biomechanical model is generated using a deep variational inference algorithm.

[0009] In some embodiments of the present application, S3 specifically includes: S31: Based on the comprehensive image data, a preliminary three-dimensional bone structure model is generated using a generative adversarial network algorithm; S32: Based on the preliminary three-dimensional bone structure model, optimizing the details of a local area of ​​the bone structure to obtain a three-dimensional bone structure model after the details are optimized; S33: based on the detail-optimized three-dimensional bone structure model, optimizing the spatial resolution thereof to obtain an enhanced three-dimensional bone structure model; S34: comparing and correcting the real clinical data acquired in advance with the enhanced three-dimensional bone structure model, and performing accuracy correction on key areas of the enhanced three-dimensional bone structure model; S35: Output the accuracy-corrected three-dimensional bone structure model.

[0010] In some embodiments of the present application, S3 further includes: S31 ′: pre-process the integrated image data from S2 to generate pre-processed integrated image data for use in S31 .

[0011] In some embodiments of the present application, S31 specifically includes: S311: Initialize the parameters of the generator and discriminator in the generative adversarial network; S312: The generator extracts multi-level feature information from the comprehensive image data using a convolutional neural network, and the generator generates a preliminary three-dimensional bone structure model according to the multi-level feature information; The discriminator receives the preliminary three-dimensional bone structure model generated by the generator, compares it with the multimodal image, uses a convolutional neural network to evaluate the geometric and anatomical consistency of the preliminary three-dimensional bone structure model, and feeds the discrimination result back to the generator; By calculating the output of the generator and the discriminator, the total loss function L is calculated. total =L content +λL style , where L content is the content loss and L style is the style loss, λ is the weight coefficient; Update the parameters of the generator and discriminator through the back-propagation algorithm; S313: Repeat the training process of S312 until the preliminary three-dimensional bone structure model generated by the generator reaches the expected bone structure data.

[0012] In some embodiments of the present application, S33 uses an adaptive multi-scale convolutional neural network to optimize the spatial resolution of the three-dimensional bone structure model after detail optimization, specifically including: S331: Receive the three-dimensional bone structure model after detail optimization and initialize the convolution kernel C through an adaptive multi-scale convolutional neural network k and the bias term B k ; S332: Extracting multi-scale feature maps from the detail-optimized three-dimensional bone structure model based on an adaptive multi-scale convolutional neural network; S333: fusing feature maps at different scales by weighted averaging to obtain a fused three-dimensional bone structure model; S334: Based on an adaptive convolutional neural network, the spatial resolution of the fused three-dimensional bone structure model is enhanced.

[0013] In some embodiments of the present application, S4 specifically includes: S41: Initialize the mechanical characteristic parameters, boundary conditions and load conditions required for mechanical characteristic simulation; The mechanical property parameters include Young's modulus, Poisson's ratio and density of bone tissue, and the boundary conditions include fixed support conditions B fixed and symmetric boundary condition B sym , the load conditions include external load F ext and the compressive load F under physiological conditions compression ; S42: Based on the geometric shape of the three-dimensional bone structure model, the bone structure is meshed to generate a finite element mesh; S43: Receive the finite element mesh and apply boundary conditions B fixed and B sym , and define the external load F ext and the compressive load F under physiological conditions compression , input finite element analysis model: K(G)·U=F ext +F compression, where K(G) is the stiffness matrix of the meshed bone structure, and U is the displacement vector; S44: The geometry and stress distribution of the three-dimensional bone structure model are simulated by iteratively solving the nonlinear finite element equation K(U)·U=F(U); Among them, K(U) is the stiffness matrix under the current displacement state, and F(U) is the external force matrix acting on the bone structure under the current displacement state; S44: Output the mechanical properties of bone tissue: σ=C·ε; Among them, σ is the stress tensor, C is the elastic matrix of the material, and ε is the strain tensor.

[0014] In some embodiments of the present application, S44 specifically includes: S441: Initialize the parameters required for the nonlinear finite element solution model, and establish the initial stiffness matrix K(U) and external force matrix F(U); S442: In each iteration process of solving the nonlinear finite element equation using the nonlinear finite element solution model, the stiffness matrix K(U) is updated: K(U)=K0+△K(U), until the updated amount of the displacement vector after the iteration meets the iteration convergence condition; Among them, K0 is the initial stiffness matrix, and △K(U) is the increment of the stiffness matrix updated due to deformation.

[0015] In some embodiments of the present application, S5 specifically includes: S51: receiving the stress tensor, strain tensor and displacement vector of the bone tissue, and the three-dimensional bone structure model, and initializing the model parameters of the bone biomechanics model; S52: Use deep variational inference algorithm to process the three-dimensional bone structure model and the mechanical properties of bone tissue, infer the potential variability and uncertainty of bone tissue, and infer the model parameters of the bone biomechanical model; S53: optimizing the model parameters of the inferred bone biomechanical model using a global optimization algorithm; S54: Generate a personalized bone biomechanical model based on the optimized mechanical properties of bone tissue and the three-dimensional bone structure model.

[0016] The embodiments of the present invention have the following advantages and beneficial effects: (1) Combining multimodal images, deep variational inference algorithms, and adversarial network algorithms provides more precise details for the reconstruction of three-dimensional bone structure models; (2) Combined with the nonlinear finite element method, it achieves efficient simulation of the mechanical properties of complex bone structures, avoids the limitations of traditional mechanical simulation based on simplified assumptions, and can perform precise analysis under tiny details and complex stress distribution, thus improving the accuracy and reliability of mechanical simulation results; (3) Using the deep variational inference algorithm, we can learn the potential distribution of bone tissue, infer the potential variability and uncertainty of bone tissue, and generate a personalized bone biomechanical model that conforms to individual characteristics. This allows the generated bone biomechanical model to fully reflect individual differences, meet clinical personalized needs, and have higher simulation accuracy and stronger adaptability.

[0017] In addition, the present application also relates to a bone biomechanics modeling system based on multimodal images, comprising: A multimodal image acquisition module, which is used to acquire CT images and MRI images of the bone structure to be measured and generate a multimodal image; An image registration and fusion module, which combines a deep variational inference algorithm and a generative adversarial network algorithm to fuse information of the multimodal images and generate comprehensive image data; A three-dimensional bone reconstruction module, which is connected to the image registration and fusion module and reconstructs a three-dimensional bone structure model based on the comprehensive image data; A mechanical property simulation module is connected to the three-dimensional bone reconstruction module and, based on the three-dimensional bone structure model, uses a nonlinear finite element method to simulate the mechanical property parameters of the three-dimensional bone structure model under different external loads and physiological states, and outputs the mechanical properties of the bone tissue; A bone biomechanics construction module is connected to the mechanical property simulation module and generates a personalized bone biomechanics model based on the three-dimensional bone structure model and the mechanical properties of bone tissue using a deep variational inference algorithm.

[0018] In some embodiments of the present application, the bone biomechanics building block comprises: A bone biomechanical model initialization module, which is connected to the mechanical property simulation module and is used to receive the stress tensor, strain tensor and displacement vector of the bone tissue, and the three-dimensional bone structure model, and initialize the model parameters of the bone biomechanical model; A deep variational inference algorithm module, which is connected to the bone biomechanical model initialization module, and is used to process the three-dimensional bone structure model and the mechanical properties of its bone tissue using a deep variational inference algorithm, and infer the model parameters of the bone biomechanical model to obtain the potential variability and uncertainty of the bone tissue; A global optimization algorithm module, which is connected to the deep variational inference algorithm module and is used to optimize the model parameters of the inferred bone biomechanics model using a global optimization algorithm; The bone biomechanical model generation module is connected to the global optimization algorithm module and is used to generate a personalized bone biomechanical model based on the model parameters of the optimized bone biomechanical model and the three-dimensional bone structure model.

[0019] After reading the specific embodiments of the present invention in conjunction with the accompanying drawings, other features and advantages of the present invention will become more clear. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, a brief introduction is given below to the drawings required for use in the embodiments of the present invention or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 A flowchart of the bone biomechanical modeling method based on multimodal images proposed by the present invention; Figure 2 A block diagram of the bone biomechanics modeling system based on multimodal images proposed by the present invention; Figure 3 A block diagram of a bone biomechanics building block in the bone biomechanics modeling system based on multimodal images proposed by the present invention; Reference numerals: 10. Multimodal image acquisition module; 20. Image registration and fusion module; 30. Three-dimensional bone reconstruction module; 40. Mechanical property simulation module; 50. Bone biomechanics construction module; 51. Bone biomechanics model initialization module; 52. Deep variational inference algorithm module; 53. Global optimization algorithm module; 54. Bone biomechanics model generation module. DETAILED DESCRIPTION

[0022] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0023] Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. In the description of the present invention, it should be understood that the orientation or position relationship indicated by the terms "center", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc. is based on the orientation or position relationship shown in the accompanying drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0024] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0025] The terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "plurality" means two or more.

[0026] In order to provide a bone biomechanical model with high accuracy and meeting personalized needs, the present application provides a bone biomechanical modeling method and system based on multimodal images. The bone biomechanical modeling method is implemented by the bone biomechanical modeling system. As follows, the bone biomechanical modeling method will be described in combination with the bone biomechanical modeling system.

[0027] S1: Acquire CT images and MRI images of the bone structure to be measured and generate a multimodal image.

[0028] In some embodiments of the present application, the bone biomechanics modeling system includes a multimodal image acquisition module 10 .

[0029] The multimodal image acquisition module 10 integrates a computer tomography device and a magnetic resonance imaging device, wherein the computer tomography device is used to acquire image data of bone structure to obtain CT (Computed Tomography) images, and the magnetic resonance imaging device is used to acquire image data of soft tissue to obtain MRI (Magnetic Resonance Imaging) images.

[0030] The multimodal image as described above includes the CT image and the MRI image acquired as described above.

[0031] S2: A deep variational inference algorithm is used to extract the common features of multimodal images, and a generative adversarial network algorithm is used to optimize the common features, perform information fusion on multimodal images, and generate comprehensive image data.

[0032] In some embodiments of the present application, the bone biomechanics modeling system further includes an image registration and fusion module 20 .

[0033] The image registration and fusion module 20 is used to extract common features of multimodal images using a deep variational inference algorithm, and optimize the common features using a generative adversarial network algorithm, perform information fusion on multimodal images, and generate comprehensive image data.

[0034] The deep variational inference algorithm and the generative adversarial network algorithm are both commonly used algorithms. In this application, image information is fused based on the deep variational inference algorithm and the generative adversarial network algorithm.

[0035] In some embodiments of the present application, a self-supervised generative adversarial network algorithm may be selected.

[0036] In some embodiments of the present application, a deep variational inference algorithm is responsible for extracting common features (such as bone morphology and density, etc., which also include potential common features) from multimodal images, and a self-supervised generative adversarial network algorithm is used to optimize these common features, enhance the details and consistency of the image, and at the same time make the details clearer.

[0037] In addition, the deep variational inference algorithm also provides potential common structures in multimodal images, and the self-supervised generative adversarial network algorithm enhances these potential common structures through adversarial training, ultimately generating more accurate comprehensive image data and improving the accuracy of subsequent three-dimensional bone structure modeling.

[0038] In addition, the combined use of deep variational inference algorithm and self-supervised generative adversarial network algorithm provides a more dynamic and intelligent optimization process.

[0039] S3: Reconstruct a three-dimensional bone structure model based on the comprehensive image data.

[0040] In some embodiments of the present application, the bone biomechanics modeling system further includes a three-dimensional bone reconstruction module 30 .

[0041] The three-dimensional bone reconstruction module 30 is connected to the image registration and fusion module 20, receives the integrated image data from the image registration and fusion module 20, and reconstructs the three-dimensional bone structure model M based on the integrated image data.

[0042] In some embodiments of the present application, the three-dimensional bone reconstruction module 30 reconstructs the three-dimensional bone structure model M using a generative adversarial network algorithm.

[0043] In order to improve the modeling accuracy of the three-dimensional bone structure model, in some embodiments of the present application, a three-dimensional bone structure model is generated based on comprehensive image data by a generative adversarial network algorithm, as described in detail as follows.

[0044] S31: Generate a preliminary 3D bone structure model M based on comprehensive image data using a generative adversarial network algorithm input .

[0045] In some embodiments of the present application, the bone biomechanics modeling system also includes a generative adversarial network three-dimensional modeling module, which receives comprehensive image data and reconstructs a three-dimensional bone structure model.

[0046] In some embodiments of the present application, in order to improve the modeling accuracy of the three-dimensional bone structure model, the comprehensive image data needs to be preprocessed before modeling is performed using the comprehensive image data.

[0047] In some embodiments of the present application, the bone biomechanics modeling system further includes an image data receiving and processing module, which is connected to the image registration and fusion module 20 .

[0048] The image data receiving and processing module receives the integrated image data from the image registration and fusion module 20, and pre-processes the received integrated image data to generate pre-processed integrated image data.

[0049] In some embodiments of the present application, a generative adversarial network three-dimensional modeling module receives preprocessed comprehensive image data and reconstructs a three-dimensional bone structure model through a generative adversarial network algorithm, thereby improving the modeling accuracy of the three-dimensional bone structure model.

[0050] Specifically, the generative adversarial network includes a generator and a discriminator. The generator generates a preliminary three-dimensional bone structure model M through a deep convolutional neural network. input , the discriminator evaluates the preliminary 3D bone structure model M input The geometric and anatomical consistency of the image is obtained and the information is fed back to the generator to optimize the generator, so that the generator can continuously adjust the three-dimensional bone structure model and optimize the accuracy and details of the model, so that the three-dimensional bone structure model can be as close to the comprehensive image data as possible.

[0051] The training process between the generator and the discriminator can be described by the following optimization formula (1): (1) Among them, L GAN is the loss function of the generative adversarial network, D(x) represents the judgment of the discriminator on the real data x, x represents the multimodal image data, G(z) is the fake data generated by the generator based on the latent variable z, z is generated by using random noise, p data (x) is the real data distribution, p z (z) is the latent variable distribution and E is the expectation.

[0052] As described above, the generative adversarial network includes a generator and a discriminator. The specific description of using the generator and the discriminator to generate a preliminary three-dimensional bone structure model is as follows.

[0053] The execution process of the generative adversarial network includes an initialization process, a training process and an iteration process, wherein the initialization process includes the following S311, the training process includes the following S312, and the iteration process includes the following S313.

[0054] S311: A step of initializing the parameters of the generator and discriminator in the generative adversarial network.

[0055] In some embodiments of the present application, the bone biomechanics modeling system also includes a generative adversarial network initialization module, which is connected to the image data receiving and processing module to receive the pre-processed comprehensive image data.

[0056] In some embodiments of the present application, the generative adversarial network initialization module uses a Gaussian initialization method to set the initial parameters of the generator and the discriminator.

[0057] S312: Generator and discriminator adversarial training.

[0058] In some embodiments of the present application, the bone biomechanics modeling system also includes a generator network module, which is connected to the generative adversarial network initialization module and uses a deep convolutional neural network to generate a preliminary three-dimensional bone structure model M'.

[0059] Specifically, the generator extracts multi-level feature information of the comprehensive image data through multiple convolutional layers, deconvolution layers and activation functions. The multi-level feature information helps the generator understand the details and structures in the comprehensive image data, and constructs a preliminary three-dimensional bone structure model M' based on the feature information.

[0060] In some embodiments of the present application, the bone biomechanics modeling system also includes a discriminator network module, which is connected to both the generator network module and the adversarial network initialization module.

[0061] The discriminator network module receives the preliminary three-dimensional bone structure model M' generated by the generator and compares it with the multimodal image. It uses a deep convolutional neural network to evaluate the geometric and anatomical consistency of the preliminary three-dimensional bone structure model M' and feeds the discrimination result back to the generator.

[0062] During the adversarial training process of the generative adversarial network, it is necessary to calculate the total loss function to measure the similarity between the three-dimensional bone structure model M' and the comprehensive image data, and continuously and accurately optimize the generator so that the three-dimensional bone structure model output by the generator is closer to the real bone structure in terms of anatomical consistency and detail accuracy.

[0063] In some embodiments of the present application, the bone biomechanics modeling system also includes a loss function calculation module, which is connected to the discriminator network module.

[0064] The loss function calculation module obtains the total loss function by calculating the output results of the generator and the discriminator to evaluate the training effect of the generative adversarial network.

[0065] The total loss function includes the content loss L content and style loss L style , content loss L content It measures the geometric similarity between the three-dimensional bone structure model M' and the comprehensive image data. It usually calculates the error between the two based on the feature extraction layer, and the style loss L style The consistency between the three-dimensional bone structure model M' and the comprehensive image data in terms of global features, edge information, and spatial distribution was evaluated, and the difference was calculated using the weighted summation of feature maps.

[0066] Therefore, the total loss function L total It can be expressed as follows (2).

[0067] L total =L content +λ·L style (2).

[0068] Among them, λ is the weight coefficient, which controls the relative influence of content loss and style loss.

[0069] In some embodiments of the present application, the bone biomechanics modeling system also includes a training optimization module, which is connected to the loss function calculation module.

[0070] The training optimization module updates the parameters of the generator and discriminator through the back-propagation algorithm, optimizes the generator so that the three-dimensional bone structure model it generates is closer to the real bone structure data, and the optimization process is adjusted using the gradient descent method.

[0071] S313: Repeat the training process of S312 until the preliminary three-dimensional bone structure model generated by the generator reaches the expected bone structure data.

[0072] In some embodiments of the present application, the bone biomechanics modeling system further includes a three-dimensional bone structure output module, which is connected to the training optimization module and is used to output a three-dimensional bone structure model after multiple training and optimization, that is, the preliminary three-dimensional bone structure model M as described above. input The optimized three-dimensional bone structure model has higher anatomical consistency and detail accuracy than other models.

[0073] The embodiments of the present application generate an adversarial network and combine it with a deep convolutional neural network and multi-level feature information extraction to accurately generate a three-dimensional bone structure model.

[0074] In addition, the generator and discriminator are optimized through back propagation and total loss function to continuously improve the anatomical consistency and detail accuracy of the three-dimensional bone structure model, solve the accuracy and detail problems in traditional bone structure modeling methods, and can generate high-quality three-dimensional bone structure models that are closer to the real bone structure with higher accuracy and reliability.

[0075] S32: Based on the preliminary three-dimensional bone structure model M input , the details of the local area of ​​the bone structure are optimized to obtain a three-dimensional bone structure model after detail optimization.

[0076] In order to make the model more refined, in some embodiments of the present application, the bone biomechanics modeling system also includes a local detail optimization module, which is connected to the generative adversarial network three-dimensional modeling module.

[0077] In some embodiments of the present application, the local detail optimization module is based on the preliminary three-dimensional bone structure model M input , optimize the details of the local area of ​​the bone structure, and obtain the three-dimensional bone structure model M after detail optimization 细节 .

[0078] Specifically, the local detail optimization module is based on the preliminary 3D bone structure model M input , the existing image gradient calculation method and edge enhancement algorithm are applied to optimize the details of the local area of ​​the bone structure to ensure that a more refined model is obtained.

[0079] S33: 3D bone structure model based on detail optimization M input , optimize its spatial resolution and obtain the enhanced three-dimensional bone structure model M enhanced .

[0080] In order to improve the overall spatial accuracy of the model, in some embodiments of the present application, the bone biomechanics modeling system also includes a spatial resolution enhancement module, which is connected to the local detail optimization module.

[0081] In some embodiments of the present application, the spatial resolution enhancement module performs detail optimization on the three-dimensional bone structure model M. input Optimize spatial resolution.

[0082] Specifically, the spatial resolution enhancement module uses the existing adaptive multi-scale convolutional neural network (see the following formula (3)) to transform the three-dimensional bone structure model M input , optimize the spatial resolution, extract and fuse features at different scales, and optimize the three-dimensional bone structure model M input The overall spatial accuracy of the image is excellent, especially the representation of fine bone structures and minute anatomical details.

[0083] (3) Among them, M enhanced For the enhanced three-dimensional bone structure model, M input is the input three-dimensional bone structure model, C k is the k-th convolution kernel, B k is the bias term of the kth layer, K is the number of convolutional layers, and k is the convolutional layer number.

[0084] S331: Receive the three-dimensional bone structure model M after detail optimization input , and initialize the convolution kernel C through an adaptive multi-scale convolutional neural network k and the bias term B k .

[0085] In some embodiments of the present application, the bone biomechanics modeling system also includes a spatial resolution enhancement initialization module, which is connected to the local detail optimization module.

[0086] The spatial resolution enhancement initialization module receives the three-dimensional bone structure model M after detail optimization. input , and initialize the convolution kernel C through an adaptive multi-scale convolutional neural network k and the bias term B k .

[0087] S332: 3D bone structure model after detail optimization based on adaptive multi-scale convolutional neural network M input Extract multi-scale feature maps.

[0088] In some embodiments of the present application, the bone biomechanics modeling system also includes a multi-scale feature extraction module, which is connected to the spatial resolution enhancement initialization module.

[0089] The multi-scale feature extraction module is based on an adaptive multi-scale convolutional neural network (see the following formula (4)) to extract the three-dimensional bone structure model M after detail optimization. input Feature extraction is performed. Specifically, feature maps containing local texture features, edge features, and global anatomical structure information in the bone structure are extracted at different scales through multiple convolutional layers, pooling layers, and deconvolution layers.

[0090] F k =δ (C k * M input +B k ) (4).

[0091] Among them, F k represents the feature map extracted by the kth layer, and δ is the activation function used to introduce nonlinear feature representation.

[0092] S333: Fusing feature maps at different scales by weighted averaging to obtain a fused three-dimensional bone structure model.

[0093] In some embodiments of the present application, the bone biomechanics modeling system also includes a feature fusion module, which is connected to the multi-scale feature extraction module.

[0094] The feature fusion module fuses feature maps at different scales by weighted summing feature maps at different scales to synthesize optimized three-dimensional bone structure information with global and local details: (5) Among them, M fused is the three-dimensional bone structure model after fusion, ω k is the weight coefficient of the k-th layer feature map and satisfies , K is the number of convolutional layers.

[0095] S334: Based on the adaptive convolutional neural network, the fused three-dimensional bone structure model M fused Perform spatial resolution enhancement.

[0096] In some embodiments of the present application, the bone biomechanics modeling system also includes a spatial resolution enhancement module, which is connected to the feature fusion module.

[0097] The spatial resolution enhancement module is used to enhance the fused three-dimensional bone structure model M fused The spatial resolution is enhanced by multi-layer deconvolution operation and feature reconstruction method to optimize the detail performance of 3D bone structure information, especially the performance of small-scale bone structure and tiny anatomical details. The enhanced 3D bone structure model M enhanced It can be expressed by the following formula (6).

[0098] M enhanced =f(M fused ,θ) (6).

[0099] Among them, M enhanced is the enhanced 3D bone structure model, f(·) is the adaptive convolution operation, and θ is the optimized convolution kernel parameter.

[0100] The embodiments of the present application perform feature extraction and spatial resolution enhancement on the three-dimensional bone structure model through an adaptive multi-scale convolutional neural network, thereby achieving an optimized fusion of local details of the bone structure and global anatomical information, and can accurately extract and reconstruct the details of the small-scale bone structure, thereby improving the resolution and accuracy of the three-dimensional bone structure model reconstruction, and providing high-quality three-dimensional structural data for subsequent mechanical property simulation and personalized bone biomechanical modeling.

[0101] S34: Combine the real clinical data acquired in advance with the enhanced 3D bone structure model M enhancedCompare and correct the enhanced three-dimensional bone structure model M enhanced The key areas of the system are corrected for accuracy.

[0102] In some embodiments of the present application, the bone biomechanics modeling system further includes a local accuracy correction module, which is connected to the spatial resolution enhancement module and receives the enhanced three-dimensional bone structure model M. enhanced .

[0103] The local accuracy correction module uses the existing image contrast correction method to compare the pre-acquired real clinical data with the enhanced three-dimensional bone structure model M. enhanced Comparison and correction are performed to optimize the accuracy of the 3D bone structure model in critical areas (e.g., fracture areas, articular surfaces) and adjust geometric features at key anatomical locations.

[0104] This image contrast correction method combines existing image registration and correction techniques to help optimize the anatomical accuracy of the model so that it better meets clinical needs.

[0105] S35: Output the accuracy-corrected three-dimensional bone structure model.

[0106] In some embodiments of the present application, the bone biomechanics modeling system further includes a three-dimensional bone structure output module, which is connected to the local accuracy correction module and outputs an optimized three-dimensional bone structure model M. 优化 .

[0107] The optimized three-dimensional bone structure model M 优化 That is the three-dimensional bone structure model M as described above.

[0108] This application generates a three-dimensional bone structure model through a generative adversarial network algorithm, and optimizes the geometric and anatomical consistency of the model through the interaction between the generator and the discriminator. The reconstruction model combines a deep convolutional neural network with an optimized feedback mechanism to improve the accuracy and detail of the three-dimensional bone structure model, especially in the reconstruction of small bone structures and anatomical details, achieving higher modeling accuracy and providing a more reliable model basis for subsequent mechanical analysis.

[0109] S4: Based on the three-dimensional bone structure model, the nonlinear finite element method is used to simulate the mechanical characteristic parameters of the three-dimensional bone structure model under different external loads and physiological states, and the mechanical properties of the bone tissue are output.

[0110] In some embodiments of the present application, the bone biomechanics modeling system further includes a mechanical property simulation module 40 , which is connected to the three-dimensional bone reconstruction module 30 .

[0111] The three-dimensional bone structure model used in S4 may be the optimized three-dimensional bone structure model M as described above. 优化 .

[0112] Among them, the mechanical property parameters include Young's modulus E, Poisson's ratio υ and density ρ of bone tissue, which are used to describe the physical properties of bone tissue; mechanical properties refer to the actual mechanical behavior of bone structure under external loads simulated under the input mechanical property parameters, such as stress, strain and other mechanical behaviors, which are used to describe the deformation and bearing capacity of bone tissue under different external loads.

[0113] S41: Receive the optimized three-dimensional bone structure model M 优化 , and initialize the mechanical property parameters, boundary conditions and load conditions required for mechanical property simulation.

[0114] Among them, the boundary conditions include fixed support conditions B fixed and symmetric boundary condition B sym , the load conditions include external load F ext and the compressive load F under physiological conditions compression .

[0115] In some embodiments of the present application, the mechanical property simulation module 40 further includes a mechanical property simulation initialization module (not shown), which is used to execute S41 as described above.

[0116] S42: Based on the geometric shape of the three-dimensional bone structure model, the bone structure is meshed to generate a finite element mesh.

[0117] In some embodiments of the present application, the mechanical property simulation module 40 further includes a meshing module (not shown), which is connected to the mechanical property simulation initialization module to generate a mesh of the three-dimensional bone structure model M. 优化 Perform meshing processing to generate a finite element mesh G.

[0118] The meshing is based on the geometric shape of the three-dimensional bone structure model. The mesh G is generated by an adaptive mesh generation method, and the mesh density is optimized in the stress concentration area.

[0119] G=mesh(M 优化 ) (7).

[0120] Where mesh(·) represents the meshing operation.

[0121] S43: Receive the finite element mesh G and apply boundary conditions B fixed and B sym , and define the external load F ext and the compressive load F under physiological conditions compression , input the finite element analysis model.

[0122] In some embodiments of the present application, the mechanical property simulation module 40 also includes a boundary condition and load definition module (not shown), which is connected to the mesh generation module.

[0123] The Boundary Condition and Load Definition module receives the finite element mesh G and applies boundary conditions B fixed and B sym , and define the external load F ext and the compressive load F under physiological conditions compression , constrains the response of the bone structure and is input into the finite element analysis model (see formula (8)).

[0124] K(G)·U=F ext +F compression (8).

[0125] Among them, K(G) is the stiffness matrix of the meshed bone structure, and U is the displacement vector.

[0126] The stiffness matrix is ​​used to analyze the mechanical response of bone tissue when subjected to external forces. By calculating the stiffness matrix, the deformation and stress distribution of bone tissue can be predicted, thereby evaluating its mechanical properties and stability.

[0127] Each element in the stiffness matrix represents the stiffness coefficient of the meshed bone structure, which can be calculated through factors such as Young's modulus E, Poisson's ratio υ and density ρ of bone tissue.

[0128] S44: The geometry and stress distribution of the three-dimensional bone structure model are simulated by iteratively solving the nonlinear finite element equation K(U)·U=F(U).

[0129] In some embodiments of the present application, the mechanical property simulation module 40 further includes a nonlinear finite element solution module (not shown), which is connected to the boundary condition and load definition module.

[0130] The nonlinear finite element solution module is based on the applied boundary conditions B fixed , B sym and load condition F ext 、F compression , by iteratively solving the nonlinear finite element equation K(U)·U=F(U), a stable displacement solution is obtained, thereby simulating the geometric shape and stress distribution of the three-dimensional bone structure model.

[0131] The geometry and stress distribution of the three-dimensional bone structure model essentially use mechanical property parameters to calculate the performance of the bone structure under external loads.

[0132] Among them, K(U) is the stiffness matrix under the current displacement state, and F(U) is the external force matrix acting on the bone structure under the current displacement state.

[0133] In some embodiments of the present application, in order to solve the nonlinear finite element equation K(U)·U=F(U) to simulate the geometry and stress distribution of the three-dimensional bone structure model, it can be specifically described as follows.

[0134] S441: Initialize the parameters required for the nonlinear finite element solution model, and establish the initial stiffness matrix K(U) and external force matrix F(U).

[0135] In some embodiments of the present application, the nonlinear finite element solution module also includes a nonlinear finite element model initialization module, which is connected to the boundary condition and load definition module.

[0136] The nonlinear finite element model initialization module receives the boundary conditions B fixed , B sym and load condition F ext 、F compression , and initialize the parameters required for the nonlinear finite element solution model, and establish the initial stiffness matrix K(U) and external force matrix F(U).

[0137] S442: In each iteration process of solving the nonlinear finite element equation using the nonlinear finite element solution model, the stiffness matrix K(U) is updated: K(U)=K0+△K(U) until the updated displacement vector after the iteration meets the iteration convergence condition.

[0138] Among them, K0 is the initial stiffness matrix, and △K(U) is the increment of the stiffness matrix updated due to deformation.

[0139] In some embodiments of the present application, at each iteration, the local stiffness of the material is calculated based on the displacement vector U and adjusted based on the constitutive model and deformation state of the material, wherein the constitutive model of the material is updated based on the current stress tensor, strain tensor and displacement vector, and the updated stiffness matrix affects the next solution process.

[0140] In some embodiments of the present application, the above nonlinear finite element equations are iteratively solved using, for example, the Newton-Raphson method.

[0141] In some embodiments of the present application, during the iterative solution process, the displacement update amount after each iteration is monitored, and whether the iteration converges is determined by setting a convergence threshold ε. When the update amount of the displacement vector satisfies the condition ||U n+1 -U n ||<ε, the iterative process is considered to have converged and a stable displacement solution is output; otherwise, the iteration continues until convergence.

[0142] S45: Output the mechanical properties of bone tissue: σ=C·ε.

[0143] In some embodiments of the present application, the mechanical property simulation module 40 further includes a mechanical property output module (not shown), which is connected to the nonlinear finite element solution module to output the mechanical properties of the bone tissue.

[0144] Among them, σ is the stress tensor, C is the elastic matrix of the material, and ε is the strain tensor.

[0145] The embodiments of the present application can simulate the deformation and stress distribution of bone structures under external loads and physiological conditions in detail through accurate mechanical property parameter simulation and nonlinear finite element analysis, which helps to generate a high-precision bone biomechanical model.

[0146] And through adaptive meshing and iterative solution, the computational efficiency and accuracy of the model are optimized, providing a more accurate mechanical property analysis tool for orthopedic clinical applications, with the advantages of high efficiency and reliability.

[0147] S5: Based on the three-dimensional bone structure model and the mechanical properties of bone tissue, a personalized bone biomechanical model is generated using a deep variational inference algorithm.

[0148] In some embodiments of the present application, the bone biomechanics modeling system further includes a bone biomechanics construction module 50 , which is connected to the mechanical property simulation module 40 .

[0149] In some embodiments of the present application, S5 specifically includes the following.

[0150] S51: receiving the stress tensor, strain tensor and displacement vector of the bone tissue, and the three-dimensional bone structure model, and initializing the model parameters of the bone biomechanics model.

[0151] In some embodiments of the present application, the bone biomechanics construction module 50 includes a bone biomechanics model initialization module 51 , which is connected to the mechanical property simulation module 40 .

[0152] The bone biomechanics model initialization module 51 receives the stress tensor, strain tensor and displacement vector of the bone tissue, and the three-dimensional bone structure model, and initializes the model parameters of the bone biomechanics model.

[0153] The initialization step includes establishing the preliminary structure of the bone biomechanical model and setting the initial mechanical property parameters, geometric shape, boundary conditions and load conditions.

[0154] S52: Use deep variational inference algorithm to process the three-dimensional bone structure model and the mechanical properties of its bone tissue, and infer the model parameters of the bone biomechanical model to obtain the potential variability and uncertainty of bone tissue.

[0155] In some embodiments of the present application, the bone biomechanics construction module 50 also includes a deep variational inference algorithm module 52 , which is connected to the bone biomechanics model initialization module 51 .

[0156] The deep variational inference algorithm module 52 uses the existing deep variational inference algorithm to process the three-dimensional bone structure model and the mechanical properties of its bone tissue, infer the potential variability and uncertainty of the bone tissue, and incorporate its uncertainty and variability into the bone biomechanical model.

[0157] In the deep variational inference algorithm, by constructing the variational inference loss function L var , a variational Bayesian method is used to infer and optimize the model parameters of the bone biomechanical model to infer and quantify the potential variability and uncertainty of bone tissue, so that the bone biomechanical model can more accurately simulate individual differences and uncertainties in the data, thereby generating a more personalized and reliable bone biomechanical model.

[0158] L var =E[logp(M, σ,ε| )]-KL[q(U, σ,ε)||p(U, σ,ε)] (9).

[0159] In formula (9), M is the three-dimensional bone structure model, are the model parameters of the bone biomechanical model to be optimized, KL is the Kullback-Leibler divergence, q(·) is the variational posterior distribution, p(·) is the target posterior distribution, and E is the expectation.

[0160] Among them, the model parameters of the bone biomechanical model refer to the latent variables of the bone biomechanical model (including mechanical properties such as stress tensor, strain tensor, displacement vector, etc.), as well as bone tissue material parameters that reflect individual differences (for example, Young's modulus, Poisson's ratio, density). These model parameters are inferred through a deep variational inference algorithm to obtain the potential variability and uncertainty of bone tissue.

[0161] S53: Optimizing model parameters of the inferred bone biomechanical model using a global optimization algorithm.

[0162] The objective function in the global optimization algorithm is formula (10).

[0163] (10).

[0164] in, is the model parameter of the optimized bone biomechanical model. In formula (10), L var Compared with L in formula (9) var Both refer to the variational inference loss function.

[0165] In some embodiments of the present application, the bone biomechanics construction module 50 also includes a global optimization algorithm module 53 , which is connected to the deep variational inference algorithm module 52 .

[0166] The global optimization algorithm module 53 uses a global optimization algorithm to globally optimize the model parameters output by the deep variational inference algorithm module 52, and optimizes the mechanical properties of the bone tissue in the bone biomechanical model through a global optimization strategy, so that the generated bone biomechanical model adapts to individual physiological characteristics and clinical needs.

[0167] S54: Generate a personalized bone biomechanical model based on the optimized model parameters of the bone biomechanical model and the three-dimensional bone structure model.

[0168] In some embodiments of the present application, the bone biomechanics construction module 50 also includes a bone biomechanics model generation module 54 , which is connected to the global optimization algorithm module 53 .

[0169] The bone biomechanics model generating module 54 is used to execute S54.

[0170] The embodiments of the present application optimize the model parameters of the bone biomechanics model according to individual physiological characteristics and clinical needs by combining a deep variational inference algorithm and a global optimization algorithm.

[0171] By accurately inferring the potential variability and uncertainty of bone tissue, a personalized bone biomechanical model that conforms to individual characteristics is generated, which improves the accuracy and personalization of bone structure analysis and provides more efficient support for clinical applications.

[0172] In some embodiments of the present application, the bone biomechanical modeling method further includes: S5': based on the three-dimensional bone structure model obtained in S3, using existing self-supervised learning methods to perform unlabeled learning on the comprehensive image data to extract bone structure and mechanical performance characteristics.

[0173] The mechanical characteristics here are the mechanical properties of bones extracted by simulating the geometric shape and stress distribution of bone structure, such as deformation mode and stress concentration area, which are used to analyze the strength and fragility of bones.

[0174] This mechanical feature is a supplementary information that provides more detailed information about bone stress distribution and deformation, and is used to enhance the accuracy of the model during model optimization and personalized adjustment. It provides more detailed data for the generation and optimization of bone biomechanical models, ensuring that the model better meets clinical needs.

[0175] In some embodiments of the present application, in order to verify the feasibility and effectiveness of the present application, the present application is applied to the orthopedic clinical treatment and rehabilitation center of a comprehensive hospital in a certain city. The center is responsible for treating patients with various complex fractures, involving various types of fractures and large individual differences among patients. Traditional bone biomechanical analysis methods are difficult to meet the needs of accurate monitoring and personalized treatment during fracture healing.

[0176] This application provides each patient with accurate bone biomechanical assessment and personalized treatment plans through a bone biomechanical modeling system based on multimodal images, thereby effectively improving treatment outcomes and recovery speed.

[0177] To facilitate testing, some modules of the modeling system described above are simplified to form a simplified system. The simplified system mainly includes the mechanical property simulation module 40, the bone biomechanical model initialization module 51, the deep variational inference algorithm module 52, the global optimization algorithm module 53 and the bone biomechanical model generation module 54 in the modeling system described above. Through the coordinated work of these modules, the bone biomechanical model can be calculated and optimized in real time based on the patient's three-dimensional bone structure and mechanical properties, personalized bone mechanical parameters can be generated, and personalized recovery plans can be designed for the patient's fracture site.

[0178] During the application process, the patient's three-dimensional bone structure model and mechanical property data are first collected through the mechanical property simulation module 40, and the mechanical property data include stress tensor, strain tensor and displacement vector, and a preliminary bone biomechanical model is generated through the bone biomechanical model initialization module 51 (see S51 above).

[0179] Then, the deep variational inference algorithm module 52 uses a deep variational inference algorithm to further infer and optimize the bone biomechanical model, calculate the potential variability and uncertainty of the bone tissue, and adjust the model parameters of the bone biomechanical model based on this information (see S52 above).

[0180] Next, the global optimization algorithm module 53 uses a global optimization algorithm to accurately optimize the bone biomechanical model (see S53 above), so that the generated personalized bone biomechanical model can better meet the individual physiological characteristics and clinical needs of the patient.

[0181] Finally, the bone biomechanical model generation module 54 outputs the final personalized bone biomechanical model (see S54 above) and provides support for clinical treatment.

[0182] In order to verify the effect of this modeling method in clinical applications, a 30-day clinical test was conducted at the Orthopedic Rehabilitation Center of a comprehensive hospital in a certain city. The test subjects were 30 patients with different types of fractures, including femur, tibia, humerus, etc. All patients wore the simplified system as described above during treatment. The simplified system collected the mechanical property data of the fracture site in real time through sensors, and calculated the bone biomechanical model of each patient. During the patient's rehabilitation process, the simplified system adjusted the rehabilitation plan according to the personalized bone biomechanical model, and monitored the patient's recovery in real time. Table 1 is a comparative data on the effect of this application and traditional treatment methods in the fracture healing process.

[0183] Table 1

[0184] From the data in Table 1, it can be seen that after applying this application, the fracture healing rate of patients has significantly increased from 65.2% of the traditional treatment method to 89.6%. This shows that this application can optimize the treatment plan through a personalized bone biomechanical model and effectively promote the healing of fractures.

[0185] In addition, the personalized adaptation rate of the treatment plan increased from 70.5% to 94.2%, indicating that the present application can adjust the treatment plan according to the individual differences of the patient, making it more targeted and adaptable. The completion rate of patient rehabilitation training also increased significantly, from 72.3% to 96.1%, indicating that the personalized treatment plan is more operational and effective. More importantly, the patient's pain relief time was shortened from 14.5 days to 9.2 days, showing the positive effect of personalized treatment on pain relief. Patient satisfaction with the treatment plan also increased from 71.8% to 93.5%, indicating that the personalized treatment plan of this application is more in line with patient needs. The error rate was reduced from 12.4% to 4.1%, greatly improving the accuracy and reliability of treatment data.

[0186] Therefore, the bone biomechanical modeling system involved in this application can effectively solve the problems of insufficient personalized needs, unstable treatment effects and long recovery time in traditional treatment methods in fracture rehabilitation. Through real-time bone biomechanical analysis and personalized treatment plan optimization, this application significantly improves the treatment effect, shortens the recovery time, and improves patient satisfaction, and has broad clinical application prospects.

[0187] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A bone biomechanical modeling method based on multimodal images, characterized in that: include: S1: Acquire CT images and MRI images of the bone structure to be tested and generate multimodal images; S2: extracting common features of the multimodal images using a deep variational inference algorithm, optimizing the common features using a generative adversarial network algorithm, performing information fusion on the multimodal images, and generating comprehensive image data; S3: reconstructing a three-dimensional bone structure model based on the comprehensive image data; S4: Based on the three-dimensional bone structure model, a nonlinear finite element method is used to simulate the mechanical characteristic parameters of the three-dimensional bone structure model under different external loads and physiological states, and the mechanical characteristics of the bone tissue are output; S5: Based on the three-dimensional bone structure model and the mechanical properties of bone tissue, a personalized bone biomechanical model is generated using a deep variational inference algorithm.

2. The bone biomechanical modeling method based on multimodal images according to claim 1, characterized in that: S3 specifically includes: S31: Based on the comprehensive image data, a preliminary three-dimensional bone structure model is generated using a generative adversarial network algorithm; S32: Based on the preliminary three-dimensional bone structure model, optimizing the details of a local area of ​​the bone structure to obtain a three-dimensional bone structure model after the details are optimized; S33: based on the detail-optimized three-dimensional bone structure model, optimizing the spatial resolution thereof to obtain an enhanced three-dimensional bone structure model; S34: comparing and correcting the real clinical data acquired in advance with the enhanced three-dimensional bone structure model, and performing accuracy correction on key areas of the enhanced three-dimensional bone structure model; S35: Output the accuracy-corrected three-dimensional bone structure model.

3. The bone biomechanical modeling method based on multimodal images according to claim 2, characterized in that: The S3 also includes: S31 ′: pre-process the integrated image data from S2 to generate pre-processed integrated image data for use in S31 .

4. The bone biomechanical modeling method based on multimodal images according to claim 2, characterized in that: S31 specifically includes: S311: Initialize the parameters of the generator and discriminator in the generative adversarial network; S312: The generator extracts multi-level feature information from the comprehensive image data using a convolutional neural network, and the generator generates a preliminary three-dimensional bone structure model according to the multi-level feature information; The discriminator receives the preliminary three-dimensional bone structure model generated by the generator, compares it with the multimodal image, uses a convolutional neural network to evaluate the geometric and anatomical consistency of the preliminary three-dimensional bone structure model, and feeds the discrimination result back to the generator; By calculating the output of the generator and the discriminator, the total loss function L is calculated. total =L content +λL style , where L content is the content loss, L style is the style loss, λ is the weight coefficient; Update the parameters of the generator and discriminator through the back-propagation algorithm; S313: Repeat the training process of S312 until the preliminary three-dimensional bone structure model generated by the generator reaches the expected bone structure data.

5. The bone biomechanical modeling method based on multimodal images according to claim 2, characterized in that: S33 uses an adaptive multi-scale convolutional neural network to optimize the spatial resolution of the 3D bone structure model after detail optimization, including: S331: Receive the three-dimensional bone structure model after detail optimization and initialize the convolution kernel C through an adaptive multi-scale convolutional neural network k and the bias term B k , where K is the number of convolutional layers and k is the convolutional layer number; S332: Extracting multi-scale feature maps from the detail-optimized three-dimensional bone structure model based on an adaptive multi-scale convolutional neural network; S333: fusing feature maps at different scales by weighted averaging to obtain a fused three-dimensional bone structure model; S334: Based on an adaptive convolutional neural network, the spatial resolution of the fused three-dimensional bone structure model is enhanced.

6. The bone biomechanical modeling method based on multimodal images according to claim 1, characterized in that: S4 specifically includes: S41: Initialize the mechanical characteristic parameters, boundary conditions and load conditions required for mechanical characteristic simulation; The mechanical property parameters include Young's modulus, Poisson's ratio and density of bone tissue, and the boundary conditions include fixed support conditions B fixed and symmetric boundary condition B sym , the load conditions include external load F ext and the compressive load F under physiological conditions compression ; S42: Based on the geometric shape of the three-dimensional bone structure model, the bone structure is meshed to generate a finite element mesh; S43: Receive the finite element mesh and apply boundary conditions B fixed and B sym , and define the external load F ext and the compressive load F under physiological conditions compression , input finite element analysis model: K(G)·U=F ext +F compression ; Among them, K(G) is the stiffness matrix of the meshed bone structure, and U is the displacement vector; S44: The geometry and stress distribution of the three-dimensional bone structure model are simulated by iteratively solving the nonlinear finite element equation K(U)·U=F(U); Among them, K(U) is the stiffness matrix under the current displacement state, and F(U) is the external force matrix acting on the bone structure under the current displacement state; S45: Output the mechanical properties of bone tissue: σ=C·ε; Among them, σ is the stress tensor, C is the elastic matrix of the material, and ε is the strain tensor.

7. The bone biomechanical modeling method based on multimodal images according to claim 6, characterized in that: S44 specifically includes: S441: Initialize the parameters required for the nonlinear finite element solution model, and establish the initial stiffness matrix K(U) and external force matrix F(U); S442: In each iteration process of solving the nonlinear finite element equation using the nonlinear finite element solution model, the stiffness matrix K(U) is updated: K(U)=K0+△K(U), until the updated amount of the displacement vector after the iteration meets the iteration convergence condition; Among them, K0 is the initial stiffness matrix, and △K(U) is the increment of the stiffness matrix updated due to deformation.

8. The bone biomechanical modeling method based on multimodal images according to claim 1, characterized in that: S51: receiving the stress tensor, strain tensor and displacement vector of the bone tissue, and the three-dimensional bone structure model, and initializing the model parameters of the bone biomechanics model; S52: Use deep variational inference algorithm to process the three-dimensional bone structure model and the mechanical properties of bone tissue, and infer the model parameters of the bone biomechanical model to obtain the potential variability and uncertainty of bone tissue; S53: optimizing the model parameters of the inferred bone biomechanical model using a global optimization algorithm; S54: Generate a personalized bone biomechanical model based on the optimized model parameters of the bone biomechanical model and the three-dimensional bone structure model.

9. A bone biomechanical modeling system based on multimodal images, characterized in that: include: A multimodal image acquisition module, which is used to acquire CT images and MRI images of the bone structure to be measured and generate a multimodal image; An image registration and fusion module, which combines a deep variational inference algorithm and a generative adversarial network algorithm to fuse information of the multimodal images and generate comprehensive image data; A three-dimensional bone reconstruction module, which is connected to the image registration and fusion module and reconstructs a three-dimensional bone structure model based on the comprehensive image data; A mechanical property simulation module is connected to the three-dimensional bone reconstruction module and, based on the three-dimensional bone structure model, uses a nonlinear finite element method to simulate the mechanical property parameters of the three-dimensional bone structure model under different external loads and physiological states, and outputs the mechanical properties of the bone tissue; A bone biomechanics construction module is connected to the mechanical property simulation module and generates a personalized bone biomechanics model based on the three-dimensional bone structure model and the mechanical properties of bone tissue using a deep variational inference algorithm.

10. The bone biomechanics modeling system based on multimodal images according to claim 9, characterized in that: The bone biomechanics building blocks include: A bone biomechanical model initialization module, which is connected to the mechanical property simulation module and is used to receive the stress tensor, strain tensor and displacement vector of the bone tissue, and the three-dimensional bone structure model, and initialize the model parameters of the bone biomechanical model; A deep variational inference algorithm module, which is connected to the bone biomechanical model initialization module, and is used to process the three-dimensional bone structure model and the mechanical properties of its bone tissue using a deep variational inference algorithm, and infer the model parameters of the bone biomechanical model to obtain the potential variability and uncertainty of the bone tissue; A global optimization algorithm module, which is connected to the deep variational inference algorithm module and is used to optimize the model parameters of the inferred bone biomechanics model using a global optimization algorithm; The bone biomechanical model generation module is connected to the global optimization algorithm module and is used to generate a personalized bone biomechanical model based on the model parameters of the optimized bone biomechanical model and the three-dimensional bone structure model.

Images