A Bone Biomechanics Modeling Method and System Based on Multimodal Images

Abstract

The present invention relates to the field of bone biomechanics simulation technology, and discloses a bone biomechanics modeling method and system based on multimodal images. The method includes: S1: Collecting CT images and MRI images of the bone structure to be measured to generate multimodal images; S2: Using a deep variational inference algorithm to extract the common features of the multimodal images and using a generative adversarial network algorithm to optimize the common features, performing information fusion on the multimodal images to generate comprehensive image data; S3: Reconstructing a three-dimensional bone structure model based on the comprehensive image data; S4: Based on the three-dimensional bone structure model, using the nonlinear finite element method to simulate the mechanical property parameters of the three-dimensional bone structure model under different external loads and physiological states and output the mechanical properties of the bone tissue; S5: Based on the three-dimensional bone structure model and the mechanical properties of the bone tissue, using the deep variational inference algorithm to generate a personalized bone biomechanics model. The present invention can provide an accurate personalized bone biomechanics modeling solution.

Description

Technical Field

[0001] The present invention relates to the technical field of bone biomechanics simulation, and particularly relates to a bone biomechanics modeling method and system based on multimodal images. Background Art

[0002] Evaluating fracture risk or formulating and treating the clinical treatment plan after fracture mainly depends on the biomechanical properties of bone structure. Since there are few human bone samples available for experiments, it is unrealistic or limited to study the biomechanical properties of bone structure by means of ex vivo experiments. The biomechanical model established based on medical images can simulate various complex stress conditions of bone structure and can repeat experiments infinitely. Therefore, it has become a simple and fast method for analyzing bone mechanical properties.

[0003] Traditional bone structure reconstruction methods mostly rely on two-dimensional image data or single-modal images for bone modeling, lacking accurate restoration of the refined features of bone structure. Traditional mechanical property simulation methods usually based on idealized bone tissue properties, and do not fully consider individual differences, resulting in large deviations in the generated bone biomechanical models in clinical applications. The existing three-dimensional modeling and mechanical property simulation processes lack the support of advanced technologies such as deep learning, and cannot dynamically optimize and adjust the bone structure model, thus affecting the accuracy and reliability of the simulation results. The existing technologies face the following main problems.

[0004] (1) Insufficient accuracy: The existing three-dimensional bone structure reconstruction methods fail to fully extract the subtle bone structure features, resulting in insufficient spatial resolution and detail performance of the bone model.

[0005] (2) Lack of personalization: The existing mechanical property simulations often ignore individual differences and are difficult to generate personalized bone biomechanical models that conform to the physiological characteristics of each patient.

[0006] (3) Lack of effective non-linear simulation: Most of the mechanical property simulation methods of the existing technologies rely on linear assumptions and fail to accurately simulate the non-linear deformation and stress distribution of bone structure under complex physiological conditions. Summary of the Invention

[0007] In order to solve the above technical problems, the purpose of the present invention is to provide a bone biomechanics modeling method based on multimodal images, which provides an accurate personalized bone biomechanics modeling solution based on multimodal image reconstruction, deep learning algorithms and non-linear finite element analysis methods, and provides more efficient support for clinical applications.

[0008] In order to solve the above technical problems, the present invention proposes the following technical solutions to solve:

[0009] The present application relates to a bone biomechanics modeling method based on multimodal images, including:

[0010] S1: Collect the CT image and MRI image of the bone structure to be measured, and generate a multi-modal image;

[0011] S2: Use the deep variational inference algorithm to extract the common features of the multi-modal image, and use the generative adversarial network algorithm to optimize the common features, perform information fusion on the multi-modal image, and generate comprehensive image data;

[0012] S3: Based on the comprehensive image data, reconstruct a three-dimensional bone structure model;

[0013] S4: Based on the three-dimensional bone structure model, use the non-linear finite element method to simulate the mechanical property parameters of the three-dimensional bone structure model under different external loads and physiological states, and output the mechanical properties of the bone tissue;

[0014] S5: Based on the three-dimensional bone structure model and the mechanical properties of the bone tissue, use the deep variational inference algorithm to generate a personalized bone biomechanical model.

[0015] In some embodiments of the present application, S3 specifically includes:

[0016] S31: Based on the comprehensive image data, use the generative adversarial network algorithm to generate a preliminary three-dimensional bone structure model;

[0017] S32: Based on the preliminary three-dimensional bone structure model, perform detail optimization on the local area of the bone structure to obtain a three-dimensional bone structure model after detail optimization;

[0018] S33: Based on the three-dimensional bone structure model after detail optimization, perform spatial resolution optimization on it to obtain an enhanced three-dimensional bone structure model;

[0019] S34: Compare and correct the pre-acquired real clinical data with the enhanced three-dimensional bone structure model, and perform accuracy correction on the key areas of the enhanced three-dimensional bone structure model;

[0020] S35: Output the three-dimensional bone structure model after accuracy correction.

[0021] In some embodiments of the present application, S3 further includes:

[0022] S31': Preprocess the comprehensive image data from S2 to generate preprocessed comprehensive image data for S31.

[0023] In some embodiments of the present application, S31 specifically includes:

[0024] S311: Initialize the parameters of the generator and discriminator in the generative adversarial network;

[0025] S312: The generator extracts multi-level feature information from the comprehensive image data by using a convolutional neural network, and the generator generates a preliminary three-dimensional bone structure model according to the multi-level feature information;

[0026] The discriminator receives the preliminary three-dimensional bone structure model generated by the generator, compares it with the multi-modal images, evaluates the geometric and anatomical consistency of the preliminary three-dimensional bone structure model by using a convolutional neural network, and feeds back the discrimination result to the generator;

[0027] By calculating the results output by the generator and the discriminator, calculate the total loss function L total =L content +λL style where L content is the content loss and L style is the style loss, and λ is the weight coefficient;

[0028] Update the parameters of the generator and the discriminator through the backpropagation algorithm;

[0029] S313: Repeat the training process of S312 until the preliminary three-dimensional bone structure model generated by the generator reaches the expected bone structure data.

[0030] In some embodiments of the present application, S33 uses an adaptive multi-scale convolutional neural network to optimize the spatial resolution of the three-dimensional bone structure model after detail optimization, specifically including:

[0031] S331: Receive the three-dimensional bone structure model after detail optimization, and initialize the convolutional kernel C k and the bias term B k ;

[0032] S332: Extract multi-scale feature maps for the three-dimensional bone structure model after detail optimization based on the adaptive multi-scale convolutional neural network;

[0033] S333: Fuse the feature maps at different scales in a weighted average manner to obtain the fused three-dimensional bone structure model;

[0034] S334: Enhance the spatial resolution of the fused three-dimensional bone structure model based on the adaptive convolutional neural network.

[0035] In some embodiments of the present application, S4 specifically includes:

[0036] S41: Initialize the mechanical property parameters, boundary conditions, and load conditions required for mechanical property simulation;

[0037] Among them, the mechanical property parameters include the Young's modulus, Poisson's ratio, and density of bone tissue, and the boundary conditions include the fixed support condition B fixed and the symmetric boundary condition Bsym , the load conditions include an external load F ext and a compressive load F under physiological conditions compression ;

[0038] S42: Based on the geometric shape of the three-dimensional bone structure model, mesh the bone structure to generate a finite element mesh;

[0039] S43: Receive the finite element mesh and apply boundary conditions B fixed and B sym , and at the same time define the external load F ext and the compressive load F under physiological conditions compression , input into the finite element analysis model: K(G)·U = F ext +F compression , where K(G) is the stiffness matrix after meshing the bone structure, and U is the displacement vector;

[0040] S44: By iteratively solving the nonlinear finite element equation K(U)·U = F(U), simulate the geometric shape and stress distribution of the three-dimensional bone structure model;

[0041] where K(U) is the stiffness matrix in the current displacement state, and F(U) is the external force matrix acting on the bone structure in the current displacement state;

[0042] S44: Output the mechanical properties of the bone tissue: σ = C·ε;

[0043] where σ is the stress tensor, C is the elastic matrix of the material, and ε is the strain tensor.

[0044] In some embodiments of the present application, S44 specifically includes:

[0045] S441: Initialize the parameters required for the nonlinear finite element solution model, and establish the initial stiffness matrix K(U) and external force matrix F(U);

[0046] S442: In each iteration of solving the nonlinear finite element equation using the nonlinear finite element solution model, the stiffness matrix K(U) is updated: K(U) = K0 + △K(U), until the update amount of the displacement vector after iteration satisfies the iteration convergence condition;

[0047] where K0 is the initial stiffness matrix, and △K(U) is the increment of the stiffness matrix updated due to deformation.

[0048] In some embodiments of the present application, S5 specifically includes:

[0049] S51: Receive the stress tensor, strain tensor, and displacement vector of the bone tissue, as well as the three-dimensional bone structure model, and initialize the model parameters of the bone biomechanics model;

[0050] S52: Process the three-dimensional bone structure model and the mechanical properties of its bone tissue using the deep variational inference algorithm to infer the potential variability and uncertainty of the bone tissue, so as to infer the model parameters of the bone biomechanical model;

[0051] S53: Optimize the model parameters of the inferred bone biomechanical model using the global optimization algorithm;

[0052] S54: Generate a personalized bone biomechanical model based on the optimized mechanical properties of the bone tissue and the three-dimensional bone structure model.

[0053] The embodiments of the present invention have the following advantages and beneficial effects:

[0054] (1) Combining multi-modal images, the deep variational inference algorithm and the adversarial network algorithm provides more accurate details for the three-dimensional bone structure model reconstruction;

[0055] (2) Combining with the non-linear finite element method, it realizes the efficient simulation of the mechanical properties of complex bone structures, avoids the limitations based on simplified assumptions in traditional mechanical simulations, and can perform accurate analysis under minute details and complex stress distributions, improving the accuracy and reliability of the mechanical simulation results;

[0056] (3) Using the deep variational inference algorithm to learn the potential distribution of bone tissue, infer the potential variability and uncertainty of bone tissue, and generate a personalized bone biomechanical model that conforms to individual characteristics, so that the generated bone biomechanical model can fully reflect individual differences, meet the clinical personalized needs, and has higher simulation accuracy and stronger adaptability.

[0057] In addition, the present application also relates to a bone biomechanical modeling system based on multi-modal images, including:

[0058] A multi-modal image acquisition module, which is used to acquire CT images and MRI images of the bone structure to be measured and generate multi-modal images;

[0059] An image registration and fusion module, which combines the deep variational inference algorithm and the generative adversarial network algorithm to perform information fusion on the multi-modal images and generate comprehensive image data;

[0060] A three-dimensional bone reconstruction module, which is connected to the image registration and fusion module and reconstructs a three-dimensional bone structure model based on the comprehensive image data;

[0061] A mechanical property simulation module, which is connected to the three-dimensional bone reconstruction module and uses the non-linear finite element method to simulate the mechanical property parameters of the three-dimensional bone structure model under different external loads and physiological states based on the three-dimensional bone structure model, and outputs the mechanical properties of the bone tissue;

[0062] A bone biomechanical construction module, which is connected to the mechanical property simulation module, and generates a personalized bone biomechanical model based on the three-dimensional bone structure model and the mechanical properties of bone tissue by using a deep variational inference algorithm.

[0063] In some embodiments of the present application, the bone biomechanical construction module includes:

[0064] A bone biomechanical model initialization module, which is connected to the mechanical property simulation module, and is used to receive the stress tensor, strain tensor and displacement vector of bone tissue, as well as the three-dimensional bone structure model, and initialize the model parameters of the bone biomechanical model;

[0065] A deep variational inference algorithm module, which is connected to the bone biomechanical model initialization module, and is used to process the three-dimensional bone structure model and the mechanical properties of its bone tissue by using a deep variational inference algorithm, and infer the model parameters of the bone biomechanical model to obtain the potential variability and uncertainty of bone tissue;

[0066] A global optimization algorithm module, which is connected to the deep variational inference algorithm module, and is used to optimize the model parameters of the inferred bone biomechanical model by using a global optimization algorithm;

[0067] A bone biomechanical model generation module, which is connected to the global optimization algorithm module, and is used to generate a personalized bone biomechanical model based on the model parameters of the optimized bone biomechanical model and the three-dimensional bone structure model.

[0068] After reading the specific embodiments of the present invention in conjunction with the accompanying drawings, other features and advantages of the present invention will become clearer. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments of the present invention or the prior art. Obviously, the accompanying drawings described below are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0070] Figure 1 It is a flowchart of the bone biomechanical modeling method based on multi-modal images proposed by the present invention;

[0071] Figure 2 It is a block diagram of the bone biomechanical modeling system based on multi-modal images proposed by the present invention;

[0072] Figure 3 It is a block diagram of the bone biomechanical construction module in the bone biomechanical modeling system based on multi-modal images proposed by the present invention;

[0073] Reference numerals:

[0074] 10. Multimodal image acquisition module; 20. Image registration and fusion module; 30. Three-dimensional bone reconstruction module; 40. Mechanical property simulation module; 50. Bone biomechanics construction module; 51. Bone biomechanics model initialization module; 52. Deep variational inference algorithm module; 53. Global optimization algorithm module; 54. Bone biomechanics model generation module. Detailed implementation manners

[0075] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments.

[0076] Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention. In the description of the present invention, it should be understood that the terms "center", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus cannot be understood as a limitation to the present invention.

[0077] In the description of the present invention, it should be noted that unless otherwise clearly specified and limited, the terms "installed", "connected" and "connected" should be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or an integral connection. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0078] The terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the present invention, unless otherwise stated, the meaning of "plurality" is two or more.

[0079] To provide a bone biomechanics model with high accuracy and meeting personalized requirements, the present application provides a bone biomechanics modeling method and system based on multimodal images. The bone biomechanics modeling method is implemented by the bone biomechanics modeling system. As follows, the bone biomechanics modeling method will be described in conjunction with the bone biomechanics modeling system.

[0080] S1: Collect the CT image and MRI image of the bone structure to be measured, and generate a multi-modal image.

[0081] In some embodiments of the present application, the bone biomechanical modeling system includes a multi-modal image acquisition module 10.

[0082] The multi-modal image acquisition module 10 integrates a computed tomography device and a magnetic resonance imaging device. Among them, the computed tomography device is used to collect the image data of the bone structure to obtain a CT (Computed Tomography) image, and the magnetic resonance imaging device is used to collect the image data of the soft tissue to obtain an MRI (Magnetic Resonance Imaging) image.

[0083] The multi-modal image as described above includes the CT image and MRI image obtained as above.

[0084] S2: Use the deep variational inference algorithm to extract the common features of the multi-modal image, and use the generative adversarial network algorithm to optimize the common features, perform information fusion on the multi-modal image, and generate comprehensive image data.

[0085] In some embodiments of the present application, the bone biomechanical modeling system further includes an image registration and fusion module 20.

[0086] The image registration and fusion module 20 is used to extract the common features of the multi-modal image by using the deep variational inference algorithm, optimize the common features by using the generative adversarial network algorithm, perform information fusion on the multi-modal image, and generate comprehensive image data.

[0087] Both the deep variational inference algorithm and the generative adversarial network algorithm are commonly used algorithms. In the present application, image information is fused based on the deep variational inference algorithm and the generative adversarial network algorithm.

[0088] In some embodiments of the present application, a self-supervised generative adversarial network algorithm can be selected.

[0089] In some embodiments of the present application, the deep variational inference algorithm is responsible for extracting the common features from the multi-modal image (such as the shape and density of the bone, etc., and the common features also include potential common features), and the self-supervised generative adversarial network algorithm is used to optimize these common features, improve the details and consistency of the image, and at the same time make the details clearer.

[0090] Moreover, the deep variational inference algorithm also provides the potential common structures in the multi-modal images, and the self-supervised generative adversarial network algorithm enhances these potential common structures through adversarial training, finally generating more accurate comprehensive image data and improving the accuracy of establishing the three-dimensional bone structure model in the follow-up.

[0091] In addition, the combined use of the deep variational inference algorithm and the self-supervised generative adversarial network algorithm provides a more dynamic and intelligent optimization process.

[0092] S3: Reconstruct the three-dimensional bone structure model based on the comprehensive image data.

[0093] In some embodiments of the present application, the bone biomechanical modeling system further includes a three-dimensional bone reconstruction module 30.

[0094] The three-dimensional bone reconstruction module 30 is connected to the image registration and fusion module 20, receives the comprehensive image data from the image registration and fusion module 20, and reconstructs the three-dimensional bone structure model M based on the comprehensive image data.

[0095] In some embodiments of the present application, the three-dimensional bone reconstruction module 30 uses the generative adversarial network algorithm to reconstruct the three-dimensional bone structure model M.

[0096] In order to improve the modeling accuracy of the three-dimensional bone structure model, in some embodiments of the present application, based on the comprehensive image data, the three-dimensional bone structure model is generated by the generative adversarial network algorithm, and the specific description is as follows.

[0097] S31: Based on the comprehensive image data, use the generative adversarial network algorithm to generate the preliminary three-dimensional bone structure model M input 。

[0098] In some embodiments of the present application, the bone biomechanical modeling system further includes a generative adversarial network three-dimensional modeling module, which receives the comprehensive image data and reconstructs the three-dimensional bone structure model.

[0099] In some embodiments of the present application, in order to improve the modeling accuracy of the three-dimensional bone structure model, before modeling using the comprehensive image data, it is also necessary to preprocess the comprehensive image data.

[0100] In some embodiments of the present application, the bone biomechanical modeling system further includes an image data receiving and processing module, which is connected to the image registration and fusion module 20.

[0101] The image data receiving and processing module receives the comprehensive image data from the image registration and fusion module 20, and preprocesses the received comprehensive image data to generate the preprocessed comprehensive image data.

[0102] In some embodiments of the present application, the generative adversarial network three-dimensional modeling module receives the preprocessed comprehensive image data and reconstructs the three-dimensional bone structure model through the generative adversarial network algorithm, thereby improving the modeling accuracy of the three-dimensional bone structure model.

[0103] Specifically, the generative adversarial network includes a generator and a discriminator. The generator generates a preliminary three-dimensional bone structure model M through a deep convolutional neural network. input The discriminator evaluates the geometric and anatomical consistency of the preliminary three-dimensional bone structure model M input and feeds back information to the generator to optimize the generator, enabling the generator to continuously adjust the three-dimensional bone structure model and optimize the accuracy and details of the model, so that the three-dimensional bone structure model is as close as possible to the comprehensive image data.

[0104] The training process between the generator and the discriminator can be described by the following optimization formula (1):

[0105] (1)

[0106] where L GAN is the loss function of the generative adversarial network, D(x) represents the discriminator's judgment on the real data x, x represents the multi-modal image data, G(z) is the fake data generated by the generator based on the latent variable z, z is generated by using random noise, p data (x) is the real data distribution, p z (z) is the latent variable distribution, and E is the expectation.

[0107] As described above, the generative adversarial network includes a generator and a discriminator. The specific description of using the generator and the discriminator to generate the preliminary three-dimensional bone structure model is as follows.

[0108] The execution process of the generative adversarial network includes an initialization process, a training process, and an iteration process, where the initialization process includes S311 as follows, the training process includes S312 as follows, and the iteration process includes S313 as follows.

[0109] S311: A step of initializing the parameters of the generator and the discriminator in the generative adversarial network.

[0110] In some embodiments of the present application, the bone biomechanical modeling system further includes a generative adversarial network initialization module, which is connected to the image data receiving and processing module and receives the preprocessed comprehensive image data.

[0111] In some embodiments of the present application, the generative adversarial network initialization module sets the initial parameters of the generator and the discriminator by using the Gaussian initialization method.

[0112] S312: The generator and the discriminator perform adversarial training.

[0113] In some embodiments of the present application, the bone biomechanical modeling system further includes a generator network module, which is connected to the generative adversarial network initialization module and uses a deep convolutional neural network to generate a preliminary three-dimensional bone structure model M'.

[0114] Specifically, the generator extracts multi-level feature information of the comprehensive image data through multiple convolutional layers, deconvolutional layers, and activation functions. This multi-level feature information helps the generator understand the details and structure in the comprehensive image data, and constructs a preliminary three-dimensional bone structure model M' based on this feature information.

[0115] In some embodiments of the present application, the bone biomechanical modeling system further includes a discriminator network module, which is connected to both the generator network module and the adversarial network initialization module.

[0116] The discriminator network module receives the preliminary three-dimensional bone structure model M' generated by the generator, compares it with the multi-modal image, uses a deep convolutional neural network to evaluate the geometric and anatomical consistency of the preliminary three-dimensional bone structure model M', and feeds back the discrimination result to the generator.

[0117] During the adversarial training process of the generative adversarial network, it is necessary to calculate the total loss function, which is used to measure the similarity between the three-dimensional bone structure model M' and the comprehensive image data, and continuously and precisely optimize the generator to make the three-dimensional bone structure model output by the generator closer to the real bone structure in terms of anatomical consistency and detail accuracy.

[0118] In some embodiments of the present application, the bone biomechanical modeling system further includes a loss function calculation module, which is connected to the discriminator network module.

[0119] The loss function calculation module obtains the total loss function by calculating the results output by the generator and the discriminator to evaluate the training effect of the generative adversarial network.

[0120] The total loss function includes a content loss L content and a style loss L style , and the content loss L content measures the similarity in geometric structure between the three-dimensional bone structure model M' and the comprehensive image data, usually calculates the error between the two based on the feature extraction layer, while the style loss L style evaluates the consistency of the three-dimensional bone structure model M' and the comprehensive image data in terms of global features, edge information, and spatial distribution, and calculates the difference by means of weighted summation of feature maps.

[0121] Therefore, the total loss function L total can be expressed as follows (2).

[0122] L total =L content +λ·Lstyle (2).

[0123] Among them, λ is a weight coefficient that controls the relative influence of content loss and style loss.

[0124] In some embodiments of the present application, the bone biomechanical modeling system further includes a training optimization module, which is connected to the loss function calculation module.

[0125] The training optimization module updates the parameters of the generator and the discriminator through the backpropagation algorithm, optimizes the generator to make the generated three-dimensional bone structure model closer to the real bone structure data, and the optimization process is adjusted by the gradient descent method.

[0126] S313: Repeat the training process of S312 until the preliminary three-dimensional bone structure model generated by the generator reaches the expected bone structure data.

[0127] In some embodiments of the present application, the bone biomechanical modeling system further includes a three-dimensional bone structure output module, which is connected to the training optimization module and is used to output the three-dimensional bone structure model after multiple trainings and optimizations, that is, the preliminary three-dimensional bone structure model M as described above input , and the optimized three-dimensional bone structure model has higher anatomical consistency and detail accuracy than other models.

[0128] Embodiments of the present application accurately generate a three-dimensional bone structure model through a generative adversarial network, combined with a deep convolutional neural network and multi-level feature information extraction.

[0129] Moreover, the generator and the discriminator are optimized through backpropagation and the total loss function, continuously improving the anatomical consistency and detail accuracy of the three-dimensional bone structure model, solving the accuracy and detail problems in traditional bone structure modeling methods, and being able to generate a high-quality three-dimensional bone structure model closer to the real bone structure, with high accuracy and reliability.

[0130] S32: Based on the preliminary three-dimensional bone structure model M input , perform detail optimization on the local area of the bone structure to obtain a three-dimensional bone structure model with optimized details.

[0131] In order to make the model more refined, in some embodiments of the present application, the bone biomechanical modeling system further includes a local detail optimization module, which is connected to the three-dimensional modeling module of the generative adversarial network.

[0132] In some embodiments of the present application, the local detail optimization module performs detail optimization on the local area of the bone structure based on the preliminary three-dimensional bone structure model M input , to obtain a three-dimensional bone structure model M with optimized details 细节 .

[0133] Specifically, the local detail optimization module is based on the preliminary three-dimensional bone structure model M input , and applies existing image gradient calculation methods and edge enhancement algorithms to optimize the local area of the bone structure, ensuring a more refined model is obtained.

[0134] S33: Based on the three-dimensional bone structure model M after detail optimization input , perform spatial resolution optimization on it to obtain the enhanced three-dimensional bone structure model M enhanced .

[0135] To improve the overall spatial accuracy of the model, in some embodiments of the present application, the bone biomechanical modeling system further includes a spatial resolution enhancement module, which is connected to the local detail optimization module.

[0136] In some embodiments of the present application, the spatial resolution enhancement module performs spatial resolution optimization on the three-dimensional bone structure model M after detail optimization input .

[0137] Specifically, the spatial resolution enhancement module uses the existing adaptive multi-scale convolutional neural network (see formula (3) below) to perform spatial resolution optimization on the three-dimensional bone structure model M input , and optimizes the overall spatial accuracy of the three-dimensional bone structure model M by extracting features and fusing them at different scales input , especially the performance of fine bone structures and minute anatomical details.

[0138] (3)

[0139] Wherein, M enhanced is the enhanced three-dimensional bone structure model, M input is the input three-dimensional bone structure model, C k is the k-th layer convolutional kernel, B k is the k-th layer bias term, K is the number of convolutional layers, and k is the convolutional layer number.

[0140] S331: Receive the three-dimensional bone structure model M after detail optimization input , and initialize the convolutional kernel C k and the bias term B k through the adaptive multi-scale convolutional neural network.

[0141] In some embodiments of the present application, the bone biomechanical modeling system further includes a spatial resolution enhancement initialization module, which is connected to the local detail optimization module.

[0142] The spatial resolution enhancement initialization module receives the three-dimensional bone structure model M after detail optimization input , and initializes the convolutional kernel C through the adaptive multi-scale convolutional neural networkk and the bias term B k .

[0143] S332: Extract multi-scale feature maps from the three-dimensional bone structure model M after detail optimization based on an adaptive multi-scale convolutional neural network input Extract multi-scale feature maps.

[0144] In some embodiments of the present application, the bone biomechanical modeling system further includes a multi-scale feature extraction module, which is connected to the spatial resolution enhancement initialization module.

[0145] The multi-scale feature extraction module performs feature extraction on the three-dimensional bone structure model M after detail optimization based on an adaptive multi-scale convolutional neural network (see formula (4) below). Specifically, through multiple convolutional layers, pooling layers, and deconvolution layers, feature maps containing local texture features, edge features, and global anatomical structure information in the bone structure are extracted at different scales. input F

[0146] = δ(C k * M k + B input ) (4). k

[0147] where F k represents the feature map extracted from the k-th layer, and δ is an activation function used to introduce non-linear feature representation.

[0148] S333: Obtain the fused three-dimensional bone structure model by fusing the feature maps at different scales in a weighted average manner.

[0149] In some embodiments of the present application, the bone biomechanical modeling system further includes a feature fusion module, which is connected to the multi-scale feature extraction module.

[0150] The feature fusion module fuses the feature maps at different scales specifically by performing weighted summation on the feature maps at different scales to synthesize optimized three-dimensional bone structure information with global and local details:

[0151] (5)

[0152] where M fused is the fused three-dimensional bone structure model, ω k is the weighting coefficient of the k-th layer feature map and satisfies , and K is the number of convolutional layers.

[0153] S334: Enhance the spatial resolution of the fused three-dimensional bone structure model M based on an adaptive convolutional neural network fused Perform spatial resolution enhancement.

[0154] ​In some embodiments of the present application, the bone biomechanical modeling system further includes a spatial resolution enhancement module, which is connected to the feature fusion module.

[0155] The spatial resolution enhancement module enhances the spatial resolution of the fused three-dimensional bone structure model M fused Specifically, through multi-layer deconvolution operations and feature reconstruction methods, it optimizes the detailed performance of three-dimensional bone structure information, especially the performance on small-scale bone structures and minute anatomical details. The enhanced three-dimensional bone structure model M enhanced Can be represented by the following formula (6).

[0156] M enhanced = f(M fused , θ) (6).

[0157] Wherein, M enhanced Is the enhanced three-dimensional bone structure model, f(·) is an adaptive convolution operation, and θ is the optimized convolution kernel parameter.

[0158] The embodiments of the present application perform feature extraction and spatial resolution enhancement on the three-dimensional bone structure model through an adaptive multi-scale convolutional neural network, realizing the optimal fusion of local details and global anatomical information of the bone structure, being able to accurately extract and reconstruct the details of small-scale bone structures, improving the resolution and accuracy of the three-dimensional bone structure model reconstruction, and providing high-quality three-dimensional structure data for subsequent mechanical property simulation and personalized bone biomechanical modeling.

[0159] S34: Compare and correct the pre-acquired real clinical data with the enhanced three-dimensional bone structure model M enhanced And perform precision correction on the key areas of the enhanced three-dimensional bone structure model M enhanced

[0160] In some embodiments of the present application, the bone biomechanical modeling system further includes a local precision correction module, which is connected to the spatial resolution enhancement module and receives the enhanced three-dimensional bone structure model M enhanced .

[0161] The local precision correction module uses existing image comparison and correction methods to compare and correct the pre-acquired real clinical data with the enhanced three-dimensional bone structure model M enhanced To optimize the precision of the three-dimensional bone structure model in key areas (such as fracture areas, joint surfaces) and adjust the geometric features of key anatomical positions.

[0162] This image comparison and correction method combines existing image registration and correction techniques to help optimize the anatomical accuracy of the model, making it better adapt to clinical needs.

[0163] ​S35: Output the three-dimensional bone structure model after precision correction.

[0164] In some embodiments of the present application, the bone biomechanics modeling system further includes a three-dimensional bone structure output module, which is connected to the local precision correction module and outputs the optimized three-dimensional bone structure model M. 优化 。

[0165] The optimized three-dimensional bone structure model M 优化 is the three-dimensional bone structure model M as described above.

[0166] In the present application, a three-dimensional bone structure model is generated through a generative adversarial network algorithm, and the geometric and anatomical consistency of the model is optimized through the interaction between the generator and the discriminator. Moreover, the way of reconstructing the model combines a deep convolutional neural network and an optimization feedback mechanism, improving the precision and detail performance of the three-dimensional bone structure model. Especially in the reconstruction of small bone structures and anatomical details, higher modeling precision is achieved, providing a more reliable model basis for subsequent mechanical analysis.

[0167] S4: Based on the three-dimensional bone structure model, use the nonlinear finite element method to simulate the mechanical property parameters of the three-dimensional bone structure model under different external loads and physiological states, and output the mechanical properties of the bone tissue.

[0168] In some embodiments of the present application, the bone biomechanics modeling system further includes a mechanical property simulation module 40, which is connected to the three-dimensional bone reconstruction module 30.

[0169] The three-dimensional bone structure model used in S4 can be the optimized three-dimensional bone structure model M as described above. 优化 。

[0170] Among them, the mechanical property parameters include the Young's modulus E, Poisson's ratio υ, and density ρ of the bone tissue, which are used to describe the physical properties of the bone tissue; the mechanical properties refer to the actual mechanical behaviors reflected by the bone structure under the action of external loads when the mechanical property parameters are input, such as mechanical behaviors such as stress and strain, which are used to describe the deformation and bearing capacity of the bone tissue under different external loads.

[0171] S41: Receive the optimized three-dimensional bone structure model M 优化 , and initialize the mechanical property parameters, boundary conditions, and load conditions required for mechanical property simulation.

[0172] Among them, the boundary conditions include the fixed support condition B fixed and the symmetric boundary condition B sym , and the load conditions include the external load F ext and the compressive load F under physiological conditions. compression 。

[0173] In some embodiments of the present application, the mechanical property simulation module 40 further includes a mechanical property simulation initialization module (not shown), which is used to execute S41 as described above.

[0174] S42: Based on the geometric shape of the three-dimensional bone structure model, perform meshing on the bone structure to generate a finite element mesh.

[0175] In some embodiments of the present application, the mechanical property simulation module 40 further includes a mesh generation module (not shown), which is connected to the mechanical property simulation initialization module and performs meshing on the three-dimensional bone structure model M 优化 to generate a finite element mesh G.

[0176] The meshing is based on the geometric shape of the three-dimensional bone structure model, and the mesh G is generated through an adaptive mesh generation method, and the mesh density is optimized in the stress concentration area.

[0177] G = mesh(M 优化 ) (7).

[0178] Wherein, mesh(·) represents the meshing operation.

[0179] S43: Receive the finite element mesh G, and apply boundary conditions B fixed and B sym , and at the same time define the external load F ext and the compressive load F under physiological conditions compression , and input them into the finite element analysis model.

[0180] In some embodiments of the present application, the mechanical property simulation module 40 further includes a boundary condition and load definition module (not shown), which is connected to the mesh generation module.

[0181] The boundary condition and load definition module receives the finite element mesh G, and applies boundary conditions B fixed and B sym , and at the same time defines the external load F ext and the compressive load F under physiological conditions compression , constrains the response of the bone structure, and inputs it into the finite element analysis model (see formula (8)).

[0182] K(G)·U = F ext +F compression (8).

[0183] Wherein, K(G) is the stiffness matrix after meshing of the bone structure, and U is the displacement vector.

[0184] The stiffness matrix is used to analyze the mechanical response of bone tissue when subjected to external forces. By calculating the stiffness matrix, the deformation and stress distribution of bone tissue can be predicted, so as to evaluate its mechanical properties and stability.

[0185] Each element in the stiffness matrix represents the stiffness coefficient of the meshed bone structure, and these coefficients can be calculated from factors such as the Young's modulus E, Poisson's ratio υ, and density ρ of the bone tissue.

[0186] S44: Simulate the geometric shape and stress distribution of the three-dimensional bone structure model by iteratively solving the non-linear finite element equation K(U)·U = F(U).

[0187] In some embodiments of the present application, the mechanical property simulation module 40 further includes a non-linear finite element solving module (not shown), which is connected to the boundary condition and load definition module.

[0188] The non-linear finite element solving module is based on the applied boundary conditions B fixed 、B sym and load conditions F ext 、F compression , and obtains a stable displacement solution by iteratively solving the non-linear finite element equation K(U)·U = F(U), thereby realizing the simulation of the geometric shape and stress distribution of the three-dimensional bone structure model.

[0189] The geometric shape and stress distribution of the three-dimensional bone structure model are essentially to calculate the performance of the bone structure under external loads using mechanical property parameters.

[0190] Among them, K(U) is the stiffness matrix in the current displacement state, and F(U) is the external force matrix acting on the bone structure in the current displacement state.

[0191] In some embodiments of the present application, in order to solve the non-linear finite element equation K(U)·U = F(U) to simulate the geometric shape and stress distribution of the three-dimensional bone structure model, it can be specifically described as follows.

[0192] S441: Initialize the parameters required for the non-linear finite element solving model, and establish the initial stiffness matrix K(U) and external force matrix F(U).

[0193] In some embodiments of the present application, the non-linear finite element solving module further includes a non-linear finite element model initialization module, which is connected to the boundary condition and load definition module.

[0194] The non-linear finite element model initialization module receives the boundary conditions B fixed 、B sym and load conditions F ext 、F compression , and initializes the parameters required for the non-linear finite element solving model, and establishes the initial stiffness matrix K(U) and external force matrix F(U).

[0195] S442: In each iteration process of solving the non - linear finite - element equation using the non - linear finite - element solution model, the stiffness matrix K(U) is updated: K(U)=K0 + △K(U), until the update amount of the displacement vector after iteration meets the iteration convergence condition.

[0196] Among them, K0 is the initial stiffness matrix, and △K(U) is the increment of the stiffness matrix updated due to deformation.

[0197] In some embodiments of the present application, in each iteration, the local stiffness of the material is calculated according to the displacement vector U, and adjusted according to the constitutive model and deformation state of the material. Among them, the constitutive model of the material is updated according to the current stress tensor, strain tensor and displacement vector, and the updated stiffness matrix affects the next solution process.

[0198] In some embodiments of the present application, an iterative method such as the Newton - Raphson method is used to solve the above non - linear finite - element equation.

[0199] In some embodiments of the present application, during the iterative solution process, the displacement update amount after each iteration is monitored, and it is judged whether the iteration converges by setting a convergence threshold ε. When the update amount of the displacement vector satisfies the condition ||U n+1 -U n ||<ε, it is considered that the iteration process converges, and a stable displacement solution is output. Otherwise, continue the iteration until convergence.

[0200] S45: Output the mechanical properties of bone tissue: σ = C·ε.

[0201] In some embodiments of the present application, the mechanical property simulation module 40 further includes a mechanical property output module (not shown), which is connected to the non - linear finite - element solution module and outputs the mechanical properties of bone tissue.

[0202] Among them, σ is the stress tensor, C is the elastic matrix of the material, and ε is the strain tensor.

[0203] Embodiments of the present application can precisely simulate the deformation and stress distribution of bone structures under external loads and physiological conditions through accurate simulation of mechanical property parameters and non - linear finite - element analysis, which helps to generate a high - precision bone biomechanical model.

[0204] And through adaptive meshing and iterative solution, the computational efficiency and accuracy of the model are optimized, providing a more accurate mechanical property analysis tool for orthopedic clinical applications, with the advantages of high efficiency and reliability.

[0205] S5: Based on the three - dimensional bone structure model and the mechanical properties of bone tissue, use the deep variational inference algorithm to generate a personalized bone biomechanical model.

[0206] In some embodiments of the present application, the bone biomechanical modeling system further includes a bone biomechanical construction module 50, which is connected to the mechanical property simulation module 40.

[0207] In some embodiments of the present application, S5 specifically includes the following.

[0208] S51: Receive the stress tensor, strain tensor, displacement vector of the bone tissue, and the three-dimensional bone structure model, and initialize the model parameters of the bone biomechanical model.

[0209] In some embodiments of the present application, the bone biomechanical construction module 50 includes a bone biomechanical model initialization module 51, which is connected to the mechanical property simulation module 40.

[0210] The bone biomechanical model initialization module 51 receives the stress tensor, strain tensor, displacement vector of the bone tissue, and the three-dimensional bone structure model, and initializes the model parameters of the bone biomechanical model.

[0211] Among them, the initialization step includes establishing the preliminary structure of the bone biomechanical model, and setting the initial mechanical property parameters, geometric shape, boundary conditions, and load conditions.

[0212] S52: Process the three-dimensional bone structure model and the mechanical properties of its bone tissue using the deep variational inference algorithm, and infer the model parameters of the bone biomechanical model to obtain the potential variability and uncertainty of the bone tissue.

[0213] In some embodiments of the present application, the bone biomechanical construction module 50 further includes a deep variational inference algorithm module 52, which is connected to the bone biomechanical model initialization module 51.

[0214] The deep variational inference algorithm module 52 processes the three-dimensional bone structure model and the mechanical properties of its bone tissue using the existing deep variational inference algorithm, infers the potential variability and uncertainty of the bone tissue, and incorporates its uncertainty and variability into the bone biomechanical model.

[0215] In the deep variational inference algorithm, by constructing the variational inference loss function L var , the variational Bayesian method is used to infer and optimize the model parameters of the bone biomechanical model to infer and quantify the potential variability and uncertainty of the bone tissue, so that the bone biomechanical model can more accurately simulate the individual differences and the uncertainty in the data, thereby generating a more personalized and reliable bone biomechanical model.

[0216] L var =E[logp(M, σ,ε| )]-KL[q(U, σ,ε)||p(U, σ,ε)] (9).

[0217] In Equation (9), M is a three-dimensional bone structure model, is the model parameter of the bone biomechanical model to be optimized, KL is the Kullback-Leibler divergence, q(·) is the variational posterior distribution, p(·) is the target posterior distribution, and E is the expectation.

[0218] Among them, the model parameters of the bone biomechanical model refer to the latent variables of the bone biomechanical model (including mechanical properties such as stress tensors, strain tensors, displacement vectors, etc.) and the bone tissue material parameters reflecting individual differences (for example, Young's modulus, Poisson's ratio, density). These model parameters are inferred through a deep variational inference algorithm to obtain the latent variability and uncertainty of bone tissue.

[0219] S53: Optimize the model parameters of the inferred bone biomechanical model using a global optimization algorithm.

[0220] The objective function in the global optimization algorithm is Equation (10).

[0221] (10).

[0222] Among them, is the model parameter of the optimized bone biomechanical model. In Equation (10), L var and L in Equation (9) var both refer to the variational inference loss function.

[0223] In some embodiments of the present application, the bone biomechanical construction module 50 further includes a global optimization algorithm module 53, which is connected to the deep variational inference algorithm module 52.

[0224] The global optimization algorithm module 53 globally optimizes the model parameters output by the deep variational inference algorithm module 52 using a global optimization algorithm, and optimizes the mechanical properties of the bone tissue in the bone biomechanical model through a global optimization strategy, so that the generated bone biomechanical model adapts to individual physiological characteristics and clinical needs.

[0225] S54: Generate a personalized bone biomechanical model based on the model parameters of the optimized bone biomechanical model and the three-dimensional bone structure model.

[0226] In some embodiments of the present application, the bone biomechanical construction module 50 further includes a bone biomechanical model generation module 54, which is connected to the global optimization algorithm module 53.

[0227] The bone biomechanical model generation module 54 is used to execute S54.

[0228] Embodiments of the present application optimize the model parameters of the bone biomechanical model for individual physiological characteristics and clinical needs by combining a deep variational inference algorithm and a global optimization algorithm.

[0229] Moreover, by accurately inferring the potential variability and uncertainty of bone tissue, a personalized bone biomechanical model that conforms to individual characteristics is generated, improving the accuracy and personalization of bone structure analysis and providing more efficient support for clinical applications.

[0230] In some embodiments of the present application, the bone biomechanical modeling method further includes: S5': Based on the three-dimensional bone structure model obtained in S3, use existing self-supervised learning methods to perform unlabeled learning on the comprehensive image data, and extract bone structure and mechanical performance features.

[0231] The mechanical features here are the bone mechanical performance features extracted when simulating the geometric shape and stress distribution of the bone structure. For example, the deformation mode and stress concentration area are used to analyze the strength and vulnerability of the bone.

[0232] This mechanical feature is supplementary information, providing more detailed information about the bone stress distribution and deformation, used to enhance the accuracy of the model during the model optimization and personalized adjustment process, providing more detailed data for the generation and optimization of the bone biomechanical model, and ensuring that the model better meets clinical needs.

[0233] In some embodiments of the present application, to verify the feasibility and effectiveness of the present application, the present application is applied to the orthopedic clinical treatment and rehabilitation center of a comprehensive hospital in a certain city. This center is responsible for treating various complex fracture patients, involving a variety of fracture types and significant individual differences among patients. Traditional bone biomechanical analysis methods are difficult to meet the accurate monitoring and personalized treatment needs during the fracture healing process.

[0234] The present application provides accurate bone biomechanical assessments and personalized treatment plans for each patient through a multi-modal image-based bone biomechanical modeling system, thereby effectively improving the treatment effect and recovery speed.

[0235] For convenient testing, some modules of the above-mentioned modeling system are simplified to form a simplified system. The simplified system mainly includes the mechanical property simulation module 40, the bone biomechanical model initialization module 51, the deep variational inference algorithm module 52, the global optimization algorithm module 53, and the bone biomechanical model generation module 54 in the above-mentioned modeling system. Through the collaborative work of these modules, it is possible to calculate and optimize the bone biomechanical model in real time based on the patient's three-dimensional bone structure and mechanical properties, generate personalized bone mechanical parameters, and design a personalized recovery plan for the patient's fracture site.

[0236] During the application process, first, the mechanical property simulation module 40 collects the three-dimensional bone structure model and mechanical property data of the patient. The mechanical property data includes stress tensor, strain tensor, and displacement vector, and the preliminary bone biomechanical model is generated through the bone biomechanics model initialization module 51 (see S51 above).

[0237] Then, the deep variational inference algorithm module 52 uses the deep variational inference algorithm to further infer and optimize the bone biomechanical model, calculates the potential variability and uncertainty of bone tissue, and adjusts the model parameters of the bone biomechanical model according to this information (see S52 above).

[0238] Next, the global optimization algorithm module 53 uses the global optimization algorithm to precisely optimize the bone biomechanical model (see S53 above), so that the generated personalized bone biomechanical model can better conform to the individual physiological characteristics and clinical needs of the patient.

[0239] Finally, the bone biomechanical model generation module 54 outputs the final personalized bone biomechanical model (see S54 above) and provides support for clinical treatment.

[0240] To verify the effectiveness of this modeling method in clinical applications, a 30-day clinical test was conducted in the orthopedic rehabilitation center of a comprehensive hospital in a certain city. The test subjects were 30 patients with different types of fractures, including femoral, tibial, humeral, etc. All patients wore the simplified system described above during the treatment. The simplified system collected the mechanical property data of the fracture site in real time through sensors and calculated the bone biomechanical model of each patient. During the rehabilitation process of the patients, the simplified system adjusted the rehabilitation plan according to the personalized bone biomechanical model and monitored the recovery of the patients in real time. Table 1 shows the comparison data of the effects of this application and traditional treatment methods during the fracture healing process.

[0241] Table 1

[0242]

[0243] It can be seen from the data in Table 1 that after applying this application, the fracture healing rate of the patients has increased significantly, from 65.2% of the traditional treatment method to 89.6%. This shows that this application can optimize the treatment plan through the personalized bone biomechanical model and effectively promote the healing of fractures.

[0244] In addition, the personalized adaptation rate of the treatment plan increased from 70.5% to 94.2%, indicating that the present application can adjust the treatment plan according to the individual differences of patients, making it more targeted and adaptable. The completion rate of the patients' rehabilitation training also increased significantly, from 72.3% to 96.1%, showing that the personalized treatment plan is more operable and effective. More importantly, the pain relief time of the patients was shortened from 14.5 days to 9.2 days, demonstrating the positive impact of personalized treatment on pain relief. The satisfaction rate of the patients with the treatment plan also increased from 71.8% to 93.5%, indicating that the personalized treatment plan of the present application better meets the needs of patients. The error rate decreased from 12.4% to 4.1%, greatly improving the accuracy and reliability of the treatment data.

[0245] Therefore, the bone biomechanical modeling system involved in the present application can effectively solve the problems of insufficient personalized needs, unstable treatment effects, and too long recovery time in fracture rehabilitation of traditional treatment methods. Through real-time bone biomechanical analysis and optimization of personalized treatment plans, the present application significantly improves the treatment effect, shortens the recovery time, and improves the satisfaction of patients, having broad clinical application prospects.

[0246] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A bone biomechanical modeling method based on multimodal images, characterized in that: include: S1: Acquire CT images and MRI images of the bone structure to be tested and generate multimodal images; S2: extracting common features of the multimodal images using a deep variational inference algorithm, optimizing the common features using a generative adversarial network algorithm, performing information fusion on the multimodal images, and generating comprehensive image data; S3: reconstructing a three-dimensional bone structure model based on the comprehensive image data; S4: Based on the three-dimensional bone structure model, a nonlinear finite element method is used to simulate the mechanical characteristic parameters of the three-dimensional bone structure model under different external loads and physiological states, and the mechanical characteristics of the bone tissue are output; S5: Based on the three-dimensional bone structure model and the mechanical properties of bone tissue, a personalized bone biomechanical model is generated using a deep variational inference algorithm, specifically including: S51: receiving the stress tensor, strain tensor and displacement vector of the bone tissue, and the three-dimensional bone structure model, and initializing the model parameters of the bone biomechanics model; S52: Use deep variational inference algorithm to process the three-dimensional bone structure model and the mechanical properties of bone tissue, and infer the model parameters of the bone biomechanical model to obtain the potential variability and uncertainty of bone tissue; S53: optimizing the model parameters of the inferred bone biomechanical model using a global optimization algorithm; S54: Generate a personalized bone biomechanical model based on the optimized model parameters of the bone biomechanical model and the three-dimensional bone structure model.

2. The bone biomechanical modeling method based on multimodal images according to claim 1, characterized in that: S3 specifically includes: S31: Based on the comprehensive image data, a preliminary three-dimensional bone structure model is generated using a generative adversarial network algorithm; S32: Based on the preliminary three-dimensional bone structure model, optimizing the details of a local area of ​​the bone structure to obtain a three-dimensional bone structure model after the details are optimized; S33: based on the detail-optimized three-dimensional bone structure model, optimizing the spatial resolution thereof to obtain an enhanced three-dimensional bone structure model; S34: comparing and correcting the real clinical data acquired in advance with the enhanced three-dimensional bone structure model, and performing accuracy correction on key areas of the enhanced three-dimensional bone structure model; S35: Output the accuracy-corrected three-dimensional bone structure model.

3. The bone biomechanical modeling method based on multimodal images according to claim 2, characterized in that: The S3 also includes: S31 ′: pre-process the integrated image data from S2 to generate pre-processed integrated image data for use in S31 .

4. The bone biomechanical modeling method based on multimodal images according to claim 2, characterized in that: S31 specifically includes: S311: Initialize the parameters of the generator and discriminator in the generative adversarial network; S312: The generator extracts multi-level feature information from the comprehensive image data using a convolutional neural network, and the generator generates a preliminary three-dimensional bone structure model according to the multi-level feature information; The discriminator receives the preliminary three-dimensional bone structure model generated by the generator, compares it with the multimodal image, uses a convolutional neural network to evaluate the geometric and anatomical consistency of the preliminary three-dimensional bone structure model, and feeds the discrimination result back to the generator; By calculating the output of the generator and the discriminator, the total loss function L is calculated. total =L content +λL style , where L content is the content loss, L style is the style loss, λ is the weight coefficient; Update the parameters of the generator and discriminator through the back-propagation algorithm; S313: Repeat the training process of S312 until the preliminary three-dimensional bone structure model generated by the generator reaches the expected bone structure data.

5. The bone biomechanical modeling method based on multimodal images according to claim 2, characterized in that: S33 uses an adaptive multi-scale convolutional neural network to optimize the spatial resolution of the 3D bone structure model after detail optimization, including: S331: Receive the three-dimensional bone structure model after detail optimization and initialize the convolution kernel C through an adaptive multi-scale convolutional neural network k and the bias term B k , where K is the number of convolutional layers and k is the convolutional layer number; S332: Extracting multi-scale feature maps from the detail-optimized three-dimensional bone structure model based on an adaptive multi-scale convolutional neural network; S333: fusing feature maps at different scales by weighted averaging to obtain a fused three-dimensional bone structure model; S334: Based on an adaptive convolutional neural network, the spatial resolution of the fused three-dimensional bone structure model is enhanced.

6. The bone biomechanical modeling method based on multimodal images according to claim 1, characterized in that: S4 specifically includes: S41: Initialize the mechanical characteristic parameters, boundary conditions and load conditions required for mechanical characteristic simulation; The mechanical property parameters include Young's modulus, Poisson's ratio and density of bone tissue, and the boundary conditions include fixed support conditions B fixed and symmetric boundary condition B sym , the load conditions include external load F ext and the compressive load F under physiological conditions compression ; S42: Based on the geometric shape of the three-dimensional bone structure model, the bone structure is meshed to generate a finite element mesh; S43: Receive the finite element mesh and apply boundary conditions B fixed and B sym , and define the external load F ext and the compressive load F under physiological conditions compression , input finite element analysis model: K(G)·U=F ext +F compression ; Among them, K(G) is the stiffness matrix of the meshed bone structure, and U is the displacement vector; S44: The geometry and stress distribution of the three-dimensional bone structure model are simulated by iteratively solving the nonlinear finite element equation K(U)·U=F(U); Among them, K(U) is the stiffness matrix under the current displacement state, and F(U) is the external force matrix acting on the bone structure under the current displacement state; S45: Output the mechanical properties of bone tissue: σ=C·ε; Among them, σ is the stress tensor, C is the elastic matrix of the material, and ε is the strain tensor.

7. The bone biomechanical modeling method based on multimodal images according to claim 6, characterized in that: S44 specifically includes: S441: Initialize the parameters required for the nonlinear finite element solution model, and establish the initial stiffness matrix K(U) and external force matrix F(U); S442: In each iteration process of solving the nonlinear finite element equation using the nonlinear finite element solution model, the stiffness matrix K(U) is updated: K(U)=K0+△K(U), until the updated amount of the displacement vector after the iteration meets the iteration convergence condition; Among them, K0 is the initial stiffness matrix, and △K(U) is the increment of the stiffness matrix updated due to deformation.

8. A bone biomechanical modeling system based on multimodal images, characterized in that: include: A multimodal image acquisition module, which is used to acquire CT images and MRI images of the bone structure to be measured and generate a multimodal image; An image registration and fusion module, which combines a deep variational inference algorithm and a generative adversarial network algorithm to fuse information of the multimodal images and generate comprehensive image data; A three-dimensional bone reconstruction module, which is connected to the image registration and fusion module and reconstructs a three-dimensional bone structure model based on the comprehensive image data; A mechanical property simulation module is connected to the three-dimensional bone reconstruction module and, based on the three-dimensional bone structure model, uses a nonlinear finite element method to simulate the mechanical property parameters of the three-dimensional bone structure model under different external loads and physiological states, and outputs the mechanical properties of the bone tissue; A bone biomechanics construction module is connected to the mechanical property simulation module and generates a personalized bone biomechanics model based on the three-dimensional bone structure model and the mechanical properties of bone tissue using a deep variational inference algorithm. The bone biomechanics construction module includes: A bone biomechanical model initialization module, which is connected to the mechanical property simulation module and is used to receive the stress tensor, strain tensor and displacement vector of the bone tissue, and the three-dimensional bone structure model, and initialize the model parameters of the bone biomechanical model; A deep variational inference algorithm module, which is connected to the bone biomechanical model initialization module, and is used to process the three-dimensional bone structure model and the mechanical properties of its bone tissue using a deep variational inference algorithm, and infer the model parameters of the bone biomechanical model to obtain the potential variability and uncertainty of the bone tissue; A global optimization algorithm module, which is connected to the deep variational inference algorithm module and is used to optimize the model parameters of the inferred bone biomechanics model using a global optimization algorithm; The bone biomechanical model generation module is connected to the global optimization algorithm module and is used to generate a personalized bone biomechanical model based on the model parameters of the optimized bone biomechanical model and the three-dimensional bone structure model.

Images