Osteoporosis fracture risk prediction method and system based on multi-modal radiomics

Abstract

The invention provides an osteoporosis fracture risk prediction method and system based on multi-modal imageomics. The method comprises the following steps: S1, acquiring multi-modal image data of the lumbar vertebra of a user; s2, performing feature correction according to the acquired multi-modal image data to obtain a time-space aligned multi-modal feature tensor; s3, according to the obtained multi-modal feature tensor, the integrity of the bone microstructure at each position is evolved based on the trained bone microstructure evolution model, and a bone microstructure integrity evolution result is obtained; and S4, according to the obtained bone microstructure integrity evolution result, extracting a fracture risk characteristic part, and further performing fracture risk prediction on the fracture risk characteristic part to obtain a fracture risk prediction result. According to the invention, the accuracy and reliability of osteoporotic fracture risk prediction can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of multi-modal imageomics, and in particular to a method and system for predicting osteoporotic fracture risk based on multi-modal imageomics. BACKGROUND

[0002] The currently widely used osteoporotic fracture prediction technology is mainly based on single image modality feature analysis, and this technical route has fundamental limitations. Since different types of medical imaging devices have their own characteristics, CT scanning can clearly show the density distribution characteristics of the skeleton, but the presentation effect of bone marrow microstructure is limited; MRI imaging is good at showing soft tissue details, but it is insufficient in bone density quantification. The inherent defects of this single modality analysis lead to insufficient comprehensive feature information obtained by the prediction model, making it difficult to accurately reflect the real state of the skeleton. It is observed that the osteoporosis or fracture prediction analysis results based on a single image modality are often biased, and this technical limitation makes some high-risk patients' fracture signs be ignored, while some low-risk patients may receive unnecessary preventive treatment, affecting the effect of intelligent medical image analysis technology. SUMMARY

[0003] In view of the above problems, the present application aims to provide a method and system for predicting osteoporotic fracture risk based on multi-modal imageomics.

[0004] The purpose of the present application is achieved by the following technical solutions:

[0005] In a first aspect, a method for predicting osteoporotic fracture risk based on multi-modal imageomics is provided, comprising the following steps:

[0006] S1, acquiring multi-modal image data of the user's lumbar vertebrae, wherein the multi-modal image data includes CT images and MRI images;

[0007] S2, performing feature correction according to the acquired multi-modal image data to obtain a spatio-temporal aligned multi-modal feature tensor;

[0008] S3, based on the obtained multi-modal feature tensor, evolving the bone microstructure integrity at each position based on a trained bone microstructure evolution model to obtain a bone microstructure integrity evolution result;

[0009] S4, extracting a fracture risk feature part according to the obtained bone microstructure integrity evolution result, and further predicting the fracture risk for the fracture risk feature part to obtain a fracture risk prediction result.

[0010] Preferably, the method further comprises:

[0011] S5, generating a personalized intervention scheme according to the fracture risk prediction result.

[0012] Preferably, step S1 comprises:

[0013] obtaining a CT image sequence of the lumbar vertebra of the user;

[0014] obtaining an MRI image sequence of the lumbar vertebra of the user;

[0015] composing multi-modal image data according to the obtained CT image sequence and MRI image sequence.

[0016] Preferably, step S2 comprises:

[0017] performing alignment processing according to the obtained CT image sequence and MRI image sequence, aligning the two based on the position of the spine, to obtain aligned multi-modal image data;

[0018] performing sharpness compensation correction on the CT image sequence, to obtain a corrected CT image sequence;

[0019] performing feature extraction processing according to the corrected CT image sequence and MRI image sequence, to obtain a multi-modal feature tensor of the multi-modal image data, wherein the multi-modal feature tensor includes pixel position-aligned trabecular bone fracture tendency features, microstructure degradation features, and bone marrow fat heterogeneity features.

[0020] Preferably, step S3 comprises:

[0021] wherein the bone microstructure evolution model includes an input layer, a convolutional layer, a convLSTM layer, and an output layer;

[0022] the input layer is used to input the multi-modal feature tensor; the convolutional layer uses a 3D convolution kernel, wherein the size of the convolution kernel is 3x3x3 and the step is 1; the convolutional layer is used to extract spatial features of the multi-modal feature tensor; the convLSTM layer encodes the obtained spatial features, captures time dependence, and evolves the spatial features, wherein an attention mechanism is added in the convLSTM layer; the output layer is used to perform full connection and classification processing on the evolved spatial features, to obtain a bone microstructure integrity evolution result.

[0023] wherein the bone microstructure integrity evolution result is in the form of a bone microstructure integrity evolution heat map, and the bone microstructure integrity evolution result at different positions is represented by the heat map.

[0024] In a second aspect, a bone fracture risk prediction system based on multi-modal imageomics is provided, including an acquisition module, a correction module, an evolution module, and a prediction module; wherein,

[0025] the acquisition module is used to obtain multi-modal image data of the lumbar vertebra of the user, wherein the multi-modal image data includes CT images and MRI images;

[0026] The correction module is configured to perform feature correction according to the obtained multi-modal image data, to obtain a spatio-temporally aligned multi-modal feature tensor;

[0027] The evolution module is configured to perform evolution of bone microstructure integrity at each position based on the obtained multi-modal feature tensor and the trained bone microstructure evolution model, to obtain a bone microstructure integrity evolution result.

[0028] The prediction module is configured to extract a bone fracture risk feature part according to the obtained bone microstructure integrity evolution result, and further perform bone fracture risk prediction on the bone fracture risk feature part, to obtain a bone fracture risk prediction result.

[0029] Preferably, the system further comprises a generation module.

[0030] The generation module is configured to generate a personalized intervention scheme according to the bone fracture risk prediction result.

[0031] The present application has the following advantages: the present application collects multi-modal image data of the lumbar vertebrae of a user based on a multi-modal image group, and first performs feature correction according to the collected CT image data sequence and MRI image data sequence, extracts corresponding features based on the image data sequence to form a multi-modal feature tensor, which serves as the basis for subsequent analysis and prediction. Based on the obtained multi-modal feature tensor, further evolution prediction of the integrity of bone microstructure at different positions is performed based on a bone microstructure evolution model, which can improve the accuracy and objectivity of the evolution prediction of bone microstructure integrity. According to the bone microstructure integrity evolution result, further extraction of a bone fracture risk feature part and further bone fracture risk prediction are performed, to obtain a bone fracture risk prediction result. The multi-modal image data obtained based on the multi-modal image group is used as the basis, and the extracted multi-modal features are used to represent the bone state, which can truly and accurately reflect the real situation of the bone. At the same time, based on the proposed bone microstructure integrity evolution and bone fracture risk prediction scheme, the bone abnormality can be predicted by evolution, which can help to improve the accuracy and reliability of the osteoporosis and bone fracture risk prediction, thereby improving the effect of medical image analysis. BRIEF DESCRIPTION OF DRAWINGS

[0032] The present application is further illustrated by the accompanying drawings, but the embodiments in the drawings do not constitute any limitation on the present application. For those skilled in the art, other drawings can be obtained without creative labor on the basis of the following drawings.

[0033] Figure 1 A method schematic diagram of the osteoporosis and bone fracture risk prediction method based on multi-modal image group in the embodiments of the present application is shown.

[0034] Figure 2A schematic diagram of a framework structure of an osteoporosis fracture risk prediction system based on multi-modal imageomics shown in an embodiment of the present application. DETAILED DESCRIPTION

[0035] The present application is further described in conjunction with the following application scenarios.

[0036] Reference is made to Figure 1 which shows an osteoporosis fracture risk prediction method based on multi-modal imageomics, comprising the following steps:

[0037] S1 acquiring multi-modal image data of the lumbar vertebrae of a user, wherein the multi-modal image data comprises CT images and MRI images;

[0038] S2 performing feature correction according to the acquired multi-modal image data to obtain a spatio-temporal aligned multi-modal feature tensor;

[0039] S3 performing evolution of the bone microstructure integrity at each position based on the trained bone microstructure evolution model according to the obtained multi-modal feature tensor to obtain a bone microstructure integrity evolution result;

[0040] S4 extracting a fracture risk feature part according to the obtained bone microstructure integrity evolution result, and further performing fracture risk prediction for the fracture risk feature part to obtain a fracture risk prediction result.

[0041] The above embodiment of the present application collects multi-modal image data of the lumbar vertebrae segment of a user based on multi-modal imageomics, first performs feature correction according to the collected CT image data sequence and MRI image data sequence, extracts corresponding features based on the image data sequence to form a multi-modal feature tensor, which serves as the basis for subsequent analysis and prediction. Based on the obtained multi-modal feature tensor as the basis, further evolution prediction of the integrity of the bone microstructure at different positions is performed based on the bone microstructure evolution model, which can improve the accuracy and objectivity of the bone microstructure integrity evolution prediction. According to the bone microstructure integrity evolution result, further extraction of the fracture risk feature part and further fracture risk prediction are performed to obtain a fracture risk prediction result. Through the above multi-modal image data based on multi-modal imageomics as the basis, and the multi-modal features extracted to represent the bone state, the real situation of the bone can be truly and accurately reflected. At the same time, based on the proposed bone microstructure integrity evolution and fracture risk prediction scheme, the bone abnormality can be predicted through evolution, which helps to improve the accuracy and reliability of osteoporosis and fracture risk prediction, thereby improving the effect of medical image analysis.

[0042] Preferably, step S1 comprises:

[0043] acquiring a CT image sequence of the lumbar vertebrae segment of the user;

[0044] Obtaining a sequence of MRI images of a lumbar segment of a user;

[0045] According to the obtained CT image sequence and MRI image sequence, multi-modal image data is formed.

[0046] In one scenario, the CT image data of the lumbar segment of the user is obtained by a 4D-CT device; and the MRI image data of the lumbar segment of the user is obtained by a 7T-MRI device.

[0047] In one scenario, the spatial resolution of the CT image data obtained is 0.3*0.3*0.5mm 3 ; the temporal resolution is 2-4 respiratory phases; and the MRI image data obtained includes a pixel-level fat fraction (FF) map generated based on MRI multi-echo DIXON technology and an IDEAL-IQ sequence.

[0048] Preferably, the step S2 comprises:

[0049] According to the obtained CT image sequence and MRI image sequence, alignment processing is performed, the two are aligned based on the position of the spine, and multi-modal image data after alignment is obtained;

[0050] The CT image sequence is subjected to sharpness compensation correction, and a corrected CT image sequence is obtained;

[0051] According to the corrected CT image sequence and the MRI image sequence, feature extraction processing is performed, and a multi-modal feature tensor of the multi-modal image data is obtained, wherein the multi-modal feature tensor includes a trabecular bone fracture tendency feature, a microstructure degradation feature, and a bone marrow fat heterogeneity feature.

[0052] In one scenario, the obtained CT image sequence and MRI image sequence can be synchronously collected through CT and MRI multi-modal fusion technology under the support of a device, such as an integrated PET / MR device that can simultaneously obtain a CT equivalent attenuation correction map and an MRI anatomical image, so as to obtain the aligned CT image sequence and MRI image sequence. Or the CT image sequence and the MRI image sequence are collected respectively, and the CT and MRI data are aligned through image fusion technology (such as a weighted average method).

[0053] In one scenario, the trabecular bone fracture tendency feature ftbt is mainly evaluated through high-resolution CT data, image texture features are extracted based on CT values in the CT image data, the obtained texture features are used to analyze fracture points, and the trabecular bone fracture tendency feature is obtained.

[0054] In one scenario, the microstructure degradation feature ms is measured by the CT image on the volume bone density of a specified region, and voxel-level bone density distribution data is obtained.

[0055] In one scenario, the marrow fat heterogeneity feature FFC is obtained by a pixel-level fat fraction (FF) map generated by MRI multi-echo DIXON technology.

[0056] In one scenario, the obtained multi-modal feature tensor is Ψ = {ftbt, ms, FFC, t}, where t represents the corresponding time.

[0057] Based on the proposed trabecular fracture tendency feature, microstructure degradation feature, and marrow fat heterogeneity feature, a multi-modal feature tensor is composed, which can comprehensively reflect the current state features of the skeleton through the multi-modal feature tensor, and serve as a basis for subsequent evolution and prediction analysis, thereby improving the accuracy and effect of subsequent analysis.

[0058] Preferably, in step S2, the sharpness compensation correction is performed on the CT image sequence, specifically including:

[0059] The cancellous bone region and the cortical bone region are extracted from the aligned MRI image sequence, and the cancellous bone magnetic resonance transverse relaxation time T2* parameter and the cortical bone magnetic resonance transverse relaxation time T2* parameter are calculated based on different regions.

[0060] The rib position is located according to the aligned CT image sequence, and the rib displacement vector parameter is tracked, and the rib displacement vector parameter is obtained.

[0061] The sharpness compensation correction is performed on the CT image sequence, and the compensation correction function used is:

[0062]

[0063] In the formula, Y'(x) represents the CT value of position x in the compensated and corrected CT image; Y(x) represents the CT value of position x in the CT image, exp() represents the exponential function with the natural constant e as the base, λ represents the set calibration adjustment parameter, t represents the variable, corresponding to the time t, where t = 1, 2, …, N, N represents the total number of CT image sampling times, D rib (t) represents the rib displacement vector at time t compared with the previous time, represents the rib displacement vector per unit time, ‖*‖ represents the norm processing, and represent the cancellous bone magnetic resonance transverse relaxation time T2* parameter and the cortical bone magnetic resonance transverse relaxation time T2* parameter, respectively.

[0064] The sharpness compensation correction of the CT image at each time is completed, and the corrected CT image sequence is obtained.

[0065] ​In view of the fact that the CT image sequence is easily affected by the respiratory motion of the patient in the actual acquisition process, leading to distortion of the measurement of the position of the bone, the accuracy of the feedback of the CT image sequence features (e.g., the bone density measurement features based on the CT image). Therefore, in the above embodiment, a technical solution for compensating and correcting the clarity based on the CT image is particularly proposed, in which the displacement amount of the lumbar spine is tracked, the respiratory motion characteristics (e.g., the superposition of artifacts) are fed back based on the bone displacement amount, and the bone turnover activity is reflected in combination with the T2* parameters of the cancellous bone and cortical bone of the lumbar spine obtained based on the aligned MRI image (to compensate for the noise influence and the feature interference below the feedback resolution), and the respiratory motion characteristics and the bone turnover activity obtained are used as the basis to compensate and correct the CT image, thereby improving the accuracy of the feedback of the CT image features.

[0066] In one scenario, the T2* value parameter obtained has a unit of ms.

[0067] In one scenario, the rib displacement vector parameter is obtained in the following manner: taking the seventh rib as a reference position, positioning the seventh rib position and tracking the displacement amount of the seventh rib based on the aligned CT image sequence to obtain a seventh rib displacement vector parameter as the rib displacement vector parameter. The positioning of the seventh rib based on the CT image sequence can be achieved in the following manner: manually calibrating based on the rib structure, or identifying the rib structure in the CT image based on a trained image processing engine, such as a deep learning network based on RF-NET, and calibrating the corresponding seventh rib position based on the rib structure identification result; and tracking the displacement based on the calibrated seventh rib position to obtain a seventh rib displacement tracking result.

[0068] In one scenario, the value of the calibration adjustment parameter λ is in the range of λ∈[0.01, 0.1], for example, λ=0.05.

[0069] Preferably, step S3 comprises:

[0070] The bone microstructure evolution model comprises an input layer, a convolutional layer, a convLSTM layer, and an output layer.

[0071] The input layer is used to input the multi-modal feature tensor; the convolutional layer uses a 3D convolutional kernel, wherein the size of the convolutional kernel is 3x3x3 and the step is 1; and is used to extract the spatial features of the multi-modal feature tensor; the convLSTM layer encodes the obtained spatial features, captures the time dependence, and evolves the spatial features, wherein an attention mechanism is added in the convLSTM layer; and the output layer is used to perform full connection and classification processing on the evolved spatial features to obtain the bone microstructure integrity evolution result.

[0072] The bone microstructure integrity evolution result is in the form of a bone microstructure integrity evolution heat map, and the bone microstructure integrity evolution results at different positions are represented by the heat map.

[0073] In one scenario, the input layer obtains a multi-modal feature tensor, wherein the size dimension of the multi-modal feature tensor is 512x512xtx3; the input layer first performs Z-score standardization processing on the input data, processes missing data by linear interpolation, and ensures the spatiotemporal continuity of the input features; the convolution layer adopts a three-dimensional convolution structure, is configured with 64 groups of 3x3x3 separable convolution kernels, maintains a spatial dimension of 512x512 resolution, and adopts causal convolution in the time dimension to ensure the temporal causality. The parameterized LeakyReLU activation (α=0.15) and cross-channel batch normalization are introduced, and the convolution kernel is initialized by He normal distribution. The convLSTM layer includes 128 3x3 convolution kernels and is configured with 4 heads of spatiotemporal attention mechanisms. A 0.3 cycle dropout rate and a Hard Sigmoid output gate are adopted, and the time dimension is gradually compressed through cycle calculation. The output layer is connected to a fully connected network after three-dimensional adaptive pooling, and finally outputs a heat map matrix A (bone microstructure integrity evolution heat map) of 512x512xK dimensions, wherein K is the total prediction length, and the matrix element A∈[0,1] quantitatively represents the integrity degree of each spatial position at K prediction time points, 0 represents complete fracture, and 1 represents complete integrity.

[0074] In one scenario, the bone microstructure integrity evolution result is limited to the range of [0, 1] for different positions of the value A, and the hierarchical atlas is used to display the bone microstructure integrity evolution result according to the different value sizes.

[0075] In the training of the bone microstructure evolution model, a large number of key feature data and calibration results extracted from clinical cases are used as training data sets to complete the training of the model, and the test data set is used to further verify and optimize the trained model, and finally the bone microstructure evolution model is obtained.

[0076] Based on the establishment of the three-dimensional convolution layer and the convolution long short-term memory network layer, the multi-modal feature tensor can be adaptively extracted and the bone microstructure feature can be judged, and the evolution analysis of the bone microstructure feature can be realized. Through the bone microstructure evolution model, the bone microstructure features at different positions can be accurately fed back and evolution predicted based on the multi-modal feature tensor, the bone features at different positions can be represented in a dynamic manner, and the fracture and osteoporosis of the bone (bone trabecula) can be accurately evolved, thereby laying a foundation for subsequent extraction of fracture risk feature positions and further fracture risk prediction.

[0077] Preferably, the step S4 comprises:

[0078] According to the obtained bone microstructure integrity evolution result, the bone fracture risk feature position is extracted; wherein according to the bone microstructure integrity evolution result, the position with abnormal result is marked as the bone fracture risk feature position.

[0079] In one scenario, the position with the bone microstructure integrity evolution result not being 1, or the position corresponding to the region not meeting the set standard (such as threshold value, etc.) is marked as the bone fracture risk feature position.

[0080] Preferably, the step S4 comprises:

[0081] According to the obtained bone microstructure integrity evolution result, the bone fracture risk probability of the bone fracture risk feature position is calculated, wherein the bone fracture risk probability prediction function used is:

[0082]

[0083] Wherein, P(x) represents the bone fracture risk probability prediction result of the position x; β1, β2, β3 represent the set reference weight, used to adjust the fitting degree of the prediction result, δ(x) represents the local bone toughness feature of the position x, represents the maximum unit prediction time change rate of the bone microstructure integrity evolution result of the position x with the prediction time, FF(x) represents the fat fraction of the position x, obtained through the MRI image data, γ represents the set fat fraction reference value, σ(x) represents the simulated peak stress of the position x, S0 represents the reference stress value, obtained by extracting the CT data of the same position of the healthy adult;

[0084] According to the bone fracture probability prediction results of different positions, a risk classification atlas is generated, wherein when the bone fracture risk probability prediction result is less than a first threshold value, the corresponding region is marked as low risk (green); when the bone fracture risk probability prediction result is between the first threshold value and a second threshold value, the corresponding region is marked as medium risk (yellow), and when the bone fracture risk probability prediction result is greater than the second threshold value, the corresponding region is marked as high risk (red).

[0085] In one scenario, the first threshold value is set to 0.1, and the second threshold value is set to 0.3;

[0086] According to the risk classification atlas, the low risk region is marked as green, the medium risk region is marked as yellow, and the high risk region is marked as red.

[0087] In the above embodiment, according to the obtained bone microstructure integrity evolution result, a technical scheme for predicting the bone fracture risk probability is proposed, wherein through the proposed bone fracture risk probability prediction function, the local bone toughness feature (the bone degradation phenomenon will be more obvious when the osteoporosis occurs, and the FF value to be fed back will be higher than the normal index) can be comprehensively fed back according to the bone degradation condition and the current actual FF value, and further combined with the simulated peak stress obtained from the CT image to serve as a comprehensive judgment basis, so as to accurately predict the fracture condition of the patient part, which helps to improve the accuracy and reliability of the fracture risk prediction, and the multi-feature comprehensive judgment also improves the objectivity of the fracture risk prediction.

[0088] Preferably, the method further comprises:

[0089] S5 generating a personalized intervention scheme according to the fracture risk prediction result.

[0090] Preferably, the step S5 comprises generating a personalized intervention scheme according to the obtained risk grading atlas and the prediction result of different positions.

[0091] In a scenario, when the bone fracture risk probability of the anterior part of the L3 vertebral body is red high risk, the intervention scheme is generated as targeted strengthening treatment;

[0092] When the comprehensive bone fracture risk probability is medium risk, but there is no local risk prediction as high risk, the intervention scheme is generated as regular calcium supplementation.

[0093] Based on the multi-modal image fusion technology and the feature extraction technology, the multi-modal feature tensor which can truly reflect the bone state is obtained, and through the multi-modal feature fusion mode, the combination of different image data features can be realized, and the representation effect of the multi-modal feature tensor on the bone features is improved.

[0094] Based on the proposed bone microstructure evolution model, the current bone state and the degradation trajectory of the bone tissue can be truly evolved, and the evolution result is represented in the form of a heat map to assist in discovering the potential degradation trend earlier, and accurate data support is provided for subsequent bone fracture risk prediction and targeted intervention. The model continuously learns the evolution law of a large number of clinical cases, and significantly improves the sensitivity and specificity of the prediction.

[0095] Based on the bone microstructure integrity evolution result, the positions that may have risks are marked, and further bone fracture risk probability prediction is performed based on the feature positions, so as to comprehensively evaluate and numerically display the bone fracture risk of the feature positions by combining the evolution result and the current real feature data, which helps to improve the accuracy and reliability of the bone fracture risk prediction, and provides a basis for subsequent further intervention treatment scheme formulation for the risk positions.

[0096] In a second aspect, a bone fracture risk prediction system based on multi-modal imageomics is provided, comprising an acquisition module, a correction module, an evolution module, and a prediction module, wherein

[0097] The acquisition module is configured to acquire multi-modal image data of the lumbar vertebrae of a user, wherein the multi-modal image data comprises CT images and MRI images.

[0098] The correction module is configured to perform feature correction based on the acquired multi-modal image data to obtain a multi-modal feature tensor that is spatio-temporally aligned.

[0099] The evolution module is configured to perform evolution of bone microstructure integrity at each position based on the obtained multi-modal feature tensor and a trained bone microstructure evolution model to obtain an evolution result of the bone microstructure integrity.

[0100] The prediction module is configured to extract a bone fracture risk feature position based on the obtained evolution result of the bone microstructure integrity, and further perform bone fracture risk prediction for the bone fracture risk feature position to obtain a bone fracture risk prediction result.

[0101] Preferably, the system further comprises a generation module.

[0102] The generation module is configured to generate a personalized intervention scheme based on the bone fracture risk prediction result.

[0103] The bone fracture risk prediction system based on multi-modal imageomics provided in the above embodiments can also be used to implement various specific embodiments of corresponding steps in the bone fracture risk prediction method based on multi-modal imageomics as shown in Figure 1 The present application will not be repeated here.

[0104] It should be noted that each functional unit / module in each embodiment of the present application can be integrated in one processing unit / module, or each unit / module can exist physically independently, or two or more units / modules can be integrated in one unit / module. The integrated unit / module can be realized in the form of hardware or in the form of a software functional unit / module.

[0105] Those skilled in the art can clearly understand from the description of the above embodiments that the embodiments described herein can be implemented in hardware, software, firmware, middleware, code or any appropriate combination thereof. For hardware implementation, the processor can be implemented in one or more of the following units: application specific integrated circuit (ASIC), digital signal processor (DSP), digital signal processing device (DSPD), programmable logic device (PLD), field programmable gate array (FPGA), processor, controller, microcontroller, microprocessor, other electronic units designed to implement the functions described herein, or a combination thereof. For software implementation, part or all of the processes of the embodiments can be instructed by a computer program to relevant hardware. When implemented, the above program can be stored in a computer readable medium or transmitted as one or more instructions or codes on a computer readable medium. The computer readable medium includes computer storage medium and communication medium, wherein the communication medium includes any medium facilitating the transmission of computer programs from one place to another. The storage medium can be any available medium accessible by a computer. The computer readable medium can include, but is not limited to, RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer.

[0106] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not limited to the scope of the present application. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should analyze that the technical solutions of the present application can be modified or replaced by equivalents without departing from the essence and scope of the present application.

Claims

1. A method for predicting osteoporotic fracture risk based on multi-modal imaging radiomics, characterized in that, The method comprises the following steps: S1, acquiring multi-modal image data of a user's lumbar vertebrae, wherein the multi-modal image data comprises CT images and MRI images; S2, performing feature correction according to the acquired multi-modal image data to obtain a spatio-temporal aligned multi-modal feature tensor; S3, performing evolution of bone microstructure integrity at each position based on the obtained multi-modal feature tensor and a trained bone microstructure evolution model to obtain a bone microstructure integrity evolution result; S4, extracting a bone fracture risk feature part according to the obtained bone microstructure integrity evolution result, and further performing bone fracture risk prediction on the bone fracture risk feature part to obtain a bone fracture risk prediction result. 2.The method of predicting osteoporotic fracture risk based on multi-modal radiomics according to claim 1, wherein, The method further comprises: S5, generating a personalized intervention scheme according to the bone fracture risk prediction result. 3.The method of predicting osteoporotic fracture risk based on multi-modal radiomics according to claim 1, characterized in that, Step S1 comprises: acquiring a CT image sequence of a user's lumbar vertebrae segment; acquiring an MRI image sequence of the user's lumbar vertebrae segment; composing multi-modal image data according to the obtained CT image sequence and MRI image sequence. 4.The method of predicting osteoporotic fracture risk based on multi-modal radiomics according to claim 1, wherein, Step S2 comprises: performing alignment processing according to the obtained CT image sequence and MRI image sequence, aligning the two based on the position of the spine to obtain aligned multi-modal image data; performing sharpness compensation correction on the CT image sequence to obtain a corrected CT image sequence; performing feature extraction processing on the corrected CT image sequence and MRI image sequence to obtain a multi-modal feature tensor of the multi-modal image data, wherein the multi-modal feature tensor comprises pixel position aligned bone trabecula fracture tendency features, microstructure degradation features and bone marrow fat heterogeneity features. 5.The method of predicting osteoporotic fracture risk based on multi-modal image-based radiomics according to claim 4, characterized in that, In step S2, the sharpness compensation correction on the CT image sequence specifically comprises: The cancellous bone region and the cortical bone region are extracted respectively according to the aligned MRI image sequence, and the magnetic resonance transverse relaxation time T2* parameters of the cancellous bone and the cortical bone are calculated respectively based on different regions and the magnetic resonance transverse relaxation time T2* parameters of the cortical bone locating the rib position and tracking the rib displacement amount according to the aligned CT image sequence to obtain a rib displacement vector parameter; performing sharpness compensation correction on the CT image sequence, wherein the compensation correction function used is: In the formula, Y'(x) represents the CT value of position x in the CT image after compensation correction; Y(x) represents the CT value of position x in the CT image, exp() represents an exponential function with the natural constant e as the base, λ represents a set calibration adjustment parameter, t represents a variable corresponding to the t moment, wherein t = 1, 2,..., N, N represents the total number of CT image sampling moments, D rib (t) represents the rib displacement vector compared with the previous moment at the t moment, represents the rib displacement vector per unit time, and ‖*‖ represents norm processing, and respectively represent the cancellous bone magnetic resonance transverse relaxation time T2* parameter and the cortical bone magnetic resonance transverse relaxation time T2* parameter; performing sharpness compensation correction on each CT image at each time to obtain a corrected CT image sequence. 6.The method of predicting osteoporotic fracture risk based on multi-modal image-based radiomics according to claim 1, wherein, Step S3 comprises: The bone microstructure evolution model comprises an input layer, a convolution layer, a convLSTM layer and an output layer; the input layer is used to input the multi-modal feature tensor; the convolution layer uses a 3D convolution kernel, wherein the size of the convolution kernel is 3×3×3 and the step is 1; the convolution layer is used to extract the spatial features of the multi-modal feature tensor; the convLSTM layer encodes the obtained spatial features, captures time dependence and evolves the spatial features, wherein an attention mechanism is added in the convLSTM layer; the output layer is used to perform full connection and classification processing on the evolved spatial features to obtain a bone microstructure integrity evolution result; The bone microstructure integrity evolution result is in the form of a bone microstructure integrity evolution heat map, which represents the bone microstructure integrity evolution result at different positions through the heat map. 7.The method of predicting osteoporotic fracture risk based on multi-modal image-based radiomics according to claim 1, wherein, Step S4 comprises: According to the bone microstructure integrity evolution result, marking the part with an abnormal result as a bone fracture risk feature part; calculating the bone fracture risk probability of the bone fracture risk feature part position, wherein the bone fracture risk probability prediction function used is: wherein P(x) represents the prediction result of the fracture risk probability at position x; β1, β2, β3 represent the set reference weights for adjusting the fitness of the prediction result; and δ(x) represents the local bone toughness feature at position x, represents the maximum unit prediction time rate of change of the bone microstructure integrity evolution result according to position x with prediction time; FF(x) represents the fat fraction at position x, which is obtained through MRI image data; γ represents the set fat fraction reference value; σ(x) represents the simulated peak stress at position x; and S0 represents the reference stress value, which is extracted through CT data of the same position of a healthy adult; A risk grading atlas is generated according to the fracture probability prediction results of different positions, wherein when the fracture risk probability prediction result is less than a first threshold, the corresponding region is marked as low risk; when the fracture risk probability prediction result is between the first threshold and a second threshold, the corresponding region is marked as medium risk, and when the fracture risk probability prediction result is greater than the second threshold, the corresponding region is marked as high risk.

8. An osteoporotic fracture risk prediction system based on multi-modal imaging radiomics, characterized by, The system comprises an acquisition module, a correction module, an evolution module and a prediction module, wherein The acquisition module is configured to acquire multi-modal image data of the lumbar vertebrae of a user, wherein the multi-modal image data comprises CT images and MRI images; The correction module is configured to perform feature correction according to the acquired multi-modal image data to obtain a spatio-temporal aligned multi-modal feature tensor; The evolution module is configured to perform evolution of the bone microstructure integrity of each position based on the trained bone microstructure evolution model according to the obtained multi-modal feature tensor to obtain a bone microstructure integrity evolution result; The prediction module is configured to extract a bone fracture risk feature part according to the obtained bone microstructure integrity evolution result, and further perform bone fracture risk prediction on the bone fracture risk feature part to obtain a bone fracture risk prediction result. 9.The multi-modal image-based osteoporotic fracture risk prediction system according to claim 8, wherein, The system further comprises a generation module; The generation module is configured to generate an individualized intervention scheme according to the bone fracture risk prediction result.

Images