A CT-based artificial intelligence method for predicting spinal refracture in patients with OVCF

Abstract

The present invention discloses a prediction method for spinal re-fracture of OVCF patients based on artificial intelligence of CT images. The method is based on artificial intelligence deep learning, and the re-fracture is predicted by comprehensive interpretation of the patient's clinical examination test results and objective results such as bone density, CT imaging genomics, etc. The method includes: information collection, model establishment and model evaluation. The model of the present invention identifies high-risk patients with re-fracture and predicts the risk of re-fracture, evaluates the condition of osteoporosis and finds risk factors related to osteoporosis. The method is based on comprehensive interpretation of the patient's clinical examination test results and objective results such as bone density, CT imaging genomics, etc., to achieve the purpose of predicting re-fracture.

Description

Technical Field

[0001] The present invention relates to the technical field of spinal re-fracture prediction, and in particular to a method for predicting spinal re-fracture of OVCF patients using artificial intelligence based on CT images. Background Art

[0002] There are many auxiliary methods for diagnosing osteoporosis in clinical practice, such as bone mineral density (BMD), X-ray and computed tomography (CT). However, each method has its advantages and disadvantages. At present, the standard diagnosis of osteoporosis is mainly based on bone density of dual-energy X-ray absorptiometry (DXA), which is also an important evidence for predicting fracture risk in clinical practice. In addition to its advantages, it also has some limitations. First, fat can affect the accuracy of X-ray examination. Secondly, it is a two-dimensional projection technology that does not involve bone shape and microstructure. A study of 150,000 postmenopausal women found that 82% of osteoporotic fractures occurred in women who did not meet the DXA osteoporosis criteria. Therefore, bone density is only an important reference indicator and cannot be used as a determining factor. In previous studies, the detection and segmentation of spinal fractures as well as identification and grading were performed by calculating the area loss rate on X-ray images. However, using height loss to determine the presence of fractures is one-sided to a certain extent. Only experienced senior doctors can easily judge whether a patient has osteoporosis based on X-rays. However, this can only be qualitative and cannot be used for quantitative diagnosis or grading. The overlap of bones and soft tissues on X-ray images can lead to inaccurate reports of osteoporosis. CT can intuitively reflect the condition of bones, and some experts recommend using QCT values ​​to assess osteoporosis. However, current research focuses on the use of artificial intelligence and CT to achieve automatic diagnosis of fractures, and there is still a gap between the diagnosis and prediction of re-fractures.

[0003] According to a domestic survey, Chinese citizens have an extremely poor understanding of osteoporosis. The detection rate of osteoporosis in China is much lower than the figures reported in Europe. About 80% of osteoporosis patients fail to receive standardized treatment. Unfortunately, osteoporotic fractures have a cascading effect. Therefore, early screening and intervention for patients with osteoporotic fractures are very important, especially for those high-risk patients who are prone to re-fracture. Summary of the invention

[0004] 1. Technical issues to be resolved

[0005] In view of the shortcomings of the existing technology, the present invention provides an artificial intelligence method based on CT images to predict spinal re-fracture in OVCF patients. The model can achieve the purpose of predicting re-fracture through comprehensive interpretation of the patient's clinical examination results and objective results such as bone density and CT imaging.

[0006] (II) Technical solution

[0007] To achieve the above-mentioned purpose, the present invention provides the following technical solutions: a method for predicting spinal re-fracture in OVCF patients based on artificial intelligence using CT images, which predicts re-fracture by comprehensively interpreting the patient's clinical examination test results and objective results such as bone density and CT imaging, and comprises the following steps:

[0008] S1, information collection, collecting a large amount of patients’ CT image information and clinical information as data input for the model;

[0009] S2. Model establishment. Information of 50 patients was randomly selected. 3DSLICER was used to manually segment the images as the basis for model learning. Then the U-net model was used to automatically segment the ROI. When outlining the ROI area, two ROI outlining methods were selected, one was ROI outlining with the background removed, and the other was ROI outlining with all the background retained. Univariate and multivariate analysis was performed based on the clinical information of the patients to find independent risk factors. Then, three different models were established based on the independent risk factors and the different ROI outlining methods of CT images, including T-P1NT, 3D_Full, and 3D_RoiOnly.

[0010] S3. Model evaluation,Densenet 121-3D model is used to model the cropped region and evaluate the prediction accuracy among three different models.

[0011] Preferably, S1 includes collecting CT images and clinical information of 384 patients from three hospitals, including 240 OVCFs in a training set, 64 in a validation set, and 80 in a test set.

[0012] Preferably, 50 patients' images are randomly selected in S2 for manual outlining, and then the system is allowed to learn, and then the remaining patients' images are automatically outlined. When outlining the ROI, it includes outlining at three different levels of the spinal CT: sagittal, coronal, and axial.

[0013] Preferably, in S3, a ROC curve is constructed to evaluate the diagnosis of the deep learning model, a calibration curve is generated to evaluate the calibration performance, and a decision curve analysis (DCA) is used to evaluate the clinical utility of the prediction model.

[0014] Preferably, the method for using the prediction model includes:

[0015] Step 1: Use univariate and multivariate analysis to find independent risk factors for multiple fractures;

[0016] Step 2: First, the anatomical orientation of the image and label is standardized to conform to the RAS axial encoding; then, the image and label are resampled to achieve a voxel spacing of 1x1x1 mm, using bilinear interpolation for the image and nearest neighbor interpolation for the label; secondly, the intensity value of the image is adjusted to the range [0, 2048] by linear transformation and optional clipping; finally, the background area in the image and label is eliminated based on the foreground mask generated from the source image;

[0017] Step 3: Randomly cut subvolumes from images and labels, taking into account both positive and negative labels. This process includes the specification of spatial size and number of samples, which is adopted during training to provide a different set of images for each training iteration;

[0018] Step 4: Conduct DICE index analysis;

[0019] Step 5: Hyperparameter analysis, using the Adam optimizer with an initial learning rate of 1e-3; the model was trained for approximately 18,000 iterations and stopped early after 32 rounds;

[0020] Step 6: Image post-processing. During the ROI region prediction process, a sliding window of size 96*96*48 is used to process the input data. During the entire prediction process, in some cases, the misidentified voxels are scattered throughout the image space, which is usually called scattering. Since the ROI region usually shows continuity in the input data space, in order to minimize model misidentification, all regions with a volume less than 10000 mm^3 are deleted;

[0021] Step 7. In order to verify the performance of the automatic segmentation algorithm in the following modeling, the ROIs of all samples were not manually marked during data labeling; instead, the automatic segmentation model trained in the previous stage was used to automatically delineate the ROI regions of all samples, which were used for the following modeling; all spinal regions were 3D cropped to obtain the final ROI region.

[0022] Preferably, step one also includes studying whether there are significant differences in nutritional status, electrolyte levels, liver and kidney function between patients with multiple fractures and patients with single fractures.

[0023] Preferably, due to the inherent variability of automatic segmentation, two different ROI region cropping methods were evaluated, including:

[0024] Method 1: The outer boundary cube of the ROI area is represented by 3D\U Full;

[0025] Method 2: remove the background and keep only the ROI area, called 3D\U RoiOnly.

[0026] Preferably, the cropping area is modeled using a Densenet121-3D model, comprising the following steps:

[0027] A1. For the input data of the model, a minimum-maximum transformation is performed to normalize the grayscale values ​​and scale them to the range between -1 and 1;

[0028] A2. Subsequently, each cropped sub-region image is resized to 96x96x48 using the closest interpolation as input to the model. The learning rate is chosen to enhance generalization given the limited availability of image data.

[0029] A3. We constructed ROC curves to evaluate the diagnostics of the deep learning model in the test cohort, generated calibration curves to evaluate the calibration performance of the bar graph, and used the Hosmer-Lemeshow goodness-of-fit test to evaluate its calibration ability.

[0030] A4. Decision curve analysis (DCA) was performed to evaluate the clinical utility of the prediction model.

[0031] (III) Beneficial effects

[0032] Compared with the prior art, the present invention provides an artificial intelligence method based on CT images to predict spinal re-fracture in OVCF patients, which has the following beneficial effects: the model of the present invention predicts the possibility of re-fracture in the patient by identifying high-risk patients with re-fracture, and implements personalized intervention and follow-up to achieve the purpose of treatment and reduce the incidence of re-fracture. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A flow chart of the model building method of the present invention. DETAILED DESCRIPTION

[0034] In order to better understand the purpose, structure and function of the present invention, the following is a further detailed description of a method for predicting spinal refracture in OVCF patients based on artificial intelligence using CT images in conjunction with the accompanying drawings and specific embodiments.

[0035] Example 1

[0036] See also Figure 1 The present invention is a method for predicting spinal re-fracture in OVCF patients based on artificial intelligence using CT images. The method uses an artificial intelligence deep learning method to predict re-fracture by comprehensively interpreting the patient's clinical examination test results and objective results such as bone density and CT imaging, and includes the following steps:

[0037] S1, information collection, collecting a large amount of patients’ CT image information and clinical information as data input for the model;

[0038] S2. Model establishment: 50 patients were randomly selected and manually classified using 3DSLICER.

[0039] The segmented image was used as the basis for model learning, and then the U-net model was used to automatically segment the ROI. When outlining the ROI area, two ROI outlining methods were selected, one was ROI outlining with the background removed, and the other was ROI outlining with all the background retained. Univariate and multivariate analysis was performed based on the patient's clinical information to find independent risk factors. Then, three different models were established based on the independent risk factors and the different ROI outlining methods of CT images, including T-P1NT, 3D_Full, and 3D_RoiOnly;

[0040] S3. Model evaluation,Densenet 121-3D model is used to model the cropped region and evaluate the prediction accuracy among three different models.

[0041] Specifically, the cropping area in the present invention refers to ROI, the Densenet 121-3D model refers to a specific deep learning analysis method, and different models are analyzed, while T-P1NT is modeled only with clinical indicators, 3D-FULL is the outer boundary cube containing the ROI area in the CT image, where the image coverage includes the background; 3D-ROIONLY refers to the image background removal, leaving only bone. However, the defect of this model is that it may be accompanied by background removal, and part of the bone will be filtered out, resulting in missing images. The present invention evaluates the diagnosis of the deep learning model by constructing an ROC curve.

[0042] Furthermore, S1 included the collection of 384 CT images and clinical information from three hospitals, including 240 OVCFs in the training set, 64 in the validation set, and 80 in the test set. In S2, 50 patients’ images were randomly selected for manual outlining, and then the system was allowed to learn, and then the remaining patients’ images were automatically outlined. When outlining the ROI, it included outlining at three different levels of the spinal CT: sagittal, coronal, and axial. In S3, the ROC curve was constructed to evaluate the prediction accuracy of the deep learning model, and a calibration curve was generated to evaluate the calibration performance. Decision curve analysis (DCA) was used to evaluate the clinical utility of the prediction model.

[0043] In the present invention, decision curve analysis (DCA) is used to evaluate the clinical utility of the prediction model. The DCA curve is a supplementary analysis of ROC, which can better illustrate the accuracy of diagnostic prediction between different models. The area under the ROC curve is a commonly used conventional analysis method generally used to evaluate whether the diagnosis of different models is statistically significant. The closer it is to 1, the higher the accuracy. The diagnosis of the deep learning model is evaluated by constructing the ROC curve.

[0044] Example 2

[0045] The method for using the prediction model of the present invention includes:

[0046] Step 1: Use univariate and multivariate analysis to find independent risk factors for multiple fractures;

[0047] Step 2: First, we normalized the anatomical orientation of the images and labels to conform to the RAS axial encoding. Subsequently, we resampled the images and labels to achieve a voxel spacing of 1x1x1mm, using bilinear interpolation for images and nearest neighbor interpolation for labels. Third, we adjusted the intensity values ​​of the images to the range [0, 2048] by linear transformation and optional clipping. Finally, we eliminated background areas in the images and labels based on the foreground mask generated from the source image.

[0048] Step 3: Randomly cut subvolumes from images and labels, taking into account both positive and negative labels. This process includes the specification of spatial dimensions and number of samples. Online data augmentation techniques such as spacing adjustment and random cropping are employed during training to provide a different set of images for each training iteration.

[0049] Step 4: Conduct DICE index analysis.

[0050] Step 5. Hyperparameter analysis. We use the Adam optimizer with an initial learning rate of 1e-3. The model is trained for approximately 18,000 iterations (equivalent to 600 calendar elements) and stops early after 32 epochs.

[0051] Step 6. Image post-processing. During the ROI region prediction process, we use a sliding window of size 96*96*48 to process the input data. Throughout the prediction process, in some cases, the misidentified voxels are scattered throughout the image space, which is often called scattering. Since the ROI region usually exhibits continuity in the input data space, in order to minimize model misidentification, we choose to delete all regions with a volume less than 10000mm^3.

[0052] Step 7. In order to verify the performance of the automatic segmentation algorithm in the following modeling, we did not manually mark the ROIs of all samples during data labeling; instead, we used the automatic segmentation model trained in the previous stage to automatically delineate the ROI regions of all samples, which were used for the following modeling. We performed 3D cropping on all spine regions to obtain the final ROI regions. Due to the inherent variability of automatic segmentation, we evaluated two different ROI region cropping methods: 1. The outer bounding cube of the ROI region, represented by 3D\U Full. 2. Remove the background and keep only the ROI region, called 3D\U RoiOnly. For more information on specific data formats, see the following sections:

[0053] Step 8. We modeled the cropped regions using the Densenet121-3D model. For the input data of the model, we performed a min-max transformation to normalize the grayscale values, scaling them to a range between -1 and 1. Subsequently, we resized each cropped sub-region image to 96x96x48 using the closest interpolation as the input to the model. Considering the limited availability of image data, we carefully selected the learning rate to enhance generalization. For this purpose, we implemented the cosine decay learning rate algorithm.

[0054] Step 9. Evaluate the diagnosis of the deep learning model in the test cohort by constructing an ROC curve. We generated a calibration curve to evaluate the calibration performance of the bar graph and used the Hosmer-Lemeshow goodness-of-fit test to assess its calibration ability. In addition, we performed a decision curve analysis (DCA) to evaluate the clinical utility of the prediction model.

[0055] Patients with a history of fragility fractures are more likely to have another fracture in the future. The more frequently fractures occur, the greater the risk of subsequent fractures. Especially within 1-2 years after the initial fracture, the patient's risk of another fracture increases significantly, which is called "urgent fracture risk." After the initial fracture, the patient's physical condition may undergo some changes, which sows the seeds for future fractures. In this case, it is very helpful to explore the clinical and imaging characteristics of patients with re-fractures to determine which patients are more likely to have re-fractures.

[0056] Although a large number of studies have reported on intelligent diagnosis of osteoporosis, there is still a gap between the diagnosis of osteoporotic fractures and the prediction of re-fractures. Carmelo Messina conducted a study using artificial intelligence combined with dual-energy X-ray absorptiometry (DXA) and bone strain index (BSI) to identify patients at high risk of recurrent fractures. At the same time, some researchers use deep learning technology combined with DXA to measure bone density to determine osteoporosis, claiming to achieve 88% accuracy. However, due to its technical defects that may lead to bias, DXA alone is not suitable for diagnosing osteoporosis. First, fat can affect the accuracy of X-rays. Second, it is a two-dimensional projection technology that does not involve the shape and microstructure of bones. A study of 150,000 postmenopausal women found that 82% of osteoporotic fractures occurred in women who did not meet the DXA osteoporosis criteria. This result means that although BMD alone is an important evidence of osteoporosis in clinical practice, it cannot be used to identify and screen patients at high risk of re-fracture.

[0057] Some risk factors include age, smoking, and some blood test results that have been repeatedly studied in previous studies, rather than recruiting bone turnover markers (BTMs). Although BTMs cannot be used for the diagnosis of osteoporosis, they play an important role in predicting fracture risk and monitoring treatment compliance. However, there is no such study to investigate the role of BTMs in predicting a high-risk re-fracture population. In addition, osteoporosis is a systemic disease. Dysfunction of other organs will inevitably change accordingly. In particular, the interaction between organs may also lead to secondary osteoporosis, such as liver or kidney dysfunction. Therefore, BTMs and some other osteoporosis-related blood test results were included in our study. Our study found that even with multiple fractures, most blood system tests showed no significant difference compared with a single fracture, except for one indicator, T_P1NT. T_P1NT is an early assessment of fracture risk in osteoporosis patients. The higher the cross-overlap and T_P1NT levels, the higher the fracture risk. A study on forearm fractures reported that T_P1NT was significantly higher in the fracture group compared with the control group and should be considered the best indicator of fracture. However, no study has yet explored whether T_P1NT can predict the risk of re-fracture. Based on our findings, we believe that T_P1NT is an independent risk factor for re-fracture and can help provide early feedback on osteoporosis treatment and improve physicians' awareness of protecting these patients.

[0058] Radiology combined with artificial intelligence (AI) is an important method for diagnosing osteoporosis and fractures. Some scholars have used machine learning to establish an X-ray-based model to predict patients at high risk of fracture. In fact, due to the influence of surrounding soft tissue and intestinal air, X-rays cannot truly reflect bone mass. In addition, X-rays can only be qualitative and cannot be quantitatively diagnosed or graded. Due to the advantage of directly reflecting bone quality, computed tomography (CT) has been widely used to diagnose OVCF. In a balanced validation set of 250 CT scans, the deep learning model detected OVCF with an accuracy of 89.1%, a sensitivity of 85.2%, and a specificity of 93.8%. It is beneficial to use artificial intelligence and CT imaging to achieve automatic diagnosis of fractures. Tomita et al. detected OVF based on CT using a CNN-based feature extraction module and an RNN module, which matched the level performance of practicing radiologists. However, there is no similar study to screen high-risk refracture populations based on CT. CT can clearly see sparse, thin, and wide trabeculae. Several studies have used Hu values ​​to determine osteoporosis, demonstrating the predictive value of CT for osteoporosis. Therefore, we constructed a new model for screening high-risk refracture based on CT images.

[0059] This study utilized deep learning methods and data from three medical centers, namely training, validation, and test sets. In our study, two deep learning models based on CT images (ROI-only model and 3D-FULL model) and one model based on T-P1NP were established. Among the three different modeling groups, the 3D-full model was the most accurate. Although T-P1NP was an independent risk factor for recurrent fracture, its prediction accuracy was not as high as that of the 3D-full model, indicating that imaging can better predict refracture than BTMs. When the background was masked in the ROI-only model, part of the bone was also masked, resulting in incomplete image analysis. The 3D-FULL group completely avoided this problem and obtained more comprehensive imaging features. Compared with previous studies, the 3D-full model was able to automatically segment vertebrae, detect OVCF, and automatically predict and identify high-risk refracture populations. In addition, the prediction accuracy was also higher than that of primary clinicians, which makes our model more suitable for real-world translation.

[0060] It is understood that the present invention is described by some embodiments, and it is known to those skilled in the art that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teachings of the invention, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the scope of protection of the present invention.

Claims

1. A method for predicting spinal refracture in OVCF patients based on artificial intelligence using CT images, characterized in that: The steps include: S1, information collection, collecting a large amount of patients’ CT image information and clinical information as data input for the model; S2. Model establishment. Information of 50 patients was randomly selected. 3DSLICER was used to manually segment the images as the basis for model learning. Then, the U-net model was used to automatically segment the ROI. When outlining the ROI area, two ROI outlining methods were selected. One was ROI outlining with the background removed, and the other was ROI outlining with all the background retained. Univariate and multivariate analysis was performed based on the clinical information of the patients to find independent risk factors. Three different models were established based on the independent risk factors and the different ROI outlining methods of CT images, including T-P1NT, 3D_Full, and 3D_RoiOnly. S3,model evaluation, uses the Densenet 121-3D model to model the cropped area and evaluate the prediction accuracy between the three different models; The use of predictive models includes: Step 1: Use univariate and multivariate analysis to find independent risk factors for multiple fractures; Step 2: First, the anatomical orientation of the image and label is standardized to conform to the RAS axial encoding; then, the image and label are resampled to achieve a voxel spacing of 1x1x1 mm, using bilinear interpolation for the image and nearest neighbor interpolation for the label; secondly, the intensity value of the image is adjusted to the range [0, 2048] by linear transformation and optional clipping; finally, the background area in the image and label is eliminated based on the foreground mask generated from the source image; Step 3: Randomly cut sub-volumes from images and labels, taking into account both positive and negative labels. This process includes the specification of spatial size and number of samples, which is adopted during training to provide a different set of images for each training iteration. Step 4: Conduct DICE index analysis; Step 5: Hyperparameter analysis, using the Adam optimizer with an initial learning rate of 1e-3; the model was trained for 18,000 iterations and stopped early after 32 rounds; Step 6: Image post-processing: During the ROI region prediction process, a sliding window of size 96*96*48 is used to process the input data. During the entire prediction process, since the ROI region shows continuity in the input data space, all regions with a volume less than 10000 mm^3 are deleted; Step 7. In order to verify the performance of the automatic segmentation algorithm in the following modeling, the ROIs of all samples were not manually marked during data labeling; instead, the automatic segmentation model trained in the previous stage was used to automatically delineate the ROI regions of all samples, which were used for the following modeling; all spinal regions were 3D cropped to obtain the final ROI region; Step 8. The cropped region was modeled using the Densenet121-3D model. For the input data of the model, a minimum-maximum transformation was performed to normalize the grayscale values ​​and scale them to a range between -1 and 1. Subsequently, each cropped sub-region image was resized to 96x96x48 using the closest interpolation as the input of the model. The learning rate was selected to enhance generalization. Step 9: Evaluate the diagnostics of the deep learning model in the test cohort by constructing an ROC curve, generate a calibration curve to evaluate the calibration performance of the bar chart, and use the Hosmer-Lemeshow goodness-of-fit test to evaluate its calibration ability; and perform a decision curve analysis (DCA).

2. The method for predicting spinal refracture in OVCF patients based on artificial intelligence using CT images according to claim 1, characterized in that: The S1 includes the collection of 384 CT images and clinical information, including 240 OVCFs in the training set, 64 in the validation set, and 80 in the test set.

3. The method for predicting spinal re-fracture in OVCF patients based on artificial intelligence using CT images according to claim 1, characterized in that: In S2, images of 50 patients were randomly selected for manual outlining, and the remaining patient images were automatically outlined through artificial intelligence model learning. The ROI outlining included three different levels of sagittal, coronal, and axial planes of spinal CT.

4. The method for predicting spinal re-fracture in OVCF patients based on artificial intelligence using CT images according to claim 1, characterized in that: In S3, the accuracy of the judgment of the deep learning model is evaluated by constructing a ROC curve, a calibration curve is generated to evaluate the calibration performance, and decision curve analysis (DCA) is used to evaluate the clinical utility of the prediction model.

5. The method for predicting spinal re-fracture in OVCF patients based on artificial intelligence using CT images according to claim 1, characterized in that: The step one also includes studying whether there are significant differences in nutritional status, electrolyte levels, liver and kidney function between patients with multiple fractures and single fractures.

6. The method for predicting spinal refracture in OVCF patients based on artificial intelligence using CT images according to claim 1, characterized in that: Two different ROI region cropping methods were evaluated, including: Method 1: The outer boundary cube of the ROI area is represented by 3D\U Full; Method 2: remove the background and keep only the ROI area, called 3D\U RoiOnly.

7. The method for predicting spinal refracture in OVCF patients based on artificial intelligence using CT images according to claim 1, characterized in that: The cropping area was modeled using the Densenet121-3D model, which includes the following steps: A1. For the input data of the model, a minimum-maximum transformation is performed to normalize the grayscale values ​​and scale them to the range between -1 and 1; A2. Then, each cropped sub-region image is resized to 96x96x48 using the closest interpolation as the input of the model. A3. We constructed ROC curves to evaluate the diagnostics of the deep learning model in the test cohort, generated calibration curves to evaluate the calibration performance of the bar graph, and used the Hosmer-Lemeshow goodness-of-fit test to evaluate its calibration ability. A4. Decision curve analysis (DCA) was performed to evaluate the clinical utility of the prediction model.

Images