A multimodal medical imaging data analysis method based on machine learning

Abstract

The present invention belongs to the field of medical image processing technology, and discloses a multimodal medical image data analysis method based on machine learning, and the specific steps include: MRI data collection and preprocessing, MRI feature extraction, clinical diagnosis model of trigeminal neuralgia, and model result analysis. The method of the present invention uses machine learning to extract features from medical images and establish a clinical diagnosis model, demonstrating its potential application value in assisting the diagnosis of complex diseases.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image processing, and in particular relates to a multimodal medical image data analysis method based on machine learning. Background Art

[0002] Trigeminal neuralgia (TN) is a facial neurological disease characterized by severe trigeminal nerve pain. Everyone has two trigeminal nerves, one on the left and right sides of the face, with three branches extending from near the ear to the eye, cheek, and chin area. Although trigeminal neuralgia may affect either side of the face, in most cases the pain is limited to one side. Unlike other types of facial pain, the pain of trigeminal neuralgia is often described as a severe tingling, knife-like, or electric shock-like sensation, so intense that the patient may have difficulty eating or drinking. Pain attacks can quickly spread to the entire face in just a few seconds, and as the disease progresses, the duration of pain may extend to several minutes or even longer.

[0003] Although neurovascular contact of the trigeminal nerve cistern is widely considered to be the main cause of classic trigeminal neuralgia (Classical TN), according to the International Classification of Headache Disorders 3rd edition (ICHD-3), there are still some TN patients whose etiology cannot be directly attributed to the morphological changes of the nerve roots and electrophysiological examinations and magnetic resonance imaging (MRI) do not show obvious abnormalities. Such cases are called idiopathic trigeminal neuralgia (Idiopathic TN). This poses a challenge to clinical diagnosis, because it is often difficult to confirm the diagnosis only by relying on traditional markers and tests.

[0004] Currently, the clinical diagnosis and classification of TN mainly rely on MRI technology and patient-reported pain characteristics. In particular, T2 sequences and 3D Time-of-Flight MRA (3D-TOF-MRA) sequences have shown good performance in identifying neurovascular compression in TN patients, but the performance of MRI technology is often limited by the parameters of the imaging equipment. Currently, the treatment methods for TN are diverse and usually depend on the clinical experience and personal preferences of the physician. Despite efforts to standardize the diagnostic process, the accurate diagnosis, classification, and management of TN have not yet been fully resolved. Many patients still feel unrelieved pain after drug treatment, and some experience unbearable side effects. Common surgical treatments include microvascular decompression (MVD), stereotactic radiosurgery (SR), and rhizotomy, but the high recurrence rate after these treatment interventions highlights the need for a deeper understanding of the pathological mechanisms of TN and the development of treatment strategies.

[0005] With the continuous advancement of medical imaging technology and the continuous development of artificial intelligence (AI)-assisted diagnostic models, AI-based diagnostic support systems have shown great potential in improving diagnostic accuracy and efficiency. In 2022, Kellen L. Mulford et al. collected data from 134 TN patients and developed an innovative TN diagnostic model with the help of machine learning technology. This model focuses on the comparative analysis between the left and right branches of the trigeminal nerve, aiming to identify neurovascular conflicts (NVC) associated with TN. However, although the model has been successful to a certain extent, it still faces significant challenges in distinguishing healthy nerve branches from those diseased branches that do not have neurovascular conflicts in TN patients. The contribution of this study is to use advanced AI technology to explore the automated diagnostic approach of TN, providing a new perspective in the field of medical image analysis and disease diagnosis. However, the complexity of the automated process of identifying neurovascular conflicts suggests the need to further improve the algorithm and enhance the generalization ability of the model in order to more accurately distinguish different types of trigeminal neuropathy. Therefore, this study considers integrating more dimensional data, such as patients' clinical symptoms, pain patterns, and other imaging features, to improve the diagnostic accuracy and reliability of the model.

[0006] In 2023, Julien Calderaro conducted a comprehensive phenotyping of combined hepatocellular-cholangiocarcinoma using deep learning technology, demonstrating that deep learning can distinguish the diagnosis of hepatocellular carcinoma (HCC) and cholangiocarcinoma (ICCA) with high performance. The study analyzed 405 cHCC-CCA patients and proved that the model can reclassify tumors as HCC or ICCA. Most importantly, these predictions are consistent with clinical outcomes, genetic changes, and in situ spatial gene expression analysis, further illustrating the reliability of machine learning in identifying high-dimensional information that is invisible to the human eye.

[0007] In the same year, Zhong's study aimed to develop a PET / CT-based deep learning feature to predict occult lymph node metastasis (ONM) in clinical non-small cell lung cancer. Studies have shown that although PET / CT is the most reliable non-invasive means of clinical staging, 12.9% to 39.3% of lymph node metastases are not identified by this advanced technology, but are accidentally discovered during surgery, the so-called occult lymph node metastasis. Accurate identification of ONM is crucial to determining the best treatment strategy for patients with non-small cell lung cancer. This work has confirmed the high performance of PET / CT-based deep learning features in predicting occult metastasis through comprehensive validation of internal cohorts (1911 cases), external cohorts (355 cases), and prospective cohorts (999 cases). This study also proves that the use of deep learning technology to quantify radiological images and extract detailed feature information exceeds the ability of human vision. At the same time, the multimodal characteristics of PET / CT increase the feature dimension and information richness, and also put forward higher requirements for deep learning algorithms.

[0008] In 2023, Liang et al. used deep learning methods to analyze the acquired functional magnetic resonance imaging to analyze the brain areas associated with TN pain generation and perception, providing target areas for future understanding of the central mechanisms of TN. In 2022, Ge et al. used machine learning to study the microstructural differences between the affected and healthy trigeminal cistern segments of patients, providing a basis for the intelligent diagnosis of trigeminal neuralgia. This study aims to use deep learning and radiomics technology to conduct a comprehensive analysis of multi-source MRI data. By combining the features extracted from different MRI sequences, our goal is to enhance the diagnosis of trigeminal neuralgia and describe the patient's condition more comprehensively and accurately. This effort aims to lay the foundation for personalized clinical interventions and comprehensive treatment plans tailored to the individual needs of TN patients.

[0009] These research results have inspired us that deep learning technology can accurately identify information that is invisible to the human eye in medical images, and prove to have a certain degree of credibility, which far exceeds the ability of human vision, prompting us to apply this technology to the diagnosis of trigeminal neuralgia. This shows that the application of deep learning technology in medical image analysis, especially in precision medicine and personalized treatment decision support, has important potential and value. Summary of the invention

[0010] In order to overcome the problems of the prior art, the present invention proposes a multimodal medical image data analysis method based on machine learning. The method uses machine learning to extract features from medical images and establish a clinical diagnosis model, demonstrating its potential application value in assisting the diagnosis of complex diseases.

[0011] The object of the present invention is achieved in that:

[0012] A multimodal medical image data analysis method based on machine learning comprises the following steps:

[0013] Step 1, MRI data collection and preprocessing:

[0014] S11, image acquisition:

[0015] MRI images of patients with trigeminal neuralgia who underwent surgical treatment were collected as the primary cohort and external validation cohort;

[0016] S12, image segmentation:

[0017] The MRI images acquired by S11 were segmented using ITK-SNAP software, which specifically involved: first, the 3D-TOF images were registered to the T2 images using linear interpolation; then, the T2 images were manually segmented, and the regions of interest (ROIs) were manually outlined on each slice showing the nerves; finally, the volumes of interest (VOIs) were generated by fusing the segmented tissues on each slice; thus, when the VOI segmentation was completed on the T2 image, the corresponding VOIs were automatically generated on the 3D-TOF image.

[0018] Step 2, MRI feature extraction:

[0019] S21, MRI deep feature extraction based on radiomics:

[0020] PyRadiomics software was used to extract radiomic features from MRI image data to quantitatively analyze the trigeminal nerve of each patient. Specifically, several radiomic features were extracted from each patient's T2 image and 3D-TOF image sequence. These features were divided into three categories: shape and volume features, texture features, and first-order statistical features, each of which provided different dimensions of trigeminal nerve characteristic information.

[0021] S22, MRI deep feature extraction based on 3D-ResNet:

[0022] A pre-trained 3D-ResNet-18 model was used, and the last fully connected layer of the model was removed to extract intermediate features for non-final classification. The modified 3D-ResNet-18 model was called to extract features from each three-dimensional image after converting the image and mask into NumPy arrays and normalizing and standardizing them. The feature extraction was focused on the output layer before removing the fully connected layer. Several deep learning features were extracted from the two volumetric regions of interest of each patient, T2 and 3D-TOF, and standardized using the z-score method to ensure that the feature values ​​were within a standardized range of a unified standard.

[0023] Step 3, Clinical Diagnostic Model of Trigeminal Neuralgia:

[0024] S31, feature screening and model building:

[0025] Several feature sets were constructed from the ROIs obtained in step 1: radiomics based on T2 and TOF sequences, and two deep learning feature sets based on T2 and TOF sequences;

[0026] First, each set of features was screened by t-test to exclude variables with no statistically significant differences. Second, the Pearson or Spearman correlation coefficients between all possible feature pairs were calculated, and one feature in any feature pair with a correlation coefficient exceeding 0.9 was removed to eliminate highly correlated variables. Subsequently, the least absolute shrinkage and selection operator LASSO regression method was used to screen features that were closely related to trigeminal neuralgia, and 5-fold cross-validation was applied to optimize the model selection process and improve the generalization ability of the model.

[0027] S32, Model Development:

[0028] Three types of models were developed, including: 1) Single modality model (SM), which consisted of five models based on clinical features and T2 / 3D-TOF Rad and T2 / TOF DL; 2) Pre-fusion model (Pre-FMs), which consisted of six models based on Clinic+T2 / TOF Rad, Clinic+T2 / TOF DL, T2+TOF DL and T2+TOF Rad; 3) Post-fusion model (Pre-FMs), which consisted of probability models based on Clinic+Rad, Clinic+DL and Clinic+Rad+DL;

[0029] Support vector machine was used as the classifier to develop unimodal models and Pre-FMs, and random forest was used as the classifier to develop Post-FMs.

[0030] Step 4: Model result analysis:

[0031] The area under the ROC curve (AUC) was used as the core evaluation indicator to compare the discrimination of different models, and the model with the best classification effect was selected to obtain the final trigeminal neuralgia detection results.

[0032] Specifically, the larger the AUC value, the stronger the discrimination ability of the representative model, and the better the classification effect. The advantages and beneficial effects of the present invention are:

[0033] 1. The present invention uses machine learning to extract features from medical images and establish a clinical diagnosis model. In cooperation with the First Affiliated Hospital of Anhui Medical University, magnetic resonance imaging (MRI) data of 332 patients were retrospectively collected, including imaging data of 225 patients with trigeminal neuralgia and 107 patients with facial spasm. The present invention uses deep learning and imaging genomics methods to extract features from MRI data, and uses these features to construct a multimodal clinical diagnosis model. The research in this application details the process of building a clinical diagnosis model, including key steps such as data preprocessing, model training and verification. By comparing the results of the multimodal prediction models obtained by these two different methods, the present application found that the deep learning method is more superior in extracting medical image features and demonstrated its potential application value in assisting the diagnosis of complex diseases.

[0034] 2. The present invention introduces in detail the preprocessing process for medical imaging technology, especially magnetic resonance imaging, and two advanced feature extraction methods (radiology and deep learning technology). The preprocessing part focuses on the optimization of image quality, including steps such as noise reduction, correction and standardization to ensure the accuracy and reliability of subsequent feature extraction. Two methods are discussed in the feature extraction link: one is feature extraction based on radiomics, such as edge detection and texture analysis; the other is a method based on deep learning, which can automatically learn and recognize complex patterns and features from images. Next, this application deeply explores how to use machine learning technology, especially support vector machine (SVM) to effectively model the extracted features. By constructing a model based on SVM, this application builds a clinical decision-making model for auxiliary diagnosis of trigeminal neuralgia. It can analyze the key features in MRI images and realize rapid and accurate diagnosis of trigeminal neuralgia, thereby providing patients with timely and personalized treatment plans. In addition, this application also discusses the performance evaluation of the model, including indicators such as accuracy, sensitivity and specificity, to verify the diagnostic efficacy and clinical application potential of the model. The best effect of the present invention is the pre-fusion model, with an AUC value of 0.947 in the main cohort and an AUC value of 0.842 in the external validation cohort. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0036] Figure 1 3D-ResNet architecture diagram according to an embodiment of the present invention;

[0037] Figure 2 is a flow chart of the development of a trigeminal neuralgia diagnostic model according to an embodiment of the present invention;

[0038] Figure 3 The ROC curve analysis of the single modality model in PC and EVC according to the embodiment of the present invention is shown;

[0039] Figure 4 The ROC curve analysis of the pre-fusion model in the training set and the validation set is shown;

[0040] Figure 5 The ROC curve analysis of the post-fusion model in the training set and the validation set is shown;

[0041] Figure 6 The calibration curve analysis of the pre-fusion model in the training set and the validation set according to the embodiment of the present invention is shown. DETAILED DESCRIPTION

[0042] Example:

[0043] As mentioned above, this embodiment provides a multimodal medical image data analysis method based on machine learning, and the method includes the following steps:

[0044] Step 1, MRI data collection and preprocessing:

[0045] S11, image acquisition:

[0046] In this embodiment, MRI images of patients with trigeminal neuralgia who received surgical treatment were collected, including 293 patients from the First Affiliated Hospital of Anhui Medical University (High-tech Campus) and 39 patients from the First Affiliated Hospital of Anhui Medical University (Main Campus) as the primary cohort (PC) and external validation cohort (EVC).

[0047] Inclusion criteria were: 1) complete preoperative MRI examination; 2) good image quality (no motion artifacts) for analysis; 3) clear pathological diagnosis.

[0048] Exclusion criteria were: 1) secondary trigeminal neuralgia; 2) missing T2 or 3D-TOF sequence.

[0049] There were 225 TN patients, including 82 males and 143 females, aged 40-83 years (average 63 years). There were 107 patients with facial spasms (HFS), including 27 males and 80 females, aged 15-74 years (average 53 years). The chi-square test and t-test showed no significant difference in the clinical characteristics between PC and EVC. There was no statistical difference in the sample size between PC and EVC for both positive and negative samples (TN and HFS), indicating a balance in sample size between subgroups. The clinical characteristics of PC and EVC are summarized in Table 1.

[0050] Table 1. Clinical characteristics of the primary and external validation cohorts

[0051]

[0052] The MRI images described in this embodiment were obtained using a 3.0T Philips MRI device equipped with a 64-channel head coil, using a magnetization prepared rapid gradient echo (MPRAGE) sequence (repetition time [TR] / echo time [TE] = 2400 / 2.4 ms, flip angle [FA] = 15°, field of view [FOV] = 512 × 512 mm 2, matrix size=512×512, voxel size=1×1×1mm 2 , slice thickness is 1mm). This embodiment uses the N4ITK toolkit in the 3D slicer software to successfully perform bias field correction on all MRI images, ensuring effective improvement of image quality and providing a reliable data basis for subsequent analysis. Bicubic resampling is used to standardize the image size.

[0053] S12, image segmentation:

[0054] The MRI images acquired by S11 were segmented using the open source ITK-SNAP software (http: / / www.itksnap.org), which included the following steps: first, the 3D-TOF images were registered to the T2 images using linear interpolation; then, the T2 images were manually segmented by a brain radiologist, and regions of interest (ROI) were manually outlined on each slice showing nerves (including surrounding veins); finally, a volume of interest (VOI) was generated by fusing the segmented tissue on each slice; thus, when the VOI segmentation was completed on the T2 image, the corresponding VOI was automatically generated on the 3D-TOF image.

[0055] Manual segmentation of the above T2 images was performed by at least two brain radiologists, who were blinded to the clinical and pathological findings and agreed on the radiological features to avoid subjectivity. The trigeminal cistern segment was found to have the highest relevance in the diagnosis, treatment planning, and follow-up of trigeminal neuralgia.

[0056] Step 2, MRI feature extraction:

[0057] S21, MRI deep feature extraction based on radiomics:

[0058] PyRadiomics software was used to extract radiomics features from MRI image data to perform quantitative analysis of the patients' trigeminal nerves. Specifically, a total of 640 radiomics features were extracted from each patient's T2 images and 3D-TOF image sequences. These features were divided into three major categories: shape and volume features, texture features, and first-order statistical features, each providing trigeminal nerve characteristic information in different dimensions.

[0059] First-order statistical features: also called intensity features, are calculated through histogram analysis. They cover the extreme values, variance, kurtosis and other parameters of the pixel grayscale in the region of interest, providing us with a basic description of the pixel intensity distribution in the neural region.

[0060] Shape and volume features: focus on describing the geometric features of the ROI to reflect the spatial size and morphology of the nerve. This includes sphericity and compactness (characteristics describing the shape of the tumor), surface area, volume, and maximum two-dimensional diameter, etc. These features can be used to understand the physical morphology of the nerve from multiple angles.

[0061] Texture features: mainly reveal the spatial relationship between pixels of the same or different grayscale in the region of interest. This type of feature can non-invasively reveal important biological information such as tumor heterogeneity.

[0062] S22, MRI deep feature extraction based on 3D-ResNet:

[0063] Convolutional neural networks have demonstrated their excellent performance in processing and analyzing large amounts of image data.

[56] . The general process of CNN includes convolution, activation and pooling, which work together to extract key feature representations of images. For specific application contexts, these feature representations can be connected to the fully connected layer and used as input to achieve the mapping from image to label. Therefore, the convolution layer plays a key role in mining image features. As mentioned earlier, although increasing the depth of the network often enhances the performance of the model, it will also lead to an increase in the number of model parameters and may trigger the problem of gradient disappearance. In addition, compared with traditional machine learning techniques, deep learning models usually require more data, while in the medical field, the size of the data set is relatively small and is often limited by the requirements of patient privacy protection. To address this limitation, we adopt the method of transfer learning, that is, using the model structure pre-trained on other tasks, or expanding the data set through data augmentation methods, so as to retrain to achieve a better training effect.

[0064] This embodiment uses a pre-trained 3D-ResNet-18 model and removes the last fully connected layer of the model to extract intermediate features that are not final classifications; by converting the image (Image) and mask (Mask) into NumPy arrays, and performing normalization and standardization, the modified 3D-ResNet-18 model is called to extract features from each three-dimensional image, focusing on extracting features from the output layer before removing the fully connected layer. A total of 1024 deep learning features are extracted from the two volumetric regions of interest of each patient, T2 and 3D-TOF, and standardized by the z-score method to ensure that the feature values ​​are within the standardized range of a unified standard. The model structure is as follows Figure 1 shown.

[0065] Step 3, Clinical Diagnostic Model of Trigeminal Neuralgia:

[0066] S31, feature screening and model building:

[0067] Four feature sets were constructed from the two ROIs, respectively: two radiomics based on T2 and TOF sequences and two deep learning feature sets based on T2 and TOF sequences.

[0068] Firstly, each group of features was screened by t-test, and the threshold was set at 0.05 to exclude variables with no statistically significant differences. Secondly, the Pearson or Spearman correlation coefficients between all possible feature pairs were calculated, and one feature in any feature pair with a correlation coefficient exceeding 0.9 would be eliminated to eliminate highly correlated variables. Subsequently, the Least Absolute Shrinkage and Selection Operator (LASSO) regression method was used to screen features closely related to trigeminal neuralgia, and 5-fold cross-validation was applied to optimize the model selection process and improve the generalization ability of the model.

[0069] The sparse property of LASSO regression not only helps to improve model interpretability but also effectively handles the problem of feature selection in high-dimensional data, thereby identifying the most critical features with high predictive value for TN diagnosis.

[0070] S32, Model Development:

[0071] In the past decade, machine learning technology has made significant progress in the analysis and processing of structural and functional MRI imaging data. These advances have provided new references for the diagnosis, prediction, and prognosis of disease treatment effects. By deeply mining and analyzing a large number of medical imaging images, machine learning models can identify disease-specific features and signs, thereby providing clinicians with more accurate and personalized medical decision support. At the same time, these technologies can also predict the development trend of the disease at an early stage, which is of great significance for improving treatment efficiency and improving patient prognosis. In short, the application of machine learning in the field of medical imaging has opened up new avenues for precision medicine and personalized treatment, bringing revolutionary changes to future medical research and clinical practice.

[0072] Three types of models were developed in this embodiment, including: 1) Single Models (SM), which are five models based on clinical features and T2 / 3D-TOF Rad and T2 / TOF DL; 2) Pre-Fusion Models (Pre-FMs), which are six models based on Clinic+T2 / TOF Rad, Clinic+T2 / TOF DL, T2+TOF DL and T2+TOF Rad; 3) Post-Fusion Models (Pre-FMs), which are probability models based on Clinic+Rad, Clinic+DL and Clinic+Rad+DL;

[0073] Support vector machine (SVM) was used as the classifier to develop unimodal models and Pre-FMs, and random forest (RF) was used as the classifier to develop Post-FMs.

[0074] In the primary cohort (PC) described in step 1, we further divided the data into a training cohort and an internal validation cohort. By using 5-fold cross-validation, 4 / 5 samples were randomly defined as the training cohort to train the model, and the remaining 1 / 5 samples were defined as the internal validation cohort to optimize the parameters. After five strict repetitions, the optimal hyperparameter combination was obtained. The performance of the model was then tested on the external validation cohort (EVC). The independence of internal data was strictly guaranteed, and there was no leakage between data. The development process was as follows: Figure 2 shown.

[0075] Step 4: Model result analysis:

[0076] S41, model discrimination:

[0077] The area under the ROC curve (AUC) was used as the core evaluation indicator to compare the discrimination of different models, and the model with the best classification effect was selected to obtain the final trigeminal neuralgia detection results.

[0078] Specifically, the larger the AUC value, the stronger the model's discriminative ability and the better the classification effect. If the AUC value exceeds 0.80, it means that the model has excellent discriminative ability and can accurately distinguish samples of different categories; when the AUC value is between 0.75 and 0.80, although the discriminative ability is slightly weaker, it still performs well and can meet the needs of actual applications; if the AUC value is lower than 0.60, it implies that the model's discriminative and predictive abilities are weak, and there are serious classification errors or deviations, and the model needs to be further adjusted or optimized to improve its performance.

[0079] In this embodiment, the prediction performance of single modality prediction models (SMs) is generally not as good as that of pre-fusion (Pre-FMs). The ROC curves of single modality models (SMs) in PC and EVC are as follows: Figure 3 As shown in the figure, the best model is Model 5 3D-TOF Rad. The AUC values ​​in the training set (PC) and validation set (EVC) were 0.809 [95% CI, 0.738-0.812] and 0.802 [95% CI, 0.742-0.859], respectively. When the weighted Youden index was used to calculate the working point, other indicators reflecting the prediction performance of the model in PC and EVC were: sensitivity 0.618, 0.750, specificity 0.782, 0.645, accuracy 0.737, 0.667, F1 value 0.619, 0.480.

[0080] The ROC curves of the pre-fusion models (Pre-FMs) in PC and EVC are shown in Figure 2. Figure 4 As shown. The performance of the pre-fusion model is generally improved compared with the single-modality model. Among them, the Cli+3D-TOF DL model and the T2+3D-TOF Rad model have almost the same AUC values ​​in EVC. However, other performance parameters of the Cli+3D-TOF DL model are significantly better than other models including T2+3D-TOF Rad. Therefore, among all the pre-fusion models, the Cli+3D-TOF DL model has the best effect, with AUCs of 0.915 [95% CI, 0.915, 0.847] and 0.841 [0.763-0.918] in PC and EVC, respectively, which can better distinguish TN patients from normal patients. The sensitivity of the Cli+3D-TOF DL model in PC is 0.826, and the sensitivity in EVC is 0.786. Similarly, the specificity of the model in PC is 0.853, and the specificity in EVC is 72.7%.

[0081] In Post-FM, the three models based on Cli+Rad(T2+3D-TOF), Cli+DL(T2+3D-TOF) and Cli+Rad(T2+3D-TOF) probability can well distinguish whether the patient is TN or not. The ROC curve analysis of the post-fusion model in the training set and the validation set is shown in Figure 2. Figure 5 As shown in the figure. Among all the post-fusion models, the Cli+Rad model performed best, with AUCs of 0.899 [95% CI, 0.836, 0.922] and 0.837 [0.700-0.974] in PC and EVC, respectively, which could better distinguish TN patients from normal patients. The sensitivity of the Cli+Rad model in PC was 0.747, and that in EVC was 0.815.

[0082] The DeLong test was used to compare whether there was a significant difference in AUC between the models in the two cohorts. Table 2 shows the performance comparison of each model in the anterior fusion. In EVC, the AUC value of the Cli+3D-TOF DL model was slightly lower than that of the T2+TOF Rad model (0.841 vs 0.842, p = 0.076), however, the model performance of the Cli+3D-TOF DL model was significantly different from that of the Cli+T2 DL model (0.841 vs 0.762, p = 0.037), the T2+3D-TOF DL model (0.841 vs 0.693, p = 0.419), and the Cli+3D-TOF Rad model (0.841 vs 0.816, p = 0.788).

[0083] Table 2. Comparison of the prefusion model in the primary cohort and the external validation cohort

[0084]

[0085] S41, Results and Discussion:

[0086] Calibration curves of the pre-fusion model in training (PC) and external validation cohort (EVC), as Figure 6 As shown. The x-axis of the calibration curve is the predicted result of the model measurement, and the y-axis is the actual result. The black dotted line represents the ideal evaluation of the perfect model, and the solid lines of different colors represent the discrimination ability of different pre-fusion models. The closer the model line is to the black dotted line, the better the evaluation.

[0087] exist Figure 6 It can be observed that Cli+3D-TOFDL (green) is closely aligned with the black line in both the main cohort (PC) and the external validation cohort (EVC). These results indicate that the clinical diagnostic model constructed based on clinical and 3D-TOF features extracted by deep learning is very consistent with the actual clinical situation in the prediction results of trigeminal neuralgia in PC and EVC. This important finding strongly indicates that the treatment regimen selected by the prediction model constructed according to the present invention has excellent model calibration compared to the traditional standardized treatment regimen.

[0088] Finally, it should be noted that the above is only used to illustrate the technical solution of the present invention and is not a limitation. Although the present invention has been described in detail with reference to the preferred arrangement scheme, a person skilled in the art should understand that the technical solution of the present invention (such as the use of various formulas, the sequence of steps, etc.) can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A multimodal medical image data analysis method based on machine learning, characterized in that: The method comprises the following steps: Step 1, MRI data collection and preprocessing: S11, image acquisition: MRI images of patients with trigeminal neuralgia who underwent surgical treatment were collected as the primary cohort and external validation cohort; S12, image segmentation: The MRI images acquired by S11 were segmented using ITK-SNAP software, which specifically involved: first, the 3D-TOF images were registered to the T2 images using linear interpolation; then, the T2 images were manually segmented, and the regions of interest (ROIs) were manually outlined on each slice showing the nerves; finally, the volumes of interest (VOIs) were generated by fusing the segmented tissues on each slice; thus, when the VOI segmentation was completed on the T2 images, the corresponding VOIs were automatically generated on the 3D-TOF images; Step 2, MRI feature extraction: S21, MRI deep feature extraction based on radiomics: PyRadiomics software was used to extract radiomic features from MRI image data to quantitatively analyze the trigeminal nerve of each patient. Specifically, several radiomic features were extracted from each patient's T2 image and 3D-TOF image sequence. These features were divided into three categories: shape and volume features, texture features, and first-order statistical features, each of which provided different dimensions of trigeminal nerve characteristic information. Shape and volume features focus on describing the geometric features of the ROI: including sphericity and compactness, surface area, volume, and maximum two-dimensional diameter to describe the shape characteristics of the tumor; Texture features: including the spatial relationship between pixels of the same or different gray levels in the region of interest; The first-order statistical characteristics are calculated by analyzing the histogram: including the extreme value, variance, and kurtosis of the pixel grayscale in the region of interest; S22, MRI deep feature extraction based on 3D-ResNet: A pre-trained 3D-ResNet-18 model was used, and the last fully connected layer of the model was removed to extract intermediate features that were not final classifications. The modified 3D-ResNet-18 model was called to extract features from each 3D image by converting the image and mask into NumPy arrays and performing normalization and standardization. The feature extraction was focused on the output layer before removing the fully connected layer. Several deep learning features were extracted from the two volumes of interest of each patient, T2 and 3D-TOF, and standardized using the z-score method to ensure that the feature values ​​were within a uniform standardized range. Step 3, Clinical Diagnostic Model of Trigeminal Neuralgia: S31, feature screening: Several feature sets were constructed from the ROIs obtained in step 1: a radiomics feature set based on T2 and TOF sequences, and a deep learning feature set based on T2 and TOF sequences; First, each set of features was screened by t-test to exclude variables with no statistically significant differences. Second, the Pearson or Spearman correlation coefficients between all possible feature pairs were calculated, and one feature in any feature pair with a correlation coefficient exceeding 0.9 was removed to eliminate highly correlated variables. Subsequently, the least absolute shrinkage and selection operator LASSO regression method was used to screen features that were closely related to trigeminal neuralgia, and 5-fold cross-validation was applied to optimize the model selection process and improve the generalization ability of the model. S32, Model Development: Three types of models are developed, including single-modal models (SM), pre-fusion models (Pre-FMs), and post-fusion models (Post-FMs): 1) Single modality model SM includes five models: Clinic model, T2 Rad model, T2 DL model, TOF Rad model, and TOF DL model; 2) Pre-FMs include six models: Cli+T2 DL model, Cli+T2 Rad model, Cli+TOF DL model, Cli+TOF Rad model, T2+TOF DL model and T2+TOF Rad model; 3) Post-FMs includes three models: Clinic+Rad model, Clinic+DL model, and Clinic+Rad+DL model; Support vector machine is used as classifier to develop single modal model SM and pre-fusion model Pre-FMs, and random forest is used as classifier to develop post-fusion model Post-FMs; The model development includes the following steps: on the main cohort described in step 1, the data is further divided into a training cohort and an internal validation cohort. By using 5-fold cross-validation, 4 / 5 samples are randomly defined as the training cohort to train the model, and the remaining 1 / 5 samples are defined as the internal validation cohort to optimize the parameters; after rigorous repetition for several times, the optimal hyperparameter combination is obtained; then the performance of the model is tested on the external validation set; Step 4: Model result analysis: The area under the ROC curve (AUC) was used as the core evaluation indicator to compare the discrimination of different models, and the model with the best classification effect was selected to obtain the final trigeminal neuralgia detection results. Specifically, the larger the AUC value, the stronger the discriminative ability of the model and the better the classification effect.

2. The method according to claim 1, characterized in that In step 1, the inclusion criteria for patients with trigeminal neuralgia include: 1) undergoing a complete preoperative MRI examination; 2) having good image quality for analysis; 3) having a clear pathological diagnosis; Exclusion criteria included: 1) secondary trigeminal neuralgia; 2) missing T2 or 3D-TOF sequences.

3. The method according to claim 1, characterized in that In step 4, if the AUC value exceeds 0.80, it indicates that the model has excellent discrimination ability and can accurately distinguish samples of different categories; when the AUC value is between 0.75 and 0.80, although the discrimination ability is slightly lower, it still performs well and can meet the needs of actual applications; if the AUC value is lower than 0.60, it implies that the model's discrimination and prediction capabilities are weak, and there are serious classification errors or deviations, and the model needs to be further adjusted or optimized to improve its performance.

Images