Brain glioma non-invasive grade diagnosis and IDH typing method for multi-mode MRI data missing

Abstract

The invention discloses a brain glioma non-invasive grade diagnosis and IDH typing method oriented to multi-mode MRI data missing, and mainly solves the problem that stable and reliable glioma grading and IDH typing prediction are difficult to realize in an existing multi-mode brain glioma analysis method under the condition of MRI mode missing. According to the scheme, the method comprises the following steps: constructing a cooperative compensation network which comprises four convolution encoders with the same structure, two dynamic adaptive unified interaction modules with the same structure, a 0 / 1 indication code, a shared auxiliary decoder, a missing mode main decoder and a tumor classification characterization module, and constrains two paths through KL divergence and a mean square error, wherein the cooperative compensation network comprises the four convolution encoders with the same structure, the two dynamic adaptive unified interaction modules with the same structure, the 0 / 1 indication code, the shared auxiliary decoder, the missing mode main decoder and the tumor classification characterization module; and training the network through glioma grade gold standard and IDH typing gold standard constraints to obtain a trained grading and IDH typing network, thereby realizing prediction of glioma grading and IDH typing, and generating a corresponding classification significance heat map. Experimental results show that high-precision grading and IDH typing can be carried out on glioma, the method can be used for analyzing brain MRI images, and a non-invasive reliable basis is provided for clinical auxiliary diagnosis and decision making.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing and artificial intelligence-assisted diagnosis technology, and specifically relates to a magnetic resonance imaging (MRI) method for grading gliomas and isocitrate dehydrogenase (IDH) classification, which can be applied to provide intelligent auxiliary basis for clinical brain tumor diagnosis and individualized decision-making. Background Technology

[0002] Gliomas are the most common primary malignant tumors of the central nervous system, characterized by high invasiveness, strong heterogeneity, and poor prognosis. High-grade gliomas, in particular, progress rapidly and have a high recurrence rate, seriously threatening patients' lives. Clinical studies have shown that tumor grading and key molecular markers, such as isocitrate dehydrogenase (IDH) mutation status, are crucial for determining patient prognosis, treatment plans, and follow-up strategies. Therefore, accurate delineation, grading, and molecular subtyping of gliomas are of great significance for advancing precision medicine and personalized treatment.

[0003] MRI, with its advantages of high soft tissue resolution, good contrast, and no ionizing radiation, has become a core imaging tool for the diagnosis, preoperative evaluation, and follow-up of glioma treatment. Routine clinical MRI examinations typically include multiple sequences such as Fluir (fluid attenuation recovery sequence), T1ce (T1-weighted contrast-enhanced scanning sequence), T1 (T1-weighted imaging sequence), and T2 (T2-weighted imaging sequence). Different modalities provide complementary information from the perspectives of edema extent, tumor core, necrosis area, and blood-brain barrier disruption. Multimodal MRI combined analysis is considered an important means to improve the accuracy of glioma imaging assessment.

[0004] However, in actual clinical applications, due to objective factors such as limited scanning time, insufficient patient tolerance, differences in imaging protocols, inconsistent equipment conditions, or missing data, MRI data of glioma patients often exhibit partial modality loss. This modality loss phenomenon is prevalent in real clinical scenarios, severely restricting the widespread application of intelligent diagnostic models that rely on complete multimodal input in clinical practice.

[0005] In recent years, with the rapid development of deep learning technology, significant progress has been made in automatic glioma segmentation methods based on convolutional neural networks, U-Net, and their variants. Related studies have achieved high segmentation accuracy on public datasets such as BraTS. Building on this foundation, some studies have further combined segmentation results with image features to explore downstream prediction tasks such as glioma grading and IDH subtyping.

[0006] Patent document CN120259342A discloses a multimodal MRI brain tumor image segmentation method based on the fusion of Transformer and U-Net, which introduces an attention mechanism and an inverted residual module for brain tumor image segmentation. Patent document CN114947807B discloses a multi-task prediction method for brain invasion classification and meningioma grading based on multimodal MRI, which achieves preoperative multi-task prediction of meningiomas through feature fusion, contrastive learning, and multi-task prediction. Patent document CN120809159A discloses a deep learning-based multi-source data-assisted diagnostic system for myxoid brain tumors, which achieves tumor type identification and interpretable diagnosis through multimodal feature extraction and cross-modal association analysis. However, these methods are highly dependent on the completeness of the input modality, and therefore cannot perform tumor segmentation and downstream tasks when the modality is missing, making it difficult to meet the application needs of incomplete image data in real clinical scenarios.

[0007] To address the issue of missing modalities in multimodal MRI data, existing research mainly employs two strategies: modal completion methods based on generative models, and simple modal zeroing or masking. Zhou et al.'s method, Missing DataImputation via Conditional Generator and Correlation Learning for MultimodalBrain Tumor Segmentation, addresses missing modal MRI by introducing an additional modal generation network to synthesize missing images. However, such methods typically require constructing independent generation modules, leading to complex model structures, unstable training, and the tendency for generation errors to accumulate progressively in subsequent segmentation and prediction tasks, affecting the robustness of prognostic predictions.

[0008] In contrast, modality nulling methods offer advantages such as simplicity and the absence of additional generative networks. Shi et al.'s M2FTrans (Modality-Masked Fusion Transformer for Incomplete Multi-ModalityBrain Tumor Segmentation) treats missing modalities as informationless inputs, thus achieving unified modeling of incomplete modalities. However, this method has limited ability to model the differences and complementarities between different MRI modalities, making it difficult to fully exploit the collaborative information between multiple modalities. This results in insufficient utilization of key information in modality-missing scenarios, and the model's segmentation performance remains significantly inadequate, failing to provide accurate tumor features for downstream tasks.

[0009] In summary, existing methods for glioma segmentation, grading, and IDH classification based on multimodal MRI mostly rely on complete modality input. These methods face significant performance degradation in real-world clinical scenarios with missing modalities, and lack a unified solution for adaptive modeling and effective fusion of multimodal features under missing modality conditions, making stable application in actual diagnostic and treatment processes difficult. Therefore, developing a non-invasive grading and IDH classification method for gliomas with missing multimodal MRI data, providing interpretable imaging indicators while achieving stable predictions, and enhancing the consistency and reliability between prediction results and clinicopathological evidence, is crucial to overcoming the current bottlenecks in the clinical management and precision diagnosis of gliomas. Summary of the Invention

[0010] The purpose of this invention is to address the shortcomings of the prior art by proposing a non-invasive method for grading and classifying gliomas and IDH (intracytoplasmic hemorrhage) in cases of missing multimodal MRI data. This method enables effective modeling and stable prediction of tumor structural features without relying on complete multimodal input or additional modality generation processes. Furthermore, by analyzing the relationship between the model's region of interest and tumor structural features, the model's prediction results can correspond to the pathological diagnostic criteria for glioma grading and IDH classification, thereby enhancing the clinical consistency and reliability of the prediction results.

[0011] To achieve the above objectives, the technical solution of the present invention includes:

[0012] 1. A collaborative compensation network for missing multimodal MRI data, characterized in that it comprises:

[0013] Four identical convolutional encoders were selected, each of which included L layers of three-dimensional convolutional modules for independent multi-scale feature extraction of each MRI modality, where L≥5;

[0014] Two identical dynamic adaptive unified interaction modules are constructed to perform self-enhancement and cross-modal interaction on multimodal features. During the cross-modal interaction process, the modules utilize the collaborative modeling between existing modal features to achieve indirect expression and compensation of missing modal tumor features.

[0015] Construct 0 / 1 indicator code It includes a modality presence indicator variable set for each MRI modality, where 1 indicates modality presence and 0 indicates modality absence, used to encode the presence status of each modality in the input sample;

[0016] A shared auxiliary decoder with a K-layer 3D decoding structure is selected. The structure of each layer and the connections between layers are the same as those of the convolutional encoder. It is used to decode the multimodal enhancement features and output auxiliary segmentation features, where K≥4.

[0017] A missing modality master decoder with a K-layer 3D decoding structure is selected. The structure of each layer and the connection between each layer are the same as those of the convolutional encoder. It is used to decode the features after modality compensation and output the segmentation features under the missing modality condition.

[0018] A tumor classification characterization module is constructed, which includes a cascaded K-layer 3D convolutional module, global average pooling, global max pooling, and a linear fully connected layer. Each convolutional module and the connections between each layer are the same as those of the convolutional encoder. This module is used to characterize and model the segmentation features output by the missing modality master decoder and output glioma grading and IDH subtyping prediction results.

[0019] The convolutional decoder is connected to the first dynamic adaptive unified interaction module. One output of this module is connected to the shared auxiliary decoder to form an auxiliary path, and the other output is connected to the 0 / 1 indicator encoder, the second dynamic adaptive unified interaction module, the missing modality master decoder, and the tumor classification characterization module in sequence to form the main path. These two paths form a collaborative compensation network through KL divergence constraints and mean square error constraints.

[0020] Furthermore, the L-layer 3D convolutional modules in the convolutional encoder are cascaded sequentially through convolution operations, each...

[0021] The module consists of a 3×3×3 three-dimensional convolutional layer, an activation layer, and an instance normalization layer, which are connected in sequence.

[0022] Furthermore, the dynamic adaptive unified interaction module includes:

[0023] The global context-aware submodule includes convolution operations, flattening mapping, self-attention operations, residual connections and normalization, feedforward mapping, normalization processing and spatial reconstruction, which are used to extract input features and global context information between modalities.

[0024] The local information preservation submodule is used to extract input features and local detail information between modalities;

[0025] The fusion submodule is used to fuse the output of the global context-aware submodule with the output of the local information preservation submodule to obtain the final output.

[0026] Furthermore, the outputs of the K-layer 3D convolutional modules in the tumor classification and characterization module are respectively subjected to global average pooling and...

[0027] Global max pooling is applied, and the results of the two pooling operations are concatenated. The concatenated features are then input into a linear fully connected layer to output the classification result.

[0028] 2. A method for grading gliomas using the aforementioned collaborative compensation network, characterized in that it includes:

[0029] 1) 70% of the data from the publicly available preoperative diffuse glioma MRI dataset was extracted according to its grade as the training set, 30% as the internal test set, and N clinical patients were collected as the external test set, N≥50 cases.

[0030] 2) Convert the original DICOM format data of N clinical patients to NIfTI format, register the converted data to the MNI152 standard brain template, and then perform automatic skull dissection to obtain standardized brain tissue data;

[0031] 3) Input the four MRI modalities of Flair, T1ce, T1 and T2 from the training set into the collaborative compensation network. Constrained by the gold standard for glioma segmentation and the gold standard for grading, the SGD optimizer combined with momentum parameters is used to iteratively update the model parameters until the preset number of iterations are obtained to obtain the trained collaborative compensation network.

[0032] 4) Input the MRI modalities of four different combinations (Flair, T1ce, T1 and T2) from the internal test set of glioma grading and N external test sets into the trained collaborative compensation network, and output the glioma grading prediction results and significance heatmap.

[0033] 5) Upsample the saliency heatmap of glioma grade output by the model to make its spatial resolution consistent with the original MRI image, and then overlay the upsampled saliency heatmap with the original MRI image to obtain the correspondence between different grades of glioma and image features.

[0034] 3. A method for IDH classification using the above-mentioned collaborative compensation network, characterized in that it includes:

[0035] (1) 70% of the data from the publicly available preoperative diffuse glioma MRI dataset was extracted according to its IDH classification as the training set, 30% as the internal test set, and M clinical patients were collected as the external test set, M≥65 cases;

[0036] (2) Convert the original DICOM format data of M clinical patients into NIfTI format, register the converted data into the MNI152 standard brain template, and then perform automatic skull dissection to obtain standardized brain tissue data;

[0037] (3) Input the four MRI modalities of Flair, T1ce, T1 and T2 from the training set into the collaborative compensation network, and iteratively train the network by constraining it with the gold standard for glioma segmentation and the gold standard for IDH classification;

[0038] (4) Input the four different combinations of MRI modalities of Flair, T1ce, T1 and T2 from the internal test set of IDH subtyping and the N external test sets into the trained collaborative compensation network, and output the IDH subtyping prediction results and significance heatmap.

[0039] (5) Upsample the IDH type saliency heatmap output by the model to make its spatial resolution consistent with the original MRI image, and overlay the upsampled saliency heatmap with the original MRI image to obtain the correspondence between different IDH types and image features.

[0040] Compared with the prior art, the present invention has the following advantages:

[0041] Firstly, this invention constructs a collaborative compensation network for missing multimodal MRI data. Addressing the potential loss of modalities such as Flair, T1ce, T1, and T2 in clinical MRI examinations, the network utilizes a dynamic adaptive unified interaction module, along with dual-path KL divergence constraints and mean squared error constraints, to effectively model the complementary relationships between different MRI modalities in tumor spatial structure and biological characteristics. This allows the model to stably extract discriminative features of the tumor core, infiltration boundaries, and edema regions even when key modalities are missing, thereby significantly improving the accuracy of glioma grading and IDH classification. This enables the first-ever glioma grading and IDH classification under modality loss conditions.

[0042] Secondly, this invention introduces an interpretable visualization mechanism under the condition of missing MRI modalities. It displays the model’s focus area through a saliency heatmap, so that key imaging areas such as tumor enhancement areas, non-enhancing areas, and edema areas in the prediction task form a spatial correspondence with the model’s focus area. Combined with the analysis of high-weight features and prediction results, it establishes a mapping between imaging features and pathological diagnosis results, so that the output results have clear pathological basis and clinical traceability. Attached Figure Description

[0043] Figure 1 This is a schematic diagram of the collaborative compensation network structure for missing multimodal MRI data according to the present invention;

[0044] Figure 2 This is a flowchart illustrating the implementation of a glioma grading method based on a collaborative compensation network.

[0045] Figure 3 This is a flowchart illustrating the implementation of a glioma IDH classification method based on a collaborative compensation network.

[0046] Figure 4 This invention provides a heatmap of glioma grade significance output through a glioma grading collaborative compensation network.

[0047] Figure 5This is a heatmap of IDH type saliency output by the IDH type collaborative compensation network of this invention. Detailed Implementation

[0048] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention and not all of them. Based on the embodiments of the present invention, other embodiments obtained by those skilled in the art without creative effort should all fall within the protection scope of the present invention.

[0049] Example 1: Collaborative Compensation Network for Missing Multimodal MRI Data

[0050] Reference Figure 1 This embodiment includes four structurally identical convolutional encoders 1, two dynamic adaptive unified interaction modules 2, a 0 / 1 indicator encoder 3, a shared auxiliary decoder 4, a missing modality master decoder 5, and a tumor classification characterization module 6, wherein:

[0051] The four identical convolutional encoders 1 each include L layers of three-dimensional convolutional modules, which are cascaded sequentially through convolution operations. Each three-dimensional convolutional module includes a 3×3×3 three-dimensional convolutional layer, an activation layer, and an instance normalization layer, which are connected sequentially to perform independent multi-scale feature extraction for each MRI modality.

[0052] The two identical dynamic adaptive unified interaction modules 2 are used for self-enhancement and cross-modal interaction of multimodal features. During cross-modal interaction, they utilize collaborative modeling among existing modal features to achieve indirect expression and compensation of missing modality tumor features. This includes:

[0053] The global context-aware submodule 21 is used to sequentially perform convolution operations, flattening mapping, self-attention operations, residual connection and normalization, feedforward mapping, normalization processing and spatial reconstruction operations to extract input features and global context information between modalities.

[0054] The local information preservation submodule 22 is used to perform feature weighted fusion operation with residual connections to extract input features and local detail information between modalities.

[0055] The fusion submodule 23 is used to fuse the output of the global context-aware submodule with the output of the local information preservation submodule to obtain the final output.

[0056] The 0 / 1 indicator code 3 includes a modality presence indicator variable set for each MRI modality, where 1 indicates modality presence and 0 indicates modality absence, and is used to encode the presence status of each modality feature in the input sample.

[0057] The shared auxiliary decoder 4 includes a K-layer three-dimensional decoding structure. The structure of each layer and the connections between each layer are the same as those of the convolutional encoder. It is used to decode multimodal enhancement features and output auxiliary segmentation features.

[0058] The missing modality master decoder 5 includes a K-layer three-dimensional decoding structure. The structure of each layer and the connections between each layer are the same as those of the convolutional encoder. It is used to decode the features after modality compensation and output the segmentation features under the missing modality condition.

[0059] The tumor classification characterization module 6 includes a cascaded K-layer 3D convolutional module, global average pooling, global max pooling, and a linear fully connected layer. Each convolutional module and the connections between layers are the same as those of the convolutional encoder. It is used to characterize and model the segmentation features output by the missing modality master decoder and output glioma grading and IDH subtyping prediction results. Specifically, the output of the K-layer 3D convolutional module is processed by global average pooling and global max pooling, and the results of these two pooling are concatenated. The concatenated features are then input into the linear fully connected layer to output the classification results.

[0060] The convolutional encoder 1 is connected to the first dynamic adaptive unified interaction module 2. One output of this module is connected to the shared auxiliary decoder 4 to form an auxiliary path, and the other output is connected to the 0 / 1 indicator encoder 3, the second dynamic adaptive unified interaction module 2, the missing modality master decoder 5, and the tumor classification characterization module 6 in sequence to form the main path. These two paths form a collaborative compensation network through KL divergence constraints and mean square error constraints.

[0061] The KL divergence constraint is used to reduce the overall tumor feature distribution formed after concatenation and convolution of tumor features from different modalities in the auxiliary decoder branch. Feature distribution in the main path decoder of missing modalities Differences between :

[0062] ,

[0063] in, =4 indicates the number of decoder layers. This represents the index, used to iterate through each element in the feature vector;

[0064] The mean squared error constraint is used to reduce the tumor segmentation features output by the missing modality main path decoder. Tumor segmentation features output from each modality of the shared auxiliary decoder Differences between :

[0065] ,

[0066] in, The formula for calculating the mean squared error. , 1 indicates the output of the last layer.

[0067] Example 2: A method for glioma grading based on a collaborative compensation network.

[0068] Reference Figure 2 The implementation steps of this example include the following:

[0069] Step 1. Obtain a dataset of brain MRI images.

[0070] 1.1) Cases were screened from publicly available preoperative diffuse glioma MRI datasets. All cases contained FLAIR, T1ce, T1 and T2 imaging data. In this embodiment, the number of cases obtained was no less than 495.

[0071] 1.2) Collect brain MRI image data from N clinical patients. In this embodiment, N is no less than 50 cases.

[0072] 1.3) Cases selected from the publicly available preoperative diffuse glioma MRI dataset were stratified and sampled according to their pathological grade, with 70% of the data used as the training set and the remaining 30% used as the internal test set;

[0073] 1.4) The collected image data of N clinical patients will be used as a clinical validation set to verify the clinical applicability of the model.

[0074] Step 2. Standardized MRI processing of N clinical patients.

[0075] 2.1) Using the ITK-SNAP toolkit, the original DICOM format brain MRI image data of N clinical patients were converted...

[0076] Change to NIfTI format;

[0077] 2.2) The brain imaging data that has undergone format conversion is registered to the MNI152 standard brain model using a rigid registration method.

[0078] In the template, each image data is kept consistent with the standard brain template in the spatial coordinate system;

[0079] 2.3) On the registered brain image data, the HD-BET automatic skull stripping algorithm is applied to the skull and other parts.

[0080] Non-brain tissue structures are removed, leaving only the brain parenchyma tissue areas, thereby obtaining standardized brain tissue imaging data.

[0081] Step 3. Train the glioma grading collaborative compensation network using the glioma grading training set.

[0082] 3.1) Perform random rotation, random scaling, and random flipping data augmentation operations sequentially on the brain MRI images in the training set;

[0083] 3.2) The enhanced data is input into the collaborative compensation network, and the grading prediction results of glioma are obtained through the gold standard constraint network for glioma grading. At the same time, auxiliary segmentation features are output to locate the tumor region.

[0084] 3.3) Optimize the collaborative compensation network through multi-round iterative training:

[0085] 3.3.1) Set the network training parameters, using a fixed learning rate of 0.0001, a momentum parameter of 0.9, a batch size of 1, and a weight decay coefficient of 0.001;

[0086] 3.3.2) Define the training loss for glioma grading :

[0087] Using the Dice loss function As an auxiliary loss:

[0088] ,

[0089] in, From 1 to , Indicates the first The sample lacks the segmentation features output by the modal master decoder in the collaborative compensation network. It is the first The gold standard for glioma segmentation in a single sample. This represents the number of training samples;

[0090] Cross-entropy loss is used as the hierarchical prediction loss:

[0091] ,

[0092] in, From 1 to Indicates the number of categories. The output of the tumor classification and characterization module The sample belongs to the first Predicted probability of class For the corresponding graded gold standard;

[0093] The training loss for glioma grading was calculated based on the auxiliary loss and the grading prediction loss. :

[0094] ;

[0095] 3.3.3) In each training iteration, samples are randomly selected from the training set and input into the collaborative compensation network to calculate the glioma grading training loss. ;

[0096] 3.3.4) Based on total training loss The backpropagation algorithm was executed using the SGD optimizer combined with momentum parameters to update the model parameters of the glioma grading collaborative compensation network.

[0097] 3.3.5) Repeat steps 3.3.3) - 3.3.4) until the training loss converges and reaches the preset 500 iterations, to obtain the trained glioma grading collaborative compensation network.

[0098] Step 4. Test the output of the glioma grading collaborative compensation network using the glioma grading test set and the external test set.

[0099] 4.1) Combine the four MRI modalities (FLAIR, T1ce, T1, and T2) from the internal test set for glioma grading and the N external clinical test sets, including fifteen combinations of single-modality, dual-modality, trimodality, and full-modality:

[0100] Single-modal combinations: Flair, T1ce, T1, T2;

[0101] Bimodal combinations: Flair+T1ce, Flair+T1, Flair+T2, T1ce+T1, T1ce+T2, T1+T2;

[0102] Three-modal combinations: Flair+T1ce+T1, Flair+T1ce+T2, Flair+T1+T2, T1ce+T1+T2;

[0103] Full-modal combination: Flair + T1ce + T1 + T2;

[0104] 4.2) The image data of the fifteen combinations are sequentially input into the trained glioma grading collaborative compensation network to obtain the glioma grading prediction results of the fifteen combinations.

[0105] 4.3) For the glioma grading prediction results of each modality combination, generate a glioma grading significance heatmap based on the image regions that the model focuses on during the prediction process.

[0106] Step 5. Visualize the hierarchical significance heatmap.

[0107] (5.1) Upsample the glioma grade significance heatmap to make its spatial resolution consistent with the original MRI image;

[0108] (5.2) The upsampled grade saliency heatmap is overlaid with the original MRI image and a transparency strategy is used for fusion to make the heatmap area stand out and the original image structure visible.

[0109] (5.3) Based on the overlay results, the correspondence between different glioma grades and imaging features was obtained.

[0110] Example 3: A method for glioma IDH typing based on a collaborative compensation network.

[0111] Reference Figure 3 The implementation steps of this example include the following:

[0112] Step A. Obtain a dataset of brain MRI images.

[0113] A1) Cases were screened from publicly available preoperative diffuse glioma MRI datasets, all of which included FLAIR and [other MRI findings].

[0114] In this embodiment, no fewer than 495 cases were obtained, including four modal imaging data: T1ce, T1, and T2.

[0115] A2) Collect brain MRI image data of M clinical patients. In this embodiment, M is no less than 65 patients.

[0116] A3) Cases selected from the publicly available preoperative diffuse glioma MRI dataset were stratified and sampled according to their IDH type, with 70% of the data used as the training set and the remaining 30% used as the internal test set;

[0117] A4) The collected image data of M clinical patients will be used as a clinical validation set to verify the clinical applicability of the model.

[0118] Step B. Standardized MRI processing of M clinical patients.

[0119] B1) Using the ITK-SNAP toolkit, convert the original DICOM format brain MRI images of M clinical patients into...

[0120] Change to NIfTI format;

[0121] B2) The brain imaging data that has undergone format conversion is registered to the MNI152 standard brain model using a rigid registration method.

[0122] In the template, each image data is kept consistent with the standard brain template in the spatial coordinate system;

[0123] B3) On the registered brain imaging data, the HD-BET automatic skull stripping algorithm is applied to the skull and other parts.

[0124] Non-brain tissue structures are removed, leaving only the brain parenchyma tissue areas, thereby obtaining standardized brain tissue imaging data.

[0125] Step C. Train the IDH classification collaborative compensation network using the IDH classification training set.

[0126] C1) Perform data augmentation on brain MRI images in the training set;

[0127] C2) The enhanced data is input into the collaborative compensation network, and the IDH classification prediction results are obtained through the IDH classification gold standard constraint network. At the same time, auxiliary segmentation features are output for locating the tumor region.

[0128] C3) Optimize the collaborative compensation network through multi-round iterative training, specifically including the following steps:

[0129] C31) Set the optimizer parameters, use a warm-up mechanism to dynamically adjust the learning rate, set the momentum to 0.90, the batch size to 1, the weight decay coefficient to 0.0005, and the initial learning rate to 0.0002.

[0130] C32) Define the IDH hierarchical training loss :

[0131] ,

[0132] Among them, auxiliary losses With graded auxiliary loss The calculation method is the same, and the IDH classification predicts the loss. Loss of glioma grading The calculation method is the same;

[0133] C33) In each training iteration, samples are randomly selected from the training set, input into the network, and the total training loss is calculated. ;

[0134] C34) Based on total training loss The AdamW optimizer is used to execute the backpropagation algorithm to update the model parameters of the IDH type collaborative compensation network;

[0135] C35) Repeat steps C33) - C34) until the training loss converges and reaches the preset number of 300 iterations, to obtain the trained IDH type collaborative compensation network.

[0136] Step D. Test the IDH type-based collaborative compensation network using the IDH type test set and external test set.

[0137] D1) Combine the four MRI modalities (Flair, T1ce, T1, and T2) from the internal test set for IDH subtyping and the M external clinical test set, including fifteen combinations of single-modality, dual-modality, trimodality, and full-modality:

[0138] Single-modal combinations: Flair, T1ce, T1, T2;

[0139] Bimodal combinations: Flair+T1ce, Flair+T1, Flair+T2, T1ce+T1, T1ce+T2, T1+T2;

[0140] Three-modal combinations: Flair+T1ce+T1, Flair+T1ce+T2, Flair+T1+T2, T1ce+T1+T2;

[0141] Full-modal combination: Flair + T1ce + T1 + T2;

[0142] D2) Input the fifteen modality combinations into the trained IDH subtyping collaborative compensation network in sequence to obtain the prediction results of tumor IDH subtyping under each combination;

[0143] D3) For each mode combination, generate an IDH classification significance heatmap based on the image regions that the model focuses on during the prediction process.

[0144] Step E. Visualize the heatmap of IDH typing significance.

[0145] E1) Upsample the IDH genotyping significance heatmap to make its spatial resolution consistent with the original MRI image;

[0146] E2) The upsampled IDH classification significance heatmap is overlaid with the original MRI image, and an adjustable transparency fusion strategy is used to highlight key imaging areas related to IDH type while preserving the anatomical information of the original image.

[0147] E3) Based on the overlay results, analyze the distribution of image features corresponding to different IDH types to realize the visualization of the correlation between molecular markers and image features.

[0148] It should be noted that the step numbers in the embodiments and claims of this invention are only for the purpose of clearly describing the implementation schemes of this invention and facilitating understanding, and their order is not limited.

[0149] It should be noted that the flowchart representations or method representations of Embodiments 2 and 3 above can be understood as representing modules, segments, or portions of code comprising one or more executable instructions configured to implement specific logical functions or processes. The present invention is not limited to the disclosed preferred embodiments, and its implementation may not follow the order shown or discussed, including performing functions substantially simultaneously according to the functions involved.

[0150] The effects of this invention can be further illustrated by the following simulation experiments:

[0151] I. Simulation conditions:

[0152] The simulation test platform of this invention is a PC with an Intel Core i7-4790K CPU 4.0GHz, 128GB of memory, and an NVIDIA RTX A5000 graphics card. The simulation platform is based on the Ubuntu 20.04 operating system, uses ITK-SNAP software and the PyTorch deep learning framework, and is implemented in Python.

[0153] Data sources: The data in this experiment are derived from publicly available preoperative diffuse glioma MRI data and clinically collected multimodal MRI data.

[0154] Evaluation metrics: Three mainstream classification task evaluation metrics were used: overall accuracy; harmonic mean F1-score; and area under the receiver operating characteristic curve (AUC). The closer these three metrics are to 1, the better the model performance.

[0155] The overall accuracy measures the proportion of the classification model's predictions that match the true labels across all samples, reflecting the overall correctness of the model's classification. Its calculation formula is as follows:

[0156] ,

[0157] in, Indicates a real example, Indicates a true counterexample. Indicates a false positive example. Indicates a false counterexample;

[0158] The harmonic mean F1-score is used to comprehensively evaluate the balance between precision and recall of the classification model, and can reflect the model's overall ability to identify positive samples. Its calculation formula is as follows:

[0159] ,

[0160] The Area Under the Receiver Operating Characteristic (ROC) curve (AUC) measures the overall ability of a classification model to distinguish between positive and negative samples at different discrimination thresholds. A higher AUC value indicates better model performance. The ROC curve is plotted with the false positive rate (FPR) on the horizontal axis and the true positive rate (TPR) on the vertical axis. Its calculation formula is as follows:

[0161] ,

[0162] in, This indicates the model's ability to identify positive class samples. This indicates the proportion of negative samples that are misclassified as positive samples.

[0163] II. Simulation Content and Result Analysis:

[0164] Simulation 1: This invention employs a glioma grading collaborative compensation network to grade gliomas on a publicly available preoperative diffuse glioma MRI dataset under three combined modalities: T1ce, Flair+T1ce, and Flair+T1ce+T2. The glioma grading performance of the model output is evaluated using Accuracy, F1-score, and AUC evaluation metrics. The results are shown in Table 1.

[0165] Table 1. Quantitative assessment results of glioma grading using the publicly available preoperative diffuse glioma MRI dataset.

[0166]

[0167] As shown in Table 1, the accuracy, F1-score and AUC are all above 90%, and the AUC value reaches above 99%, indicating that the present invention has stable predictive ability under these three combined modal conditions.

[0168] Based on the quantitative assessment results of glioma grading, a significant visualization analysis is performed on the key imaging regions that the model focuses on during grading prediction. Specifically, based on the discriminant feature information of the model's output grading results for image samples with different modality combinations, a corresponding glioma grading significance heatmap is generated, such as... Figure 4 As shown.

[0169] from Figure 4This paper displays the main tumor solid regions and their boundary regions of interest when using glioma grading significance heatmaps generated under three modal combinations to distinguish between high-grade and low-grade gliomas. The heatmaps also show different distribution patterns of interest under different glioma grades. Figure 4 As can be seen, this invention can intuitively present the key tumor regions on which the model grading decision depends through a saliency heatmap, thereby providing a reliable and interpretable auxiliary analysis basis for glioma grading tasks under modality-deficient conditions.

[0170] Simulation 2: This invention employs a glioma grading collaborative compensation network to grade N gliomas from an externally validated clinical MRI dataset under three combined modalities: T1ce, Flair+T1ce, and Flair+T1ce+T2. The glioma grading performance of the model output is evaluated using Accuracy, F1-score, and AUC evaluation metrics. The results are shown in Table 2.

[0171] Table 2. Results of quantitative glioma grading assessment using N MRI datasets from external clinical validation.

[0172]

[0173] As shown in Table 2, the accuracy is higher than 85% and the AUC is higher than 90%, indicating that the glioma grading collaborative compensation network exhibits good grading performance and generalization ability under all three modalities.

[0174] In simulation 3, this invention employs an IDH typing collaborative compensation network to perform IDH typing on publicly available preoperative diffuse glioma MRI datasets under three combined modalities: T1ce, Flair+T1ce, and Flair+T1ce+T2. The performance of the model's output IDH categories is evaluated using Accuracy, F1-score, and AUC metrics. The results are shown in Table 3.

[0175] Table 3. Quantitative assessment results of IDH subtyping using the publicly available preoperative diffuse glioma MRI dataset.

[0176]

[0177] The results in Table 3 show that the accuracy is higher than 90%, the F1-score is higher than 85%, and the AUC value reaches more than 95%, indicating that the IDH classification collaborative compensation network of the present invention has stable IDH classification capability under the three combined modal conditions.

[0178] Based on the quantitative assessment results of IDH genotyping, a significant visualization analysis is performed on the key image regions that the model focuses on during the IDH genotyping prediction process. Specifically, based on the discriminant feature information of the model's output of IDH type results for different combinations of image samples, a corresponding IDH genotyping significance heatmap is generated, such as... Figure 5 As shown.

[0179] Depend on Figure 5 The saliency heatmaps generated under the three modality combinations depict the distribution of attention to key imaging regions by the IDH subtyping collaborative compensation network in distinguishing between IDH mutant and IDH wild-type. High-response regions in IDH wild-type samples are mainly concentrated within the tumor and in areas related to necrosis features, while the regions of interest in IDH mutant samples exhibit a relatively diffuse distribution. This saliency heatmap demonstrates that this invention can adaptively mine potential imaging information related to subtyping by combining the imaging features corresponding to different molecular subtypes when performing IDH subtyping tasks, thereby improving the interpretability of subtyping predictions and providing reliable auxiliary analytical basis for IDH subtyping under modality-deficient conditions.

[0180] Simulation 4: This invention employs an IDH typing collaborative compensation network to perform IDH typing on M gliomas from an externally validated clinical MRI dataset under three combined modalities: T1ce, Flair+T1ce, and Flair+T1ce+T2. The performance of the model's output IDH categories is evaluated using Accuracy, F1-score, and AUC evaluation metrics. The results are shown in Table 4.

[0181] Table 4. Quantitative assessment results of IDH subtyping using an externally clinically validated M-case MRI dataset.

[0182]

[0183] Table 4 shows that, under independent clinical data conditions, the IDH typing collaborative compensation network still maintains stable and excellent IDH typing performance. The accuracy under the three modality combination conditions all exceed 90%, indicating that the IDH typing collaborative compensation network has good IDH typing generalization performance.

Claims

1. A collaborative compensation network for missing multimodal MRI data, characterized in that, include: Four convolutional encoders with identical structures were selected (1), each of which includes L layers of three-dimensional convolutional modules for independent multi-scale feature extraction of each MRI modality, where L≥5; Two identical dynamic adaptive unified interaction modules (2) are constructed to perform self-enhancement and cross-modal interaction on multimodal features. In the cross-modal interaction process, the modules utilize the collaborative modeling between existing modal features to achieve indirect expression and compensation of missing modal tumor features. Construct 0 / 1 indicator code (3) It includes a modality presence indicator variable set for each MRI modality, where 1 indicates modality presence and 0 indicates modality absence, which is used to encode the presence status of each modality feature in the input sample; A shared auxiliary decoder (4) with a K-layer three-dimensional decoding structure is selected. The structure of each layer and the connection between each layer are the same as those of the convolutional encoder. It is used to decode the multimodal enhancement features and output auxiliary segmentation features. K≥4; A missing modality master decoder (5) with a K-layer three-dimensional decoding structure is selected. The structure of each layer and the connection between each layer are the same as those of the convolutional encoder. It is used to decode the features after modality compensation and output the segmentation features under the missing modality condition. A tumor classification characterization module (6) is constructed, which includes a cascaded K-layer three-dimensional convolutional module, global average pooling, global max pooling and linear fully connected layers. Each convolutional module and the connection between each layer is the same as the convolutional encoder. It is used to characterize and model the segmentation features output by the missing modality master decoder and output the glioma grading and IDH classification prediction results. The convolutional encoder (1) is connected to the first dynamic adaptive unified interaction module (2). One output of this module is connected to the shared auxiliary decoder (4) to form an auxiliary path, and the other output is connected to the 0 / 1 indicator encoder. (3) The second dynamic adaptive unified interaction module (2), the missing modality master decoder (5), and the tumor classification characterization module (6) are connected in sequence to form the main path. These two paths form a collaborative compensation network through KL divergence constraints and mean square error constraints.

2. The network according to claim 1, characterized in that, The L-layer three-dimensional convolutional modules in the four identical convolutional encoders (1) are cascaded sequentially through convolution operations. Each module includes a 3×3×3 three-dimensional convolutional layer, an activation layer, and an instance normalization layer, which are connected sequentially.

3. The network according to claim 1, characterized in that, The dynamic adaptive unified interaction module (2) includes: The global context-aware submodule includes convolution operations, flattening mapping, self-attention operations, residual connections and normalization, feedforward mapping, normalization processing and spatial reconstruction, which are used to extract input features and global context information between modalities. The local information preservation submodule includes a feature weighted fusion operation with residual connections, used to extract input features and local detail information between modalities; The fusion submodule is used to fuse the output of the global context-aware submodule with the output of the local information preservation submodule to obtain the final output.

4. The network according to claim 1, characterized in that, The K-layer three-dimensional convolution module, global average pooling, global max pooling and linear fully connected layer in the tumor classification characterization module (6) are connected as follows: the output of the K-layer three-dimensional convolution module is processed by global average pooling and global max pooling respectively, and the results of these two pooling are concatenated by channels, and then the concatenated features are input into the linear fully connected layer to output the classification result.

5. The network according to claim 1, characterized in that, The implementation of the KL divergence constraint and mean square error constraint is shown below: The KL divergence constraint is used to reduce the overall tumor feature distribution formed after concatenation and convolution of tumor features from different modalities in the auxiliary decoder branch. Feature distribution in the main path decoder of missing modalities Differences between : ， in, =4 indicates the number of decoder layers. This represents the index, used to iterate through each element in the feature vector; The mean squared error constraint is used to reduce the tumor segmentation features output by the missing modality main path decoder. Tumor segmentation features output from each modality of the shared auxiliary decoder Differences between : ， in, The formula for calculating the mean squared error. , 1 indicates the output of the last layer.

6. A method for grading gliomas using the collaborative compensation network of claim 1, characterized in that, include: 1) 70% of the publicly available preoperative diffuse glioma MRI dataset was extracted according to its grade as the training set, 30% as the internal test set, and N clinical patients were collected as the external test set, N≥50 cases. 2) Convert the original DICOM format data of N clinical patients to NIfTI format, register the converted data to the MNI152 standard brain template, and then perform automatic skull dissection to obtain standardized brain tissue data; 3) Input the four MRI modalities of Flair, T1ce, T1 and T2 from the training set into the collaborative compensation network. Constrained by the gold standard for glioma segmentation and the gold standard for grading, the SGD optimizer combined with momentum parameters is used to iteratively update the model parameters until the preset number of iterations are obtained to obtain the trained collaborative compensation network. 4) Input the MRI modalities of four different combinations (Flair, T1ce, T1 and T2) from the internal test set of glioma grading and N external test sets into the trained collaborative compensation network, and output the glioma grading prediction results and significance heatmap. 5) Upsample the saliency heatmap of glioma grade output by the model to make its spatial resolution consistent with the original MRI image, and then overlay the upsampled saliency heatmap with the original MRI image to obtain the correspondence between different grades of glioma and image features.

7. The grading method according to claim 6, characterized in that, In step 2), the converted data is registered to the MNI152 standard brain template to perform automatic cranial dissection, obtaining standardized brain tissue data. This process includes: 2a) The converted brain imaging data are aligned to the MNI152 standard brain template using a rigid registration method to make the image data consistent with the standard template in spatial coordinates; 2b) On the registered brain image data, the HD-BET automatic skull stripping algorithm is applied to remove the skull and other non-brain tissue structures, leaving only the brain parenchyma tissue area, in order to obtain standardized brain tissue image data.

8. The grading method according to claim 6, characterized in that, In step 3), the four MRI modalities (Flair, T1ce, T1, and T2) from the training set are input into the collaborative compensation network. Constrained by the gold standards for glioma segmentation and grading, the model parameters are iteratively updated using an SGD optimizer combined with momentum parameters until a preset number of iterations are reached. This process includes: 3a) Set the optimizer parameters, including: a fixed learning rate of 0.0001, a momentum parameter of 0.9, a batch size of 1, and a weight decay coefficient of 0.001; 3b) In each training iteration, training samples are extracted from the training set and input into the collaborative compensation network. The segmentation results and grade prediction results output by the network are calculated using the Dice loss function to calculate the loss value between the segmentation results and the gold standard for glioma segmentation, and the grade prediction results are calculated using the cross-entropy loss function to calculate the loss value between the grade prediction results and the gold standard for glioma grading. 3c) Based on the loss value calculated in step 3b), the backpropagation algorithm is executed using the SGD optimizer combined with momentum parameters to update the model parameters of the cooperative compensation network; 3d) Repeat steps 3b) to 3c) until the predetermined number of training rounds is reached to obtain the glioma grading collaborative compensation network after training.

9. A method for IDH classification using the cooperative compensation network of claim 1, characterized in that, include: (1) 70% of the data from the publicly available preoperative diffuse glioma MRI dataset was extracted according to its IDH type as the training set, 30% as the internal test set, and M clinical patients were collected as the external test set, M≥65 cases. (2) Convert the original DICOM format data of M clinical patients into NIfTI format, register the converted data into the MNI152 standard brain template, and then perform automatic skull dissection to obtain standardized brain tissue data; (3) Input the four MRI modalities of Flair, T1ce, T1 and T2 from the training set into the collaborative compensation network, and iteratively train the network by constraining it with the gold standard for glioma segmentation and the gold standard for IDH classification; (4) Input the four different combinations of MRI modalities of Flair, T1ce, T1 and T2 from the internal test set of IDH subtyping and the N external test sets into the trained collaborative compensation network, and output the IDH subtyping prediction results and significance heatmap. (5) Upsample the IDH type saliency heatmap output by the model to make its spatial resolution consistent with the original MRI image, and overlay the upsampled saliency heatmap with the original MRI image to obtain the correspondence between different IDH types and image features.

10. The IDH classification method according to claim 9, characterized in that, In (3), the Flair in the training set, Four MRI modalities (T1ce, T1, and T2) are input into the collaborative compensation network. The network is iteratively trained using the gold standard for glioma segmentation and the gold standard for IDH classification, achieving the following: (3a) Set the optimizer parameters, including: using a warm-up mechanism to dynamically adjust the learning rate, setting the momentum to 0.90, the batch size to 1, the weight decay coefficient to 0.0005, and the initial learning rate to 0.0002; (3b) In each training iteration, training samples are extracted from the training set and input into the collaborative compensation network. The segmentation results and IDH prediction results output by the network are calculated using the Dice loss function to calculate the loss value between the segmentation results and the gold standard for glioma segmentation, and the loss value between the IDH prediction results and the gold standard for IDH subtyping is calculated using the cross-entropy loss function. (3c) Based on the loss value calculated in step (3b), backpropagation is performed using the AdamW optimizer to update the model parameters of the collaborative compensation network; (3d) Repeat steps (3b) to (3c) until the predetermined number of training rounds is reached to obtain the IDH classification collaborative compensation network after training.

Images