Disease of consciousness classification method based on multi-modal image and deep learning

Abstract

The invention relates to a disturbance of consciousness disease classification method based on multi-modal images and deep learning. The problems that in the prior art, single-mode medical image analysis precision is low, and medical diagnosis is affected are solved. The method comprises the following steps: S1, carrying out data cleaning and data preprocessing work on original data; s2, constructing a network model to process and analyze data; s3, selecting three different fusion modes to obtain three classification results; and S4, explaining the decision process of the deep learning model by using a visualization method. The method has the advantages that the problems of high misdiagnosis rate and insufficient model interpretability caused by dependence on subjective behavior observation in the prior art are solved; according to the method, the limitation existing when the consciousness disorder level is analyzed only by using a single mode is overcome, the ability of the deep learning model to accurately classify the consciousness disorder level of the patient is improved, a powerful tool is provided for accurately diagnosing the consciousness disorder disease, and the clinical management and service level of the consciousness disorder disease can be enhanced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image processing, in particular to a consciousness disorder disease classification method based on multi-modal images and deep learning. BACKGROUND

[0002] Disorders of Consciousness (DOC) is a class of nervous system diseases that seriously affect the patient's consciousness state, usually caused by traumatic brain injury, ischemic stroke or brain hypoxia, etc., wherein the patient's consciousness state level can be divided into Unresponsive Wakefulness Syndrome (UWS) and Minimally Conscious State (MCS). UWS patients have no obvious self-awareness and cannot perceive or understand the external environment. MCS patients have a certain degree of consciousness and can make simple reactions to external stimuli in some cases, but the overall level of consciousness is still low. In the field of consciousness disorders, accurately identifying the consciousness level of patients is a key problem in this field of research, because it will affect the prognosis of patients, treatment plan, and ethical decisions (such as life termination decisions). At present, the method of using behavior assessment scale is mainly used in clinic to diagnose the consciousness disorder level of patients, but this method is difficult to find the potential consciousness of patients, and the error rate is as high as 40%, so how to improve the accurate identification of the consciousness state of DOC patients has become a problem to be solved in the clinical and research fields.

[0003] The classification and diagnosis of DOC patients mainly rely on behavioral assessment tools, such as the commonly used clinical coma recovery scale (CRS-R), which quantifies the patient's response level through six dimensions (visual, auditory, motor response, speech function, communication, and arousal level), and assigns scores to each response to distinguish different types of consciousness disorders. However, CRS-R relies on subjective judgment and is limited by some objective factors that interfere with the comprehensive assessment of patient consciousness. In addition to assessment scales, neuroimaging techniques can also help us distinguish DOC patients, such as Positron Emission Tomography (PET), Computed Tomography (CT), and Magnetic Resonance Imaging (MRI). PET can provide information on brain metabolic activity, while CT presents high-resolution anatomical information of the brain structure. The fusion of these two modalities for multi-modal analysis can provide rich image data support for the diagnosis of DOC patients and provide a new analysis perspective for consciousness disorder research, with broad application prospects in consciousness disorder level stratification research.

[0004] Currently, deep learning technology has shown great potential in medical image analysis. In the field of DOC research, the application of deep learning is still in the exploratory stage, and its potential in the recognition and classification of DOC patients' consciousness has not been fully tapped. In particular, the deep fusion analysis of multi-modal images such as PET and CT still lacks systematic research and practice.

[0005] In the field of brain science research, some studies use PET images combined with deep learning technology to diagnose different neurodegenerative diseases. By collecting PET image datasets of three different groups of subjects (healthy controls, Alzheimer's disease, and frontotemporal dementia), after data preprocessing, a convolutional neural network is used for prediction, resulting in a three-class result, demonstrating the potential of using PET images combined with deep learning technology to assist in the diagnosis of different neurodegenerative diseases, and proving that using deep learning models can improve prediction accuracy. In other fields, some studies use deep learning methods and PET-CT multi-modal images to explore models that can accurately distinguish two different Programmed Death-Ligand 1 (PD-L1) expression states in non-small cell carcinoma patients. After preprocessing the data, different fusion strategies are used to predict the different expression states of PD-L1 (positive, negative), and the potential of PET-CT multi-modal images combined with deep learning technology in this field is explored.

[0006] Although PET can provide brain metabolic activity information to help the model to classify, due to the lack of high-resolution anatomical information, it cannot fully explain the multidimensional characteristics of the brain, resulting in limited classification accuracy; in other research fields, deep learning technology can utilize the multidimensional information provided by PET-CT multi-modal images to make predictions, but due to the black box characteristics of the deep learning model, the thinking and judgment process of the model is not clear, which affects the credibility of the model, and at present, in the field of consciousness disorder diseases, there is still a lack of deep learning method research using multi-modal images. SUMMARY

[0007] The purpose of the present application is to solve the above problems, and provide a consciousness disorder disease classification method based on multi-modal images and deep learning, which can realize the accurate typing of healthy control (HC), MCS and UWS.

[0008] In order to achieve the above purpose, the following technical scheme is adopted: a consciousness disorder disease classification method based on multi-modal images and deep learning, comprising the following steps:

[0009] S1, data cleaning and data preprocessing of original data;

[0010] S2, constructing a network model to process and analyze data;

[0011] S3, selecting three different fusion methods to obtain three classification results;

[0012] S4, using a visualization method to explain the decision-making process of the deep learning model.

[0013] Step S1 comprises the following steps:

[0014] S11, original data cleaning:

[0015] Use RadiAnt DICOM medical image viewing tool to check and classify the slice quality of the original data, and exclude data with blurred slices and missing slices;

[0016] Statistically arrange whether the patient has undergone multiple diagnoses, whether there is a consciousness disorder state transfer, and record the above conditions;

[0017] S12, data preprocessing:

[0018] Install SPM12 tool, after converting DICOM data to NIfTI format data, select Batch batch processing, enter the data preprocessing interface, select the preprocessing steps to be performed: Normalise (standardization), Smooth (smoothing), then select the file to be preprocessed, the related parameters remain default, and the data preprocessing work is completed after running.

[0019] Step S2 comprises the following steps:

[0020] S21, based on the traditional EfficientNet-B3 network, a 3D EfficientNet-B3 network model capable of processing 3D data is constructed, and the PET-CT image of each patient after resampling and normalization is taken as the input of the network model;

[0021] S22, the last fully connected layer is modified to output three probability vectors, and each vector component is used to represent the probability that the PET-CT fusion image is predicted as HC, UWS or MCS.

[0022] Step S3 comprises the following steps:

[0023] S31, early fusion: by directly splicing the original data of PET and CT, the fusion is performed at the input data level, so as to input the PET-CT multi-modal image;

[0024] S32, intermediate fusion: in the deep learning process, the features of PET and CT data are extracted respectively, and then joint analysis is performed;

[0025] S33, late fusion: each modality model is trained independently, and the prediction results are fused in the final decision stage.

[0026] Step S31 specifically comprises the following steps:

[0027] S311, resample the PET and CT images to 79x95x79, and then normalize them to the range of [0, 1];

[0028] S312, at the earliest stage of inputting the network, the processed PET and CT images are directly spliced in the channel dimension;

[0029] S313, the network backbone adopts the 3D EfficientNet-B3 structure, and four residual connection paths are established from the shallow layer to the deep layer;

[0030] S314, the final classification task is completed through global average pooling and two fully connected layers.

[0031] Step S32 specifically comprises the following steps:

[0032] S321, the pre-processing work of PET and CT images is independently completed;

[0033] S322, the modality-specific features are extracted through the respective shallow network, and a 16x20x24x20 feature map is obtained;

[0034] S323, replace the original second MBConv6 module with a feature fusion layer;

[0035] S324, in the feature fusion layer, the feature maps of the two branches are spliced in the channel dimension;

[0036] S325, get a fusion feature with a size of 48x10x12x10 through 3D convolution and BatchNorm processing;

[0037] S326, further feature extraction and optimization through a shared deep network;

[0038] S327, complete the classification task through global average pooling and fully connected layer.

[0039] 7. The consciousness disorder disease classification method based on multi-modal images and deep learning according to claim 6, characterized in that step S33 specifically comprises the following steps:

[0040] S331, each modality first undergoes independent preprocessing;

[0041] S332, feature extraction through a complete 3D EfficientNet-B3 network of each;

[0042] S333, get 1024-dimensional features through 1x1x1 convolution, and then get feature vectors through global average pooling;

[0043] S334, each branch produces independent prediction results through its own fully connected layer; meanwhile, two 1024-dimensional feature vectors are spliced into a 2048-dimensional joint feature;

[0044] S335, produce joint prediction results through another fully connected layer, and integrate through a weighted summation voting mechanism to produce the final prediction results.

[0045] The weighted summation method assigns weights to the outputs of PET, CT and PET-CT modalities, and sums these weighted results to realize fusion. The basic formula of the weighted summation method is:

[0046]

[0047] where S is the final score of class c (HC, VS, MCS),

[0048] n is the number of modalities (n=3), ω i is the weight of the i-th modality,

[0049] P i is the prediction probability of the i-th modality for class c (i=1, 2, 3);

[0050] The basis formula for predicting the category is:

[0051]

[0052] Step S4 includes the following steps:

[0053] S41 Calculate the importance weight of each feature channel As shown in the following formula:

[0054]

[0055] Wherein, represents the gradient calculation of the feature map,

[0056] Z represents the total number of feature map pixels (3x3x3=27),

[0057] y c represents the score corresponding to the category c,

[0058] represents the data of the coordinate (i, j) on the kth channel of the feature layer A,

[0059] Importance weight represents the importance of the kth feature map to the category c.

[0060] S42, combine the weight with the feature map A k Linearly, and then get the class activation map through the ReLU function, as shown in the following formula:

[0061]

[0062] Wherein, represents the heat map of the category c,

[0063] The role of ReLU is to only retain the area that has a positive contribution to the category c.

[0064] In step S42, the heat map is up-sampled to the original map size, and is superimposed with the original map. The highlighted area represents the position that the model focuses on when predicting the category c again.

[0065] Compared with the prior art, the present application has the advantages that: the problems of high misdiagnosis rate caused by relying on subjective behavior observation, significant one-sidedness of single-modal image analysis, lack of standardized process in data preprocessing, and insufficient model interpretability in the prior art are solved; the limitations in analyzing the level of consciousness disorder using only a single modality are overcome, the ability of the deep learning model to accurately classify the level of consciousness disorder of patients is improved, a powerful tool for accurately diagnosing consciousness disorder diseases is provided, and the clinical management and service level of consciousness disorder diseases are helped to be enhanced. BRIEF DESCRIPTION OF DRAWINGS

[0066] Figure 1 is the overall flowchart of the present application;

[0067] Figure 2 is the process diagram of data format conversion, standardization and smoothing using SPM tool in the present application;

[0068] Figure 3 is the detailed parameter of the 3D EfficientNet-B3 network model constructed in the present application;

[0069] Figure 4 is the network model schematic diagram of the early fusion strategy of PET-CT adopted in the present application.

[0070] Figure 5 is the network model schematic diagram of the medium-term fusion strategy of PET-CT adopted in the present application.

[0071] Figure 6 is the network model schematic diagram of the late fusion strategy of PET-CT adopted in the present application. DETAILED DESCRIPTION

[0072] The present application will be further described in detail below in combination with the drawings and specific embodiments.

[0073] As shown in Figure 1 , a consciousness disorder disease classification method based on multi-modal image and deep learning, characterized in that the method comprises the following steps:

[0074] S1, data cleaning and data preprocessing work are performed on the original data;

[0075] S2, a network model is constructed to process and analyze the data;

[0076] S3, three different fusion methods are selected to obtain three classification results;

[0077] S4, a visualization method is used to explain the decision-making process of the deep learning model.

[0078] Step S1 comprises the following steps:

[0079] S11, raw data cleaning:

[0080] The slice quality of the raw data is checked and classified using the RadiAnt DICOM medical image viewing tool, and data with blurred slices or missing slices is excluded.

[0081] Statistics are made on whether the patient has undergone multiple diagnoses, whether there is a state transfer of consciousness disorder (for example, the first diagnosis is UWS, and the second diagnosis is MCS), and the above cases are recorded.

[0082] S12, in order to make the input data suitable for deep learning model analysis, the data is preprocessed using the SPM12 (statistical parameter mapping 12) tool of Matlab, as shown in Figure 2 .

[0083] The specific steps are as follows:

[0084] Install the SPM12 tool, and enter spm pet in the interface to open Figure 2 (Left) interface. Select DICOMImport to enter Figure 2 the interface shown in the left (middle), convert the DICOM data (format of raw data) to NIfTI format (format suitable for SPM12 tool processing). After data conversion, select Batch batch processing as shown in Figure 2 (Left) to enter Figure 2 the data preprocessing interface shown in (right), select the preprocessing steps to be performed: Normalise (standardization), Smooth (smoothing), and then select the file to be preprocessed. The relevant parameters remain default. After running, the data preprocessing work is completed.

[0085] Step S2 includes the following steps:

[0086] S21, based on the traditional EfficientNet-B3 network, a 3DEfficientNet-B3 network model capable of processing 3D data is constructed, and the specific network parameters of the network model are as shown in Figure 3 . The PET-CT image (size 79x95x79) of each patient after resampling and normalization is used as the input of the network model;

[0087] S22, modify the last fully connected layer to output 3 probability vectors, and each vector component is used to represent the probability that the PET-CT fusion image is predicted as HC, UWS or MCS. Through continuous iteration, the network model with the highest accuracy and optimal performance is found as the optimal model for subsequent use.

[0088] Step S3 comprises the following steps:

[0089] S31, early fusion: fusion at the input data level by directly splicing the raw data of PET and CT, thereby inputting the PET-CT multi-modal image;

[0090] S32, intermediate fusion: in the deep learning process, features are extracted from PET and CT data respectively, and then joint analysis is performed; as shown in Figure 5 .

[0091] S33, late fusion: each modality model is trained independently, and the prediction results are fused in the final decision-making stage; as shown in Figure 6 .

[0092] Step S31 specifically comprises the following steps:

[0093] S311, resample the PET and CT images to 79x95x79, and then normalize them to the range [0, 1];

[0094] S312, at the earliest stage of the input network, directly splice the processed PET and CT images in the channel dimension; (from 1+1 channels to 2 channels, the spatial dimension remains unchanged).

[0095] S313, the network backbone adopts the 3D EfficientNet-B3 structure, but the special point is that a cross-layer residual connection strategy is designed: four residual connection channels are established from the shallow layer to the deep layer; as shown in Figure 4 . Each residual connection connects two MBConv modules. This design allows low-level features in the shallow layer to be directly passed to the deep layer, effectively alleviating the gradient vanishing problem, while achieving multi-scale feature fusion, allowing the network to utilize feature information at different levels simultaneously.

[0096] S314, complete the final classification task through global average pooling and two fully connected layers.

[0097] Among them, a Dropout layer is inserted between the average pooling layer and the first fully connected layer, which randomly disconnects 30% of the neuron connections, which can prevent over-reliance between neurons and reduce the risk of overfitting. The advantage of this fusion strategy is that it can preserve the complete original information of the two modalities at the earliest stage, allowing the network to learn the mutual relationship between modalities independently, while optimizing the feature extraction and fusion process through various attention mechanisms and feature enhancement methods.

[0098] Step S32 specifically comprises the following steps:

[0099] S321, independently complete the pretreatment work of PET and CT images; (resample to 79x95x79 size, normalize to [0, 1] range),

[0100] S322, extract modality-specific features through respective shallow networks (including 3D convolution, MBConv1 and MBConv6) to obtain a feature map of 16x20x24x20;

[0101] S323, replace the original second MBConv6 module with a feature fusion layer;

[0102] S324, in the feature fusion layer, the feature maps of the two branches are spliced in the channel dimension (becoming 32x20x24x20);

[0103] S325, obtain a fused feature of 48x10x12x10 through 3D convolution and BatchNorm processing;

[0104] S326, further feature extraction and optimization through a shared deep network;

[0105] S327, complete the classification task through global average pooling and a fully connected layer.

[0106] This design maintains the ability of modality-specific feature learning, effectively fuses features at the appropriate level, and ensures full use of deep features through residual connection.

[0107] Step S33 specifically includes the following steps:

[0108] S331, each modality first undergoes independent preprocessing; (resample to 79x95x79 size, normalize to [0, 1] range),

[0109] S332, extract features through a complete 3D EfficientNet-B3 network for each modality; including an initial 3D convolution layer (32 channels) and a series of MBConv modules (from 32 channels to 256 channels),

[0110] S333, obtain a 1024-dimensional feature through 1x1x1 convolution, and then obtain a feature vector through global average pooling;

[0111] S334, each branch produces an independent prediction result through its own fully connected layer; meanwhile, two 1024-dimensional feature vectors are spliced into a 2048-dimensional joint feature;

[0112] S335, produce a joint prediction result through another fully connected layer, and integrate through a weighted summation voting mechanism to produce the final prediction result.

[0113] The weighted summation method assigns weights to the outputs of PET, CT, and PET-CT modalities (adaptively adjusted according to the validation set performance and prediction confidence on different modalities), and sums the weighted results to realize fusion. The basic formula of the weighted summation method is:

[0114]

[0115] where S is the final score of class c (HC, VS, MCS),

[0116] n is the number of modalities (n = 3), ω i is the weight of the i-th modality,

[0117] P i is the prediction probability of the i-th modality for class c (i = 1, 2, 3);

[0118] The formula for predicting the class is:

[0119]

[0120] That is, the class with the maximum final score S is selected as the prediction result. For late fusion, this design can maximize the independence of each modality feature and avoid information loss caused by early feature interaction, while effectively fusing features through the last fully connected layer.

[0121] In step S4, the Grad-CAM method is used to generate a heat map to locate the image region on which the model relies for decision-making; thereby improving the model's explainability. Specifically, the last convolutional layer in the model (the last MBConv, with a size of 256x3x3x3) is selected, and a visual heat map is generated by calculating the gradient of its feature map to the target class score. First,

[0122] Step S4 includes the following steps:

[0123] S41 Calculate the importance weight of each feature channel as shown in the following formula:

[0124]

[0125] where, represents the gradient calculation of the feature map,

[0126] Z represents the total number of feature map pixels (3x3x3 = 27),

[0127] y c represents the score corresponding to class c,

[0128] Represents the data of coordinates (i, j) on the kth channel of feature layer A,

[0129] Importance Weight It represents the importance of the k-th feature map to category c.

[0130] S42, weight With feature map A k Linear combination, and then the ReLU function is used to obtain the class activation map, as shown in the following formula:

[0131]

[0132] in, represents the heat map of category c,

[0133] The role of ReLU is to retain only the regions that contribute positively to category c.

[0134] In step S42, the heat map is upsampled to the original image size and superimposed with the original image. The highlighted area represents the location that the model focuses on when predicting category c.

[0135] In our dual-modality model, heatmaps are generated for both the PET and CT branches. This allows us to visualize the regions of interest for each modality and, by comparing the two heatmaps, understand how the model leverages the complementary information from the different modalities. This visualization approach helps physicians understand the underlying basis for the model's decisions and verify whether the model is truly focusing on disease-relevant regions, thereby enhancing the model's interpretability and credibility.

[0136] In summary, the principle of this embodiment is: by using images of two modalities, positron emission tomography (PET) and computed tomography (CT), different fusion strategies are adopted to synthesize multimodal images, and by comparing different deep learning architectures, an optimal deep learning model is constructed. The deep learning technology is combined to enable the deep learning model to automatically extract features of two different dimensions, brain metabolism and brain structure. At the same time, visualization technology is combined to explain the classification basis of the model, so as to make an accurate classification of the patient's level of consciousness, providing theoretical support for the optimization design and clinical promotion of the model.

[0137] The specific embodiments described herein are merely illustrative of the spirit of the present invention. Persons skilled in the art may make various modifications, additions, or substitutions to the described specific embodiments without departing from the spirit of the present invention or exceeding the scope of the appended claims.

Claims

1. A method for classifying disorders of consciousness based on multimodal imaging and deep learning, characterized in that: This method comprises the following steps: S1. Perform data cleaning and data preprocessing on the original data; S2, building a network model to process and analyze data; S3, select three different fusion methods to obtain three classification results; S4. Use visualization methods to explain the decision-making process of deep learning models.

2. The method for classifying disorders of consciousness based on multimodal imaging and deep learning according to claim 1, characterized in that: Step S1 includes the following steps: S11. Raw data cleaning: Use RadiAnt DICOM medical imaging viewing tool to check and classify the slice quality of the original data, and exclude data with blurred or missing slices; Collect statistics to determine whether the patient has undergone multiple diagnoses and whether there have been any shifts in consciousness disorder, and record the above information; S12. Data preprocessing: Install the SPM12 tool and convert the DICOM data into NIfTI format. After the data conversion is complete, select Batch processing to enter the data preprocessing interface. Select the preprocessing steps to be performed: Normalize, Smooth, and then select the file to be preprocessed. Keep the relevant parameters as default. After running, the data preprocessing work is completed.

3. The method for classifying disorders of consciousness based on multimodal imaging and deep learning according to claim 2, characterized in that: Step S2 includes the following steps: S21. Based on the traditional EfficientNet-B3 network, a 3DEfficientNet-B3 network model for processing 3D data was constructed, using the resampled and normalized PET-CT images of each patient as input to the network model. S22. Modify the last fully connected layer so that its output is three probability vectors, where each vector component is used to represent the probability that the PET-CT fusion image is predicted to be HC, UWS, or MCS.

4. The method for classifying disorders of consciousness based on multimodal imaging and deep learning according to claim 3, characterized in that: Step S3 includes the following steps: S31. Early fusion: By directly splicing the raw data of PET and CT, fusion is performed at the input data level to input PET-CT multimodal images; S32, mid-term fusion: In the deep learning process, features are extracted from PET and CT data separately and then jointly analyzed; S33, Late Fusion: Train each modality model independently and fuse the prediction results in the final decision stage.

5. The method for classifying disorders of consciousness based on multimodal imaging and deep learning according to claim 4, characterized in that: Step S31 specifically includes the following steps: S311, resampling the PET and CT images to 79×95×79, and then normalizing them to the range of [0, 1]; S312, at the earliest stage of the input network, directly concatenate the processed PET and CT images in the channel dimension; The S313 network backbone adopts the 3D EfficientNet-B3 structure, establishing four residual connection paths from the shallow layer to the deep layer; S314, complete the final classification task through global average pooling and two layers of fully connected layers.

6. The method for classifying disorders of consciousness based on multimodal imaging and deep learning according to claim 4, characterized in that: Step S32 specifically includes the following steps: S321, independently complete pre-processing of PET and CT images; S322, extract modality-specific features through their respective shallow networks to obtain a 16×20×24×20 feature map; S323, replace the original second MBConv6 module with the feature fusion layer; S324: In the feature fusion layer, the feature maps of the two branches are spliced ​​in the channel dimension; S325, through 3D convolution and BatchNorm processing, a fusion feature of size 48×10×12×10 is obtained; S326, further feature extraction and optimization through the shared deep network; S327, complete the classification task after global average pooling and fully connected layer.

7. The method for classifying disorders of consciousness based on multimodal imaging and deep learning according to claim 6, characterized in that: Step S33 specifically includes the following steps: S331, each modality first undergoes independent preprocessing; S332, feature extraction through their respective complete 3D EfficientNet-B3 networks; S333, obtain 1024-dimensional features through 1×1×1 convolution, and then obtain the feature vector through global average pooling; S334, each branch generates an independent prediction result through its own fully connected layer; at the same time, the two 1024-dimensional feature vectors are spliced ​​into a 2048-dimensional joint feature; S335, after another fully connected layer, a joint prediction result is generated, which is integrated through a weighted sum voting mechanism to produce the final prediction result.

8. The method for classifying disorders of consciousness based on multimodal imaging and deep learning according to claim 7, wherein the weighted summation method assigns weights to the outputs of the three modalities of PET, CT, and PET-CT, and sums these weighted results to achieve fusion. The basic formula of the weighted summation method is as follows: in, S is the final score of category c (HC, VS, MCS), n is the number of modes (n=3), ω i is the weight of the i-th mode, P i is the predicted probability of the i-th modality for category c (i = 1, 2, 3); The formula for predicting the category is:

9. The method for classifying disorders of consciousness based on multimodal imaging and deep learning according to claim 8, wherein step S4 comprises the following steps: S41 calculates the importance weight of each feature channel As shown in the following formula: in, Represents the gradient calculation of the feature map, Z represents the total number of pixels in the feature image (3×3×3=27), y c represents the score corresponding to category c, Represents the data of coordinates (i, j) on the kth channel of feature layer A, Importance Weight Indicates the importance of the k-th feature map to category c; S42, weight With feature map A k Linear combination, and then the ReLU function is used to obtain the class activation map, as shown in the following formula: in, represents the heat map of category c, The role of ReLU is to retain only the regions that contribute positively to category c.

10. According to the method for classifying consciousness disorders based on multimodal imaging and deep learning as described in claim 8, in step S42, the heat map is upsampled to the original image size and superimposed with the original image. The highlighted area represents the location that the model focuses on when re-predicting category c.

Images