Severe depression classification method and system based on multi-stage feature pyramid optimal transmission fusion

Abstract

The invention discloses a major depression classification method based on multi-stage feature pyramid optimal transmission fusion. The method comprises the following steps: inputting a 3D whole-brain structure magnetic resonance image; labeling all the images in the 3D image set according to the severe depression level, and forming a data set by the 3D image set and labels corresponding to the images in the 3D image set; constructing a prediction network based on an end-to-end neural network framework, wherein the prediction network comprises an image preprocessing module, a multi-scale feature extraction module, a pyramid optimal transmission fusion module and a prediction module; training the prediction network by using the data set to obtain a classification model; and inputting the 3D whole-brain structure magnetic resonance image of the subject into a classification model to output a classification result of the subject. The invention further provides a major depression classification system. According to the method provided by the invention, an accurate and explainable major depression classification result is given according to the 3D T1 sMRI features, and a more perfect reference is provided for a subsequent treatment scheme.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image engineering, and particularly relates to a method and system for classifying major depressive disorder based on optimal transport fusion of multi-stage feature pyramids. Background Art

[0002] Major depressive disorder (MDD) is considered one of the most common mental health problems globally, with devastating social and personal consequences and severely affecting the social function of patients. It has attracted increasing attention worldwide, affecting more than 350 million people. Against this backdrop, the Lancet-World Psychiatric Association Report on Depression states that 5% of adults globally are troubled by depression every year.

[0003] In the process of identifying depression, existing brain imaging technologies have significantly increased our understanding of the pathology of brain diseases. These technologies help discover biomarkers valuable for diagnosis. Among these technologies, structural magnetic resonance imaging (sMRI) and functional magnetic resonance imaging (fMRI) have become widely used brain imaging methods in MDD detection due to their moderate cost, non-invasiveness, and practical applicability. In recent years, deep learning methods for MDD diagnosis by combining fMRI and sMRI have received increasing attention. In the field of deep learning, researchers have tried to use different neural network architectures, such as convolutional neural networks (CNNs), residual networks (ResNets), DenseNets, etc., and explored the impact of multi-scale feature maps on MDD classification.

[0004] Patent document CN117982104A discloses a method for brain age assessment based on optimal transport theory, including: Step 1, designing an end-to-end neural network architecture, including an OTFPF module, a 3D OLConvNeXt module, and a fusion module. Step 2, in the first three paths, each path contains four 3D OLConvNeXt modules. Step 3, fusing the feature pyramid output by 3D FPFN through OTEM, and OTEM uses the Kantorovich formula. Step 4, introducing a new subject-level ranking loss, and using linear programming and strongly convex regularization techniques to convert the ranking operation into an effectively optimizable projection operation for optimizing the network. The loss function consists of the subject-level ranking loss and the mean square error loss, where the subject-level ranking loss can be efficiently calculated through the soft ranking strategy. Step 5, connecting the outputs of the four paths and generating the final brain age estimation result through a multi-layer perceptron (MLP).

[0005] Patent document CN109035137A discloses a multimodal medical image fusion method based on optimal transport theory, belonging to the technical fields of computer graphics and medical image processing. This method includes the following steps: S1: Input two different-modal medical images to be fused; S2: Invert the source images and perform a normalization operation on the inverted source images; S3: Calculate the optimal transport centroid between the source images after the normalization operation; S4: Invert the images according to the optimal transport centroid and reconstruct the resulting fused image. The method of the present invention can eliminate more noise while retaining more source image information, and at the same time reduce the computational complexity to perform a faster and more effective fusion of multimodal medical images. Summary of the Invention

[0006] The purpose of the present invention is to provide a method and system for classifying major depressive disorder based on optimal transport fusion of multi-stage feature pyramids. This method aims to give an accurate and interpretable classification result for 3D T1 sMRI features of major depressive disorder, providing a more comprehensive reference for subsequent treatment plans.

[0007] To achieve the first object of the present invention, the following technical solution is provided: A method for classifying major depressive disorder based on optimal transport fusion of multi-stage feature pyramids, including the following steps: Input a 3D whole-brain structural magnetic resonance image, and perform image processing on the 3D whole-brain structural magnetic resonance image to obtain a 3D image set including a gray matter image, a white matter image, and a cerebrospinal fluid image; Label all the images in the 3D image set with the major depressive disorder level, and form a data set by combining the 3D image set and the labels corresponding to the images in the 3D image set; Construct a prediction network based on an end-to-end neural network framework. The prediction network includes an image preprocessing module, a multi-scale feature extraction module, a pyramid optimal transport fusion module, and a prediction module; The image preprocessing module is used to perform image processing on the input 3D whole-brain structural magnetic resonance image to output the corresponding gray matter image, white matter image, and cerebrospinal fluid image; The multi-scale feature extraction module includes three parallel feature extraction branches. A plurality of feature extractors with different scales are connected in series on each feature extraction branch. The plurality of feature extractors with different scales are used to extract the image features of the images input to the feature extraction branch where they are located. Each feature extractor is provided with a first output end and a second output end. The first output end is used to input the extracted image features into the feature extractor of the next scale. The image features include white matter features, cerebrospinal fluid features, and gray matter volume features; The pyramid optimal transport fusion module aligns and fuses the image features of different sizes extracted by the multi-scale feature extraction module based on the feature pyramid structure to output the corresponding final image features; The prediction module makes predictions based on the input final image features to output the corresponding prediction results; Use the dataset to train the prediction network to obtain a classification model for major depressive disorder classification; Input the 3D whole-brain structural magnetic resonance image of the subject into the classification model to output the classification result of the subject.

[0008] The present invention introduces the Multi-scale Feature Pyramid Optimal Transport Fusion (MFPOTF) network as an end-to-end neural network framework for this task. By taking the gray matter image, white matter image, and cerebrospinal fluid (CSF) image of 3D sMRI as inputs, and using the semi-multi-modal information of each brain and the multi-level semantic feature pyramid information, the classification performance of the final model is significantly improved.

[0009] Specifically, the pyramid optimal transport fusion module includes two parts. The first part includes the first OTGF sub-module and the TAF sub-module. The first OTGF sub-module is used to gather the image features output by the second output ends of the same-scale feature extractors in different feature extraction branches, and align and fuse the tail image features at the same scale to output the first fusion feature at the corresponding scale. The TAF sub-module is used to perform adaptive calculation and prediction feature fusion on the first features at all scales to output the second fusion feature. The image features output by the second output end are divided into two paths. One path is input into the first OTGF sub-module after being upsampled and added to the image features of the second output end of the previous stage. If the second output end is the first feature extractor in the feature extraction branch, upsampling is not required; The second part includes the second OTGF sub-module. The second OTGF sub-module is used to gather the image features output by the first output ends of the feature extractors at the very ends of all feature extraction branches, and perform alignment and fusion to output the third fusion feature; Finally, the second fusion feature and the third fusion feature are feature-stitched as the final image features and input into the prediction module.

[0010] Specifically, the alignment and fusion include optimal transport fusion and Gaussian alignment fusion, and the process is as follows: Flatten the input white matter features, cerebrospinal fluid features, and gray matter volume features to construct the corresponding feature matrices; The optimal transport fusion: Select the gray matter volume feature as the starting feature, and use the feature matrix to calculate the distance matrix between the starting feature and the other features respectively; Construct a corresponding transport matrix equation using the distance matrix between the Kantorovich formula and the features, and solve the transport matrix equation through a classical linear programming algorithm to obtain the optimal transport matrix; Map the white matter features and cerebrospinal fluid features through the optimal transport matrix to obtain the reconstructed white matter features and reconstructed cerebrospinal fluid features; Perform weighted summation on the reconstructed white matter features, reconstructed cerebrospinal fluid features, and gray matter volume features to output the first sub-feature; The Gaussian alignment fusion: perform splicing in the dimension where the gray matter volume feature is located based on the constructed feature matrix to obtain a spliced feature matrix; Calculate the corresponding global mean and standard deviation based on the spliced feature matrix; Standardize each feature according to the calculated global mean and standard deviation, and perform weighted summation on the three standardized features to output the second sub-feature; Fuse the first sub-feature and the second sub-feature to output the corresponding fused feature.

[0011] Specifically, during the solution process of the transport matrix equation, it is required that the rows and columns of the transport matrix are equal to 1.

[0012] Specifically, the distance matrix is calculated using the squared Euclidean distance, and its expression is as follows: ; where, a and b are two feature vectors, ak and bk are the k th elements of these two vectors respectively, and N is the dimension of the feature vector.

[0013] Specifically, the flattening process is based on the tensor expression of the input feature to perform flattening to obtain the corresponding feature matrix , where .

[0014] Specifically, the expression of the first sub-feature is as follows: ; where, represents the white matter feature, represents the cerebrospinal fluid feature, represents the gray matter volume feature, represents the optimal transport matrix between the gray matter volume feature and the white matter feature, represents the optimal transport matrix between the gray matter volume feature and the cerebrospinal fluid feature, represents the first sub-feature.

[0015] Specifically, the expression of the second sub-feature is as follows: ; wherein, represents the white matter feature, represents the cerebrospinal fluid feature, represents the gray matter volume feature, represents the global mean, represents the standard deviation, represents the second sub-feature.

[0016] Specifically, the TAF sub-module constructs Q, K, and V matrices as the second fusion feature through feature stacking, and its expression is as follows: ; wherein, represents the predicted image features corresponding to m scales, B represents the batch size, represents the feature length.

[0017] To achieve the second object of the present invention, the following technical solution is provided: a severe depression classification system for implementing the steps of the above-mentioned severe depression classification method based on multi-stage feature pyramid optimal transport fusion.

[0018] Compared with the prior art, the beneficial effects of the present invention are as follows: Taking the gray matter image, white matter image, and cerebrospinal fluid (CSF) image of 3D sMRI as inputs, and utilizing the semi-multi-modal information of each brain and the multi-level semantic feature pyramid information, compared with the prior art, more comprehensive abnormal information can be obtained, thereby providing the accuracy and interpretability of the classification task. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a flowchart of the severe depression classification method based on multi-stage feature pyramid optimal transport fusion provided in this embodiment; Figure 2 is a working schematic diagram of the OTGF sub-module provided in this embodiment; Figure 3 is a working schematic diagram of the optimal transport fusion provided in this embodiment; Figure 4 is a working schematic diagram of the Gaussian alignment fusion provided in this embodiment; Figure 5 is a working schematic diagram of the TAF sub-module provided in this embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. The components of the embodiments of the present invention described and illustrated herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0021] As Figure 1 shown, a method for classifying major depressive disorder based on optimal transport fusion of multi-stage feature pyramids is provided in this embodiment. It shows potential in classifying MDD for T1-weighted structural magnetic resonance imaging (T1 sMRI), and the prediction of this classification has the potential to be a valuable biomarker for identifying development-related and aging-related diseases.

[0022] The related steps are as follows: Input a 3D whole-brain structural magnetic resonance image, and perform image processing on the 3D whole-brain structural magnetic resonance image to obtain a 3D image set including gray matter images, white matter images, and cerebrospinal fluid images; Label all the images in the 3D image set with the major depressive disorder level, and form a data set by combining the 3D image set and the labels corresponding to the images in the 3D image set; Construct a prediction network based on an end-to-end neural network framework. The prediction network includes an image preprocessing module, a multi-scale feature extraction module, a pyramid optimal transport fusion module, and a prediction module; The image preprocessing module is used to perform image processing on the input 3D whole-brain structural magnetic resonance image to output the corresponding gray matter image, white matter image, and cerebrospinal fluid image; The multi-scale feature extraction module includes three parallel feature extraction branches. A plurality of feature extractors with different scales are connected in series on each feature extraction branch. The plurality of feature extractors with different scales are used to extract the image features of the images input into the corresponding feature extraction branch. Each feature extractor is provided with a first output end and a second output end. The first output end is used to input the extracted image features into the feature extractor of the next scale. The image features include white matter features, cerebrospinal fluid features, and gray matter volume features; The pyramid optimal transport fusion module aligns and fuses the image features of different sizes extracted by the multi-scale feature extraction module based on the feature pyramid structure to output the corresponding final image features; The prediction module makes predictions based on the input final image features to output corresponding prediction results; The prediction network is trained using a dataset to obtain a classification model for major depressive disorder classification; The 3D whole-brain structural magnetic resonance image of the subject is input into the classification model to output the classification result of the subject.

[0023] Furthermore, in this embodiment, it is based on an end-to-end neural network framework named MFPOTF (Multi-Stage Feature Pyramid Optimal Transport Fusion Network). This architecture mainly includes two types of cores: the Pyramid Optimal Transport Fusion (FPOTGF) module and the 3D ConvNeXt (feature extractor) for high-precision MDD classification.

[0024] This neural network framework is composed of three feature extraction branches, which are respectively used to process gray matter images, white matter images, and cerebrospinal fluid images.

[0025] A plurality of 3D ConvNeXt (feature extractors) are connected in series on each branch to extract features from the 3D whole-brain structural magnetic resonance image (sMRI). The first output results of the 3D ConvNeXt (feature extractors) at the ends of the three branches are all gathered in the second OTGF sub-module. Different image features are first flattened and then aligned and fused to obtain the third fused feature.

[0026] The second output ends of the 3D ConvNeXt (feature extractors) of the same scale on the three branches are gathered in the first OTGF sub-module for alignment and fusion to output the first fused feature. Then, through the TAF sub-module, the first fused features at multiple scales are adaptively calculated for predicted feature fusion to output the second fused feature. Finally, the third fused feature and the second fused feature are feature concatenated as the final image features and input into the prediction module.

[0027] As Figure 2 shown, this is the specific working principle of the OTGF sub-module provided in this embodiment, optimal transport fusion and Gaussian alignment fusion.

[0028] As Figure 3 shown, optimal transport fusion first calculates the distance matrix and solves the optimal transport scheme through the Kantorovich formula and linear programming algorithm, so that the white matter features and cerebrospinal fluid features can adaptively generate new features and be added and fused with the gray matter volume features to enhance the representational ability of the gray matter and play a key role in the subsequent classification stage.

[0029] Among them, the expression of the flattening process is as follows: ; ; ; Among them, represents white matter features, represents cerebrospinal fluid features, represents gray matter volume features, that is, the input feature tensor , , is flattened into a two-dimensional matrix. In this system, always represents the gray matter volume features at different iteration stages, represents the white matter volume features at different iteration stages, represents the cerebrospinal fluid volume features at different iteration stages. Originally, the shape of each input tensor is , where B represents the batch size, C represents the number of channels, and D, H, and W represent depth, height, and width respectively. After flattening, the shape of the feature matrix becomes , where .

[0030] Then, use the method of calculating the distance matrix in the following form: ; Calculate the distance matrices and , where represents the pairwise distance between the flattened features and , represents and The distance between. Each element in the distance matrix represents the squared Euclidean distance between the i-th sample in the feature a and the j-th sample in . The squared Euclidean distance is defined as: ; Among them, a and b are two feature vectors, ak and bk are the k -th elements of these two vectors respectively, N is the dimension of the feature vector.

[0031] Subsequently, adopt the Kantorovich formula to obtain the optimal transport plan through the classical linear programming algorithm: ; In the above situation, and is the distance matrix, which respectively represents the features and , as well as the features and The goal is to find a transport matrix such that the sum of the products of each element in the transport matrix and the corresponding element in the distance matrix is minimized, thus ensuring the lowest total cost during the transportation process.

[0032] In addition, during the process of optimizing the transport scheme, there are also constraint conditions, that is, the transport matrix must satisfy specific constraint conditions, that is, the sum of the rows and columns of the matrix must be 1. This means that the total mass of the transport remains unchanged, and the total mass received at each target point is equal to the total mass of the initial point.

[0033] Then, according to the optimized transport scheme, the white matter features and cerebrospinal fluid features are mapped to obtain the reconstructed white matter features and reconstructed cerebrospinal fluid features, and the reconstructed white matter features, reconstructed cerebrospinal fluid features and gray matter volume features are weighted and summed to output the first sub-feature.

[0034] Among them, through mapping, new white matter and cerebrospinal fluid features are obtained and added and fused with the gray matter features:

[0035] Using the obtained transport matrix and , the original features and are mapped to generate new feature representations and . Finally, by weighted summing the original features and the mapped features, the comprehensively fused feature representation is obtained, integrating the information of the three input modalities.

[0036] As Figure 4 shown, for Gaussian alignment fusion, first the features are concatenated, then aligned to the same feature space, the global mean and standard deviation of each input feature are calculated, and each type of feature is standardized with the same Gaussian distribution to eliminate the distribution differences between different feature sources, facilitating subsequent fusion operations. Finally, the three features are added for fusion.

[0037] Among them, the global mean is as follows:

[0038] In the above situation, represents the global mean, Brepresents the batch size, 3N represents the dimension of feature concatenation, indicates the i th sample j th eigenvalue in the concatenated feature matrix.

[0039] The calculation method of the standard deviation is as follows:

[0040] In the above situation, for each eigenvalue of each sample , calculate the squared deviation from the mean: Then sum up all the squared deviations from the mean, divide by the total number of features B×3N to obtain the mean square error, and finally take the square root to get the standard deviation.

[0041] Standardize each feature according to the calculated global mean and standard deviation, and perform weighted summation on the three standardized features to output the second sub-feature, which is expressed as follows: ; ; where represents the white matter feature, represents the cerebrospinal fluid feature, represents the gray matter volume feature, represents the global mean, represents the standard deviation, represents the second sub-feature.

[0042] In the network framework of this embodiment, the pyramid optimal transport fusion module includes two parts. The first part includes the first OTGF sub-module and the TAF sub-module. The first OTGF sub-module is used to collect the image features output by the second output ends of the same-scale feature extractors in different feature extraction branches, and perform alignment and fusion on the tail image features at the same scale to output the first fusion feature at the corresponding scale. The TAF sub-module is used to perform adaptive calculation and prediction feature fusion on the first features at all scales to output the second fusion feature. Among them, the image features output by the second output end are divided into two paths. One path is input to the first OTGF sub-module after being added to the image features of the previous stage's second output end through upsampling. If the second output end is the first feature extractor in the feature extraction branch, upsampling is not required; The second part includes the second OTGF sub-module. The second OTGF sub-module is used to collect the image features output by the first output ends of the feature extractors at the very ends of all feature extraction branches, and perform alignment and fusion to output the third fusion feature; Finally, the second fusion feature and the third fusion feature are concatenated as the final image feature and input to the prediction module.

[0043] In the above way, multi-scale and multi-level different method fusion features can be obtained. In this embodiment, its expression is as follows: \(f^i\in\mathbb{R}^{B_i\times D_i\times W_i\times H_i\times C_i}\), \(i\in\{1,2,3,4\}\). Where \(B_i\) represents the batch size, \((D_i\times W_i\times H_i)\) represents three spatial dimensions, and \(C_i\) represents the number of channels of the features. First, pass through a 1×1×1 convolutional layer to flatten the number of channels of these pyramid features, that is, process pyramid features at different levels into five-dimensional vectors with the same number of channels. The remaining features are divided into two paths. One path is upsampled and added to the features of the previous stage. If it is the features of the first stage, this path is not required. The other path flattens the five features behind the flattened features, and each pyramid feature is finally represented as a 2D tensor. Next, these 2D tensors are fused through the optimal transport fusion and Gaussian alignment fusion provided by the above embodiment.

[0044] As Figure 5 shown, the TAF sub-module uses the transformer adaptive fusion method to fuse features for multi-scale and multi-level different features, and constructs Q, K, V matrices as the second fusion features after feature stacking. Its expression is as follows: ; Among them, represents the predicted image features corresponding to m scales, B represents the batch size, represents the feature length.

[0045] In addition, during the process of fusing features, it is also necessary to calculate the correlation matrix and the normalized correlation matrix. As follows: ; ; Finally, the second fusion feature and the third fusion feature are subjected to weighted feature representation and feature aggregation. As follows: ; ; Subsequently, the final MDD classification result is generated through a multi-layer perceptron (MLP).

[0046] Regarding the data set source for the final model training, the following criteria are adopted for construction: Ages between 18 and 65 years old. Years of education exceed 5 years. Meet the diagnostic criteria for MDD in the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV). At the time of scanning, the total score of the patient on the 17-item Hamilton Depression Scale (HAMD) is 8 or above.

[0047] This embodiment also provides a severe depression classification system for implementing the steps of the severe depression classification method based on the optimal transport fusion of the multi-stage feature pyramid provided in the above embodiment.

[0048] In addition, the terms "upper", "lower", "inner", "outer", "front", and "rear" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance. Unless otherwise specifically stated, the relative steps, numerical expressions, and numerical values of the components and steps set forth in these embodiments do not limit the scope of the present invention.

[0049] Certainly, the above are only specific embodiments of the present invention and are not intended to limit the scope of the present invention. Any equivalent changes or modifications made according to the structure, features, and principles described in the scope of the patent application of the present invention should be included in the scope of the patent application of the present invention.

[0050] Finally, it should be noted that the above embodiments are only specific embodiments of the present invention, used to illustrate the technical solutions of the present invention, rather than limiting them. The protection scope of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: any person skilled in the art within the technical scope disclosed by the present invention can still modify the technical solutions described in the foregoing embodiments, or can easily conceive of changes, or equivalently replace some of the technical features; and these modifications, changes, or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A method for classifying major depressive disorder based on the optimal transport fusion of multi-stage feature pyramids, characterized in that It includes the following steps: Input a 3D whole-brain structural magnetic resonance image, and perform image processing on the 3D whole-brain structural magnetic resonance image to obtain a 3D image set including a gray matter image, a white matter image, and a cerebrospinal fluid image; Label all the images in the 3D image set with the severity level of major depressive disorder, and form a data set by combining the 3D image set and the labels corresponding to the images in the 3D image set; Construct a prediction network based on an end-to-end neural network framework, where the prediction network includes an image preprocessing module, a multi-scale feature extraction module, a pyramid optimal transport fusion module, and a prediction module; The image preprocessing module is used to perform image processing on the input 3D whole-brain structural magnetic resonance image to output the corresponding gray matter image, white matter image, and cerebrospinal fluid image; The multi-scale feature extraction module includes three parallel feature extraction branches, and a plurality of feature extractors with different scales are connected in series on each feature extraction branch. The plurality of feature extractors with different scales are used to extract the image features of the image input to the feature extraction branch where they are located. Each feature extractor is provided with a first output end and a second output end. The first output end is used to input the extracted image features into the feature extractor of the next scale. The image features include white matter features, cerebrospinal fluid features, and gray matter volume features; The pyramid optimal transport fusion module aligns and fuses the image features of different sizes extracted by the multi-scale feature extraction module based on a feature pyramid structure to output the corresponding final image features; The prediction module makes a prediction according to the input final image features to output the corresponding prediction results; Use the data set to train the prediction network to obtain a classification model for major depressive disorder classification; Input the 3D whole-brain structural magnetic resonance image of the subject into the classification model to output the classification result of the subject.

2. The method for classifying major depressive disorder based on optimal transport fusion of multi-stage feature pyramids according to claim 1, wherein The pyramid optimal transport fusion module includes two parts. The first part includes a first OTGF sub-module and a TAF sub-module. The first OTGF sub-module is used to collect the image features output by the second output ends of the feature extractors of the same scale in different feature extraction branches, and align and fuse the tail image features at the same scale to output the first fusion features at the corresponding scale. The TAF sub-module is used to perform adaptive calculation and prediction feature fusion on the first features at all scales to output the second fusion features. Among them, the image features output by the second output end are divided into two paths. One path is input to the first OTGF sub-module after being upsampled and added to the image features of the second output end of the previous stage. If the second output end is the first feature extractor in the feature extraction branch, upsampling is not required; The second part includes a second OTGF sub-module, which is used to collect the image features output by the first output ends of the feature extractors at the very ends of all feature extraction branches, and perform alignment and fusion to output the third fusion features; Finally, the second fusion features and the third fusion features are subjected to feature splicing as the final image features and input into the prediction module.

3. The method for classifying major depressive disorder based on optimal transport fusion of multi-stage feature pyramids according to claim 1 or 2, wherein The alignment and fusion include optimal transport fusion and Gaussian alignment fusion, and the process is as follows: Flatten the input white matter features, cerebrospinal fluid features, and gray matter volume features to construct corresponding feature matrices; The optimal transport fusion: Select the gray matter volume feature as the starting feature, and use the feature matrix to calculate the distance matrix between the starting feature and the other features respectively; Construct a corresponding transport matrix equation using the Kantorovich formula and the distance matrix between features, and solve the transport matrix equation through a classic linear programming algorithm to obtain the optimal transport matrix; Map the white matter features and cerebrospinal fluid features through the optimal transport matrix to obtain reconstructed white matter features and reconstructed cerebrospinal fluid features; Perform weighted summation on the reconstructed white matter features, reconstructed cerebrospinal fluid features, and gray matter volume features to output the first sub-feature; The Gaussian alignment fusion: Concatenate in the dimension where the gray matter volume feature is located based on the constructed feature matrix to obtain a concatenated feature matrix; Calculate the corresponding global mean and standard deviation based on the concatenated feature matrix; Standardize each feature according to the calculated global mean and standard deviation, and perform weighted summation on the three standardized features to output the second sub-feature; Fuse the first sub-feature and the second sub-feature to output the corresponding fused feature.

4. The method for classifying major depressive disorder based on optimal transport fusion of multi-stage feature pyramids according to claim 3, wherein During the solution process of the transport matrix equation, it is required that the rows and columns of the transport matrix are equal to 1.

5. The method for classifying major depressive disorder based on optimal transport fusion of multi-stage feature pyramids according to claim 3, wherein, The distance matrix is calculated using the squared Euclidean distance, and its expression is as follows: ； Among them, a and b are two eigenvectors, ak and bk are respectively the k -th elements of these two vectors, N is the dimension of the eigenvector.

6. The method for classifying major depressive disorder based on optimal transport fusion of multi-stage feature pyramids according to claim 3, wherein The flattening process is based on the tensor expression of the input features to perform flattening to obtain the corresponding feature matrix , where .

7. The method for classifying major depressive disorder based on optimal transport fusion of multi-stage feature pyramids according to claim 3, wherein, The expression of the first sub-feature is as follows: ； Among them, represents white matter features, represents cerebrospinal fluid features, represents gray matter volume features, represents the optimal transport matrix between gray matter volume features and white matter features, represents the optimal transport matrix between gray matter volume features and cerebrospinal fluid features, represents the first sub-feature.

8. The method for classifying major depressive disorder based on optimal transport fusion of multi-stage feature pyramids according to claim 3, wherein, The expression of the second sub-feature is as follows: ； Among them, represents white matter features, represents cerebrospinal fluid features, represents gray matter volume features, represents the global mean, represents the standard deviation, represents the second sub-feature.

9. The method for classifying major depressive disorder based on optimal transport fusion of multi-stage feature pyramids according to claim 3, wherein The TAF sub-module constructs Q, K, V matrices as the second fusion feature through feature stacking, and its expression is as follows: ； Among them, represents the predicted image features corresponding to m scales, B represents the batch size, represents the feature length.

10. A severe depression classification system, characterized in that, Steps for implementing the severe depression classification method based on multi-stage feature pyramid optimal transport fusion according to any one of claims 1 to 9.

Images