Method for identifying autism brain map through multi-mode brain map information fusion

Abstract

The invention discloses a method for identifying an autism brain image through multi-mode brain image information fusion, is applied to the field of image identification, and aims to solve the problems that multi-mode nuclear magnetic resonance image data are not fully utilized and the model interpretability is insufficient in the prior art. The method comprises the following steps: respectively extracting brain region features from three modal data of a structural magnetic resonance image, a functional magnetic resonance image and a phenotype, then carrying out multi-modal fusion, and constructing a brain region map structure; designing a drawing convolution module and a loss function to learn how to extract brain map depth features; on the basis of brain map depth features, a multi-layer perceptron classifier is trained through ABIDE data set label data, and better performance indexes such as accuracy, sensitivity and heterogeneity can be obtained; the trained multi-mode brain image fusion identification autism model can be used for assisting autism diagnosis and identification.

Description

Technical Field

[0001] The invention belongs to the field of image recognition, and in particular relates to a brain map recognition technology. Background Art

[0002] Autism Spectrum Disorder (ASD), also known as autism, is a common mental development disorder in infants, young children and adolescents. At present, the diagnosis of autism is mainly based on the experience of the diagnostician and the measurement of autism-related scales, but the diagnosis is relatively subjective. Magnetic resonance imaging (MRI) provides objective biomarkers for autism diagnosis. Among them, structural magnetic resonance imaging (sMRI) and functional magnetic resonance imaging (fMRI) provide information about the brain from two perspectives: brain structural anatomy and functional connectivity. In recent years, deep learning has received increasing attention in the field of autism identification. Although some methods have made initial progress, there are still problems such as failure to fully utilize multimodal MRI image data and insufficient model interpretability. Summary of the invention

[0003] In order to solve the above technical problems, the present invention proposes a method for identifying autism brain maps by fusing multimodal brain map information, which can obtain better performance indicators such as accuracy, sensitivity, and heterogeneity.

[0004] The technical solution adopted by the present invention is: a method for identifying autism brain maps by fusing multimodal brain map information, comprising:

[0005] S1. Preprocessing the three modal data of structural magnetic resonance imaging, functional magnetic resonance imaging, and non-image phenotypic data of several labeled subjects respectively, and obtaining the brain region structural data, brain region functional data, and phenotypic data of each labeled subject. The brain region structural data, brain region functional data, and phenotypic data of each labeled subject are taken as a sample, thereby constructing a training sample set; the labels include autism labels and normal reference labels;

[0006] S2. Construct a recognition model, wherein the recognition model includes: an MRI sequence feature encoder, a phenotypic feature embedding module, a fusion module, a graph convolution module, and an MLP classifier;

[0007] The MRI sequence feature encoder generates a brain region connection edge feature matrix by fusing functional and structural magnetic resonance image information based on brain region structure data and brain region function data;

[0008] The phenotypic feature embedding module is used to extract phenotypic features from phenotypic data, and generate a brain region node feature matrix by fusing two modal information, namely, subject phenotype and brain region correlation;

[0009] The graph convolution module includes several layers of graph convolution networks connected in sequence, and an important node selection layer; the graph convolution module obtains significant brain regions of interest based on the input brain region connection edge feature matrix, brain region node feature matrix and corresponding sample labels;

[0010] The MLP classifier outputs a classification result according to the input significant brain regions of interest;

[0011] S3, training the recognition model based on the training sample set constructed in step S1; using labeled sample data and loss definition function, iteratively training and adjusting the learnable parameters of the recognition model to achieve the goal of convergence of loss value;

[0012] S4. Collect three modal data of structural magnetic resonance imaging, functional magnetic resonance imaging, and non-image phenotypic data of the object to be identified, and preprocess them respectively. Input the preprocessed brain region structural data, brain region functional data, and phenotypic data into the trained recognition model to obtain the classification result.

[0013] Beneficial effects of the present invention: The present invention proposes a method for identifying autism by fusion of multimodal brain map information, describing the brain map of each subject as BrainG(A,Z), where A represents the brain region connection edge feature and Z represents the brain region node feature. Brain region features are extracted from three modal data, namely structural magnetic resonance images, functional magnetic resonance images and phenotypes, respectively, and then multimodal fusion is performed to construct a brain region map structure; a graph convolution module and a loss function are designed to learn how to extract deep features of brain maps; based on the deep features of brain maps, a multi-layer perceptron (MLP) classifier is trained using ABIDE data set label data to obtain better performance indicators such as accuracy, sensitivity and heterogeneity; the trained multimodal brain map fusion model for identifying autism can be used to assist in the diagnosis and identification of autism. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a block diagram of the multimodal brain map fusion recognition model;

[0015] Figure 2 embedding process for phenotypic traits;

[0016] Figure 3 It is the graph convolution and TOP-K pooling layer;

[0017] Figure 4 It is the training process of multimodal brain map fusion recognition model;

[0018] Figure 5 Schematic diagram of important ROI-ROI connections related to autism. DETAILED DESCRIPTION

[0019] To facilitate those skilled in the art to understand the technical content of the present invention, the present invention is further explained below with reference to the accompanying drawings.

[0020] like Figure 1 As shown, a multimodal brain map information fusion recognition method of the present invention includes the following processing steps:

[0021] 1. Data preprocessing

[0022] The ABIDE public dataset MRI imaging data (sMRIs structural magnetic resonance image sequences, fMRIs functional magnetic resonance image sequences) and non-imaging phenotypic data were preprocessed; those skilled in the art should know that the ABIDE dataset includes 1009 subjects, including 493 ASD patients and 516 normal controls, and ASD and TC are the true labels corresponding to the patients.

[0023] 1.1: For MRI data, the existing medical segmentation atlas is used to divide the brain regions of interest (ROIs) of different modal MRI data. After processing, X f ∈R v×t , X s ∈R v×3 . Wherein, v represents the number of ROIs divided according to the CC200 brain atlas. In this embodiment, v is 200. t represents the length of the blood oxygen concentration level dependent (BOLD) signal time series. In this embodiment, t is 100. 3 represents the extraction of three types of volume data, namely gray matter, white matter, and cerebrospinal fluid, from sMRI. R represents a real number set. X f is the BOLD time series obtained after processing the original fMRI data, X s It is the volume data of gray matter, white matter, and cerebrospinal fluid obtained after preprocessing the original sMRI data.

[0024] 1.2: For phenotypic data, consider representative information that most subjects have, including: gender, age, full-scale IQ (FIQ), performance IQ (PIQ), verbal IQ (VIQ), handedness, eye status during the test, etc. For gender, eye status during the test, and handedness, single hot encoding is used. The rest of the information, such as age, uses the original data. FIQ, PIQ, and VIQ are standardized to an average of 100 points. After standardization, the phenotypic data can be expressed as X p ∈R 7 .

[0025] 2. Generation of brain region edge connection features

[0026] Functional connectivity of brain regions is an important component of MRI analysis, which can reveal certain functional disorders or abnormalities and also plays an important role in the performance of subsequent classification and the mining of biomarkers.

[0027] The present invention designs an MRI sequence feature encoder, which is implemented by a multi-head attention mechanism and can obtain the brain region structure data X of the subject. s ∈R v×3 , brain region function data X f ∈R v×t Extract brain region structural features X′ s ∈R v×d′ , brain region functional characteristics X′ f ∈R v×d ; Where t represents the length of the time series, in this embodiment, t is 100, v represents the number of ROIs, in this embodiment, v is 200, d′ represents the dimension of the feature, here d′ is 8;

[0028] Then, the structural and functional brain region features are fused to obtain the MRI brain region fusion feature F MRI =X′ s +X′ f ,F MRI ∈R v×d ; Further construct the brain region connection edge feature matrix A = softmax (F MRI )[softmax(F MRI )] T ,A∈R v×v By extracting edge features from brain regions, we can make full use of the complementary information of MRI multimodal data to obtain edge features A for graph classification, which reflects the dynamics and complex connection patterns of the data.

[0029] The specific process is described as follows:

[0030] 2.1: For the input T of the lth layer m (l-1) ∈R v×d′ , where: m∈{f,s} represents the functional and structural modal input data, v represents the number of ROIs (v=200), d′ represents the feature dimension of the modal data, if it is fMRI modal data, d′=100; if it is sMRI modal data, d′=3, the input Convert to self-attention matrix Q m ,K m ,V m :

[0031]

[0032] in, is a learnable weight parameter,

[0033] 2.2: Calculate single-head attention according to formula (4):

[0034]

[0035] Among them, σ(·) is the softmax function.

[0036] The self-attention score can be used as a basis for potential biomarker identification. The value vector V m The larger the weight, the greater its contribution to the classification task.

[0037] 2.3: In order to achieve richer parallel features, a multi-head attention mechanism is introduced, and the i-th tap is defined as head i , whose expression is:

[0038]

[0039] in, Represents the weight matrix to be trained for the i-th tap.

[0040] 2.4: According to formula (6), the features of each tap are concatenated and then the concatenated features are combined with an output matrix W MLP Multiply them together to get the output of multi-layer self-attention.

[0041]

[0042] Among them, W MLP It is the learnable weight matrix used by the MRI sequence feature encoder to output the results, which is obtained by continuously updating through data sample training and learning. MLP ∈R hd×d , h is the number of attention taps, and in this embodiment, the value of h is 2.

[0043] In this embodiment, the MRI sequence feature encoder includes a three-layer structure. The first layer inputs Represents X s or X f , the input of each subsequent layer is the output of the previous layer, and the output of the last layer is used as the output of the MRI sequence feature encoder.

[0044] 2.5: Multimodal MRI data is obtained after the encoder X′ f ∈R v×d and X′ s ∈R v×d Among them, X′ f and X′ s They represent fMRI features and sMRI features respectively.

[0045] X′ f and X′ s is the output of the MRI sequence feature encoder, which uses a multi-head self-attention mechanism; it is equivalent to the following processing:

[0046] X′ f =Encoder(X f )

[0047] X′ s =Encoder(X s )

[0048] 2.6: According to formula (7), the fMRI and sMRI features are fused:

[0049] F MRI =X′ s +X′ f (7)

[0050] 2.7: Using fusion feature F MRI , generate the brain region edge connection feature matrix:

[0051] A=softmax(F MRI )[softmax(F MRI )] T (8)

[0052] Among them, A∈R v×v Represents the generated brain region edge connection feature matrix, and softmax() represents the activation function.

[0053] 3. Brain region node feature generation

[0054] In order to find out the important phenotypic information features related to ASD classification, the present invention designs a phenotypic feature embedding module, such as Figure 2 As shown, Xp ∈R 7 For the phenotypic data of each subject, the phenotypic feature embedding module consists of three fully connected layers. Except for the last fully connected layer, the other two fully connected layers are followed by a nonlinear ReLU activation function layer and a Dropout layer to avoid overfitting. Figure 2 ReLu stands for linear rectification function, and its full spelling is Rectified Linear Unit; Dropout stands for random dropout, which is a professional term in this field.

[0055] First, the phenotypic data is processed by the embedding module to obtain the phenotypic characteristics, which can be expressed as

[0056] X′ p =f MLP (X p ) (9)

[0057] Among them, X′ p ∈R v×v .

[0058] Then, the brain region function data X f Perform Pearson correlation calculation to obtain the brain region correlation feature matrix Z′∈R v×v ;

[0059] Next, in order to promote the interaction of subsequent information in the brain map, the graph convolution classifier is guided to learn more features related to heterogeneity, and the phenotypic feature X′ p The brain region node feature matrix Z is obtained by fusing it with the brain region correlation feature Z′, which can be expressed as

[0060] Z=Z′+X′ p (10)

[0061] Among them, Z∈R v×v .

[0062] 4. Graph Convolution and Top-k Pooling

[0063] Design as Figure 3 Then, using 1009 subjects in the ABIDE dataset (including 493 ASD patients and 516 normal controls) as label data, the brain region connection edge feature matrix and brain region node feature matrix of different subjects were obtained by using the previous steps, and the graph convolution module was used for supervised learning. The graph convolution module neural network link weights were continuously adjusted and optimized during the training process.

[0064] During the specific training process, the 1009 subjects will be divided into multiple batches; each training session will use the data of subjects from one batch.

[0065] The brain regions are very similar to graphs, so the present invention designs and uses graph convolutional networks to learn how to extract deep features of brain maps. A k-layer graph convolutional network algorithm is used, and each layer of node representation is updated through message passing and structural reconstruction. k is the node feature updated at the kth layer, and the update strategy is

[0066] h k =ReLu(Ah k-1 W k ) (11)

[0067] Among them, A is the brain region edge feature matrix, W k is the learnable weight parameter of the GCN (Graph Convolution Network) network, h 0 =Z.

[0068] To avoid numerical instability, a BatchNorm1D operation is appended after each graph convolution, denoted as

[0069] h k =BatchNorm1D(ReLu(Ah k-1 W k )) (12)

[0070] In this paper, a Top-k pooling important node selection layer based on self-attention mechanism is designed and set at the end of the graph convolutional network, such as Figure 3 shown.

[0071] Figure 3 In which Conv means convolution, BatchNorm means batch normalization, Leaky Relu means Rectified Linear Unit (ReLU), Top-K Pooling means Top-K pooling, Node Feature means node feature, Top-K Loss means Top-K loss, Attention score means attention score, Self-Attention means self-attention, sigmoid means S-type activation function, and Top-K reserved nodes means Top-K reserved nodes.

[0072] Through graph convolution learning, the obtained features are flattened and input into the MLP classifier before all node features may have been saturated after multiple message diffusions. By sorting the importance scores of all nodes, the top k nodes can be retained and other nodes can be deleted. Using this Top-k pooling processing method, the most indicative nodes (significant ROIs) can be automatically retained, and important brain areas related to the onset of ASD can be counted based on the CC200 brain atlas. At the same time, since most of the nodes are deleted, the dimension can also be reduced to avoid the dimensionality curse problem. In this embodiment, the value of k is 10.

[0073] The node features after graph convolutional network processing are represented as Where v is the number of ROIs, d output is the dimension of the output feature of the last convolutional layer. In order to decide which nodes to retain, the self-attention mechanism is introduced to learn the importance of the nodes. Q , W K are two weight matrices to be learned, and the expression for calculating the node importance score is

[0074]

[0075] Where s∈R v Represents the importance score of each brain region.

[0076] Obviously, a higher importance score indicates a higher probability that the relevant node will be retained.

[0077] To improve interpretability, we define

[0078]

[0079] This step can use penalty conditions to adjust the ability to calculate importance scores.

[0080] By scoring the selected nodes as 1 and the deleted nodes as 0, the graph can be sparse and interpretable significant brain region nodes can be obtained.

[0081] 5. Multilayer Perceptron

[0082] Multi-Layer Perception (MLP) was used as a classifier for ASD (Autism Spectrum Disorder) and TC (Typical Controls) classification recognition.

[0083] 6. Loss Function

[0084] Design the loss function of the model:

[0085] L=Lcl +αL sim +βL intra +γL inter +ξL TPK (15)

[0086] Among them, α, β, γ, ξ are hyperparameters.

[0087] Binary cross entropy loss function L cl Defined as

[0088]

[0089] Among them, y m,c is the true label, is the predicted value of the model. b represents the number of subjects in the current batch, and c represents the number of classes. The instance specifically refers to the top k nodes with the highest scores found through training.

[0090] Similarity loss function L sim It is used to measure the difference between the shared representations of various modalities, which is defined as

[0091]

[0092] in represents the Frobenius norm.

[0093] The present invention regards the subjects with the same label in the current batch as a group, and each batch will correspond to two groups, one is the group of subjects with autism label, and the other is the group of subjects with normal reference label. Since individuals in the same group have consistent patterns in functional connectivity, the intra-group loss is considered, which is defined as

[0094]

[0095] Among them, ‖·‖2 represents the 2-norm, c∈C is a class, where the class is the result obtained by the classifier, and the set S c ={i|Y i,c =1} contains all samples of subjects classified as c, A i Represents the brain region connection edge feature matrix A, Y generated by the sample of the i-th subject i,c =1 indicates that the MLP classifier output is 1. In this embodiment, the classifier output of 1 indicates autism classification, and the classifier output of 0 indicates normal reference classification. c and is the mean and variance of the A matrix generated by the data corresponding to the c-th subject in a group of the current batch. In the same group, this loss function minimizes the difference by reducing the variance of the A matrix, thereby further ensuring a consistent pattern.

[0096] Since there are huge differences in brain functional networks among different people, such as brain volume and gender, we consider the inter-group loss, which is defined as

[0097]

[0098] Among them, μ a and μ b It represents the mean value of the brain map depth features corresponding to the current batch group a and group b. Group a and group b correspond to the above-mentioned group of subjects with autism label and the group of subjects with normal reference label, respectively.

[0099] In the Top-k pooling layer based on the self-attention mechanism, the present invention expects that the selected nodes and the unselected nodes have obviously opposite scores. Ideally, the span is at most within [0,1], and the scores of the selected nodes and the unselected nodes should be 1 and 0 respectively. Therefore, the mth instance The loss function is defined as

[0100]

[0101] Where b is the total number of top k nodes with the highest scores found by training, and s m Represents the value of ROI in the brain region node feature matrix, Indicates m The normalized value, are scores arranged in descending order, with a value range of [0,1]. Obviously, L TPK The first k items and the rest of the items are constrained to be 1 and 0 respectively, making the two groups divergent.

[0102] The training optimization goal is to minimize the loss function L of equation (15). In this embodiment, the training stop condition is Epoch=500, that is, the training ends after 500 iterations are completed.

[0103] During the training process, the parameters that need to be adjusted in the entire recognition model are the weight matrices, including:

[0104] 1) Weight matrices to be trained for multiple taps in the MRI sequence feature encoder Fully connected layer weight matrix W MLP ;

[0105] 2) The weight matrices of the three fully connected layers in the phenotypic feature embedding module;

[0106] 3) Learnable weight parameter W of graph convolutional neural network k ;

[0107] 4) Two weight matrices W to be learned in the Top-k pooling operation Q, W K ;

[0108] 5) The fully connected weight matrix of the Multi-Layer Perception (MLP).

[0109] The specific adjustment method is based on minimizing the loss function according to the gradient descent algorithm, that is, continuously adjusting the weight matrix to reduce the error, thereby optimizing the performance of the recognition model.

[0110] The technical effects of the present invention are described below in conjunction with specific data:

[0111] The hyperparameter settings used in the model and method proposed in the present invention are shown in Table 1.

[0112] Table 1 Hyperparameter settings

[0113]

[0114] In Table 1, Epoch means running the model to complete the complete process of forward propagation and back propagation for all data; Batch size means dividing the entire sample data into multiple batches, and the number of samples in each batch is called batchsize; Learning rate means learning rate, which determines the speed and direction of updating weight parameters of the model during training; Dropout rate means the probability of randomly discarding a part of neuron outputs during training; Fold means the number of folds (number of subsets), where Fold = 5, which means dividing the training set into 5 subsets and using 5-fold cross-validation method during training.

[0115] The data set is divided into 70% as a training set, 10% as a validation set, and 20% as a test set. The proposed method is compared with 6 state-of-the-art methods, and the performance index comparison results are shown in Table 2. The comparison results show that the proposed method achieves the highest classification accuracy of 79.4%, and at the same time, considering the heterogeneity between subjects, it also achieves better balance effects in indicators such as sensitivity (80.3%) and specificity (78.4%).

[0116] Table 2 Performance comparison of different methods

[0117]

[0118] In Table 2, Accuracy represents accuracy, Sensitivity represents sensitivity accuracy, Specificity represents specificity accuracy, and AUC (Area Under Curve) represents the area under the receiver operating characteristic curve.

[0119] The loss of the multimodal brain map fusion recognition model training process can reach a convergence state, such as Figure 4 shown. Figure 4 Train Loss is the training loss and Loss is the loss value.

[0120] The comparison of the effects of different modal inputs on the classification of the graph convolution classifier is shown in Table 3. The results show that multi-modal input can achieve the highest classification accuracy.

[0121] Table 3 Comparison of model classification effects of different modal inputs

[0122]

[0123] like Figure 5 As shown, through experiments, the present invention obtains the 10 most important ROI-ROI connections in the classification, and finds that brain regions related to the frontal lobe, temporal lobe, and hippocampus are significantly associated with autism. Figure 5 In the figure, Left Lateral OccipitalCortex indicates left lateral occipital cortex, Right Frontal Pole indicates right frontal pole, Right Insular Cortex indicates right insular cortex, Left Temporal Pole indicates left temporal pole, Right Postcentral Gyrus indicates right postcentral gyrus, Right Superior Frontal Gyrus indicates right superior frontal gyrus, Right Hippocampus indicates right hippocampus, Left Middle Temporal Gyrus indicates left middle temporal gyrus, Right Frontal Pole indicates right frontal pole, Right Superior Temporal Gyrus indicates right superior temporal gyrus, Right Frontal Pole indicates right frontal pole, None indicates None, Right Supramarginal Gyrus indicates right supramarginal gyrus, Left Temporal Pole indicates left temporal pole, Right Superior Temporal Gyrus indicates right superior temporal gyrus, Left Frontal Pole indicates left frontal pole, Right Putamen indicates right nucleus, Right Parahippocampal Gyrus indicates right parahippocampal gyrus, Left ParahippocampalGyrus indicates the left parahippocampal gyrus.

[0124] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the scope of protection of the present invention is not limited to such specific statements and embodiments. For those skilled in the art, the present invention may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A method for identifying autism brain maps by fusing multimodal brain map information, characterized in that: include: S1. Preprocess the three modal data of structural magnetic resonance imaging, functional magnetic resonance imaging, and non-image phenotypic data of several labeled subjects respectively, and obtain the brain region structural data, brain region functional data, and phenotypic data of each labeled subject. The brain region structural data, brain region functional data, and phenotypic data of each labeled subject are taken as a sample, thereby constructing a training sample set; The labels include autism labels and normal reference labels; S2. Construct a recognition model, wherein the recognition model includes: an MRI sequence feature encoder, a phenotypic feature embedding module, a fusion module, a graph convolution module, and an MLP classifier; The MRI sequence feature encoder generates a brain region connection edge feature matrix by fusing functional and structural magnetic resonance image information based on brain region structure data and brain region function data; The phenotypic feature embedding module is used to extract phenotypic features from phenotypic data, and generate a brain region node feature matrix by fusing two modal information, namely, subject phenotype and brain region correlation; The graph convolution module includes several layers of graph convolution networks connected in sequence, and an important node selection layer; the graph convolution module obtains significant brain regions of interest based on the input brain region connection edge feature matrix, brain region node feature matrix and corresponding sample labels; The MLP classifier outputs a classification result according to the input significant brain regions of interest; S3, training the recognition model based on the training sample set constructed in step S1; using labeled sample data and loss definition function, iteratively training and adjusting the learnable parameters of the recognition model to achieve the goal of convergence of loss value; S4. Collect three modal data of structural magnetic resonance imaging, functional magnetic resonance imaging, and non-image phenotypic data of the object to be identified, and preprocess them respectively. Input the preprocessed brain region structural data, brain region functional data, and phenotypic data into the trained recognition model to obtain the classification result.

2. The method for identifying autism brain maps by fusion of multimodal brain map information according to claim 1, characterized in that: The MRI sequence feature encoder is implemented using a multi-head attention mechanism.

3. The method for identifying autism brain maps by fusion of multimodal brain map information according to claim 2, characterized in that: The process of constructing the brain region connection edge feature matrix is: The brain region structure data of the input MRI sequence feature encoder is recorded as X s , brain region function data is X f ; The brain region structural features output by the MRI sequence feature encoder are recorded as X′ s , the functional characteristics of the brain region are recorded as X′ f ; X′ f =Encoder(X f ) X′ s =Encoder(X s ) in, It is used to represent the input of the lth layer of the MRI sequence feature encoder. The input of the first layer of the MRI sequence feature encoder is expressed as Refers to X s or X f , CONCAT represents the concatenation operation; head1,head2,…,head h Represents each tap in the multi-head attention mechanism; W MLP represents the learnable weight matrix used by the MRI sequence feature encoder to output the results; head i represents the i-th tap, represents the weight matrix to be trained for the i-th tap; X′ s With X′ f Fusion is performed to obtain the fusion feature F MRI ; Using the fusion feature F MRI , generate the brain region edge connection feature matrix: A=softmax(F MRI )[softmax(F MRI )] T Among them, softmax() represents the activation function and the superscript T represents transposition.

4. The method for identifying autism brain maps by fusion of multimodal brain map information according to claim 3, characterized in that: The phenotypic feature embedding module includes three fully connected layers. Except for the last fully connected layer, the other two fully connected layers are followed by a nonlinear ReLU activation function layer and a Dropout layer.

5. The method for identifying autism brain maps by fusion of multimodal brain map information according to claim 4, characterized in that: The process of extracting phenotypic features is: Input non-image phenotype data X p , after being processed by the phenotypic feature embedding module, the phenotypic feature X′ is obtained p ; Brain area function data f Perform Pearson correlation calculation to obtain the brain region correlation feature matrix Z′; Fusion phenotypic feature X′ p And the brain region correlation feature matrix Z′, the brain region node feature matrix Z is obtained.

6. The method for identifying autism brain maps by fusion of multimodal brain map information according to claim 5, characterized in that: The important node selection layer is implemented based on Top-k pooling of self-attention mechanism.

7. The method for identifying autism brain maps by fusion of multimodal brain map information according to claim 6, characterized in that: The loss function during training is: L=L cl +αL sim +βL intra +γL inter +ξL TPK Among them, α, β, γ, ξ are hyperparameters; y m,c is the true label of the mth subject of category c, is the predicted value of the recognition model for the mth subject of category c, b represents the number of subjects in the current training batch, and c represents a category output by the MLP classifier; represents the Frobenius norm; ||·||2 represents the 2-norm, c∈C, C includes the different classifications output by the MLP classifier, and the set S c Contains samples of all subjects with label c, A i represents the brain region connection edge feature matrix A generated by the sample of the i-th subject, μ c and is the mean and variance of the brain region connection edge feature matrix A generated by the samples corresponding to the subjects in the current batch grouping; μ a represents the mean of the brain regions of interest corresponding to the group of subjects with autism as the true label in the current batch, μ b Indicates the mean of the brain regions of interest corresponding to the group of subjects whose true labels in the current batch are normal references; represents the attention score of the i-th brain region of interest corresponding to the m-th subject selected in the current batch, It represents the attention score of the i+kth brain region of interest corresponding to the mth subject who was not selected in the current batch, and it represents the total number of brain regions of interest for each subject.

Images