Extremely simplified Transform method for brain network learning and classification

Abstract

The invention discloses a very simplified Transform method for brain network learning and classification, which comprises the following steps of: (1) carrying out the following preprocessing on resting state functional magnetic resonance imaging signal data acquired by functional magnetic resonance equipment by using a DPARSF toolbox: (2) carrying out the following preprocessing on the resting state functional magnetic resonance imaging signal data acquired by the functional magnetic resonance equipment by using a DPARSF toolbox; and (3) based on the extracted time sequence of each brain region, inputting a Miniformer model, carrying out a learning process of the brain function network, and then entering a classification process of the brain function network. In addition, two variants of Miniform are designed, regular term constraints of sparsity and time smoothness are introduced respectively, and it is verified that the proposed architecture can flexibly integrate prior information. According to the method, the number of parameters needing to be learned is reduced, training based on limited data is more effective, and the interpretability of model parameters and a final result is enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of biomedical information processing, and in particular to a simplified Transformer method for brain network learning and classification. Background Art

[0002] As a method of simulating neural pathways at a macroscopic scale, brain functional networks play an increasingly important role in the early identification of neurological or psychiatric abnormalities such as autism spectrum disorder, Alzheimer's disease, and major depressive disorder. In mathematics, a functional brain network can be described by a graph, in which the nodes represent brain regions of interest and the edges reflect the relationship with the functional magnetic resonance imaging signals of the corresponding brain regions. Unlike traditional graphs such as molecular formulas that are unknown or clearly defined, functional brain networks do not have an accurate real definition and generally need to be estimated based on functional magnetic resonance imaging signal data. High-quality functional brain network estimation is crucial for downstream tasks.

[0003] In the past few decades, many methods have been proposed to estimate functional brain networks. Classic methods include Pearson correlation, sparse representation, and mutual information. Pearson correlation is the first and simplest method to estimate functional brain networks. Although Pearson correlation is popular, it can only calculate the complete correlation between brain regions, so it is easy to cause spurious connections in the estimated functional brain network. In contrast, sparse representation, as a method to establish partial correlation models, can obtain more reliable connections by regressing the confounding effects of irrelevant brain regions. Unlike Pearson correlation and sparse representation, which can only capture linear relationships, mutual information provides a way to model complex nonlinear interactions between different brain regions. In addition, there are many other schemes, such as high-order correlation, dynamic causal models, and Granger causality, which can be used to estimate functional brain networks.

[0004] According to the obtained functional brain network, the task of diagnosing brain diseases can be completed. Traditionally, features such as node degree, local clustering coefficient or edge weight are first extracted from the functional brain networks corresponding to different subjects. Then, these features are selected (if necessary) and input into a vector classifier (such as a support vector machine) to identify subjects with brain diseases from a normal control group. In addition, considering that the functional brain network is a graph, graph neural networks can naturally be used to classify functional brain networks. Unlike the manual feature extraction method in traditional vector schemes, graph neural network-based methods combine feature extraction with classification tasks in one framework. Although vector-based schemes and graph neural network-based methods have been successfully applied in brain disease diagnosis, they both require pre-estimation of functional brain networks during the learning process. This means that the learning of functional brain networks and downstream tasks are divided into different steps, making it difficult to achieve the best overall performance.

[0005] In order to overcome the limitations of traditional step-by-step learning methods, researchers have recently explored the joint learning of functional brain networks and downstream tasks. For example, Kazi et al. proposed combining differentiable graph modules with graph convolutional networks for the diagnosis of Alzheimer's disease. Kazi, Anees, et al. "Differentiable graph module (dgm) for graph convolutional networks." IEEE Transactions on Pattern Analysis and Machine Intelligence 45.2 (2022): 1606-1617. Jia et al. studied a method for adaptively learning graph structures based on spatiotemporal graph convolutional networks. Jia, Ziyu, et al. "GraphSleepNet: Adaptive spatial-temporal graphconvolutional networks for sleep stage classification." IJCAI. Vol. 2021. 2020. Jang et al. designed a neural network model that uses a data-driven approach to learn graph structures. Jang S, Moon SE, Lee J S. EEG-based emotional video classification via learning connectivity structure[J]. IEEE Transactions on Affective Computing, 2021, 14(2): 1586-1597. However, the architectures involved in these works are mainly designed based on strong skills, without unified principles to follow, which affects their adaptability and flexibility in new scenarios.

[0006] In contrast, Transformer, as a unified basic framework, has shown remarkable results in many fields such as natural language processing and computer vision. More importantly, it provides a natural way to model the relationship between brain regions using the self-attention mechanism. Recently, Kan et al. proposed BrainNetTF and DRAT, which use Transformer to learn and classify functional brain networks in an end-to-end manner. Kan X, Dai W, Cui H, et al. Brain network transformer [J]. Advances in Neural Information Processing Systems, 2022, 35: 25586-25599. Kan X, Gu AAC, Cui H, et al. Dynamic brain transformer with multi-level attention for functional brain network analysis [C] / / 2023 IEEE EMBS International Conference on Biomedical and Health Informatics (BHI). IEEE, 2023: 1-4. Although these Transformer-based methods have great potential, they contain a large number of parameters and therefore require sufficient data for training, which is a huge challenge for medical scenarios. In addition, such methods usually result in poor interpretability. On the one hand, the projection matrix W involved in the self-attention module Q , W K and W V , which couples the data at different time points in the fMRI signal, which poses a challenge to the interpretation of the model parameters. On the other hand, it is difficult for traditional Transformer-based methods to determine which brain regions or connections contribute to the final result, because the self-attention matrix involved in these methods is only used as an intermediate metric for learning new representations, rather than directly used as a functional brain network for brain disease classification tasks. Summary of the invention

[0007] This paper proposes a minimalist Transformer method for brain network learning and classification, which not only reduces the number of parameters that need to be learned, making training based on limited data more effective, but also enhances the interpretability of model parameters and final results. Secondly, two variants based on Miniformer are designed, which respectively impose sparsity and temporal smoothness priors on functional magnetic resonance imaging signals, and flexibly integrate prior information.

[0008] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0009] A simplified Transformer method for brain network learning and classification, characterized by comprising the following steps:

[0010] (1) The DPARSF toolbox was used to perform the following preprocessing on the resting-state functional magnetic resonance imaging signal data collected by the functional magnetic resonance equipment: 1) remove the first p unstable time points of each subject; 2) perform head motion and time layer correction; 3) regress out the influence of ventricles, white matter, and global signals; 4) align to the standard space through the Montreal Neurological Institute template; 5) perform 0.01-0.1 Hz bandpass filtering to reduce the influence of heartbeat and breathing;

[0011] (2) After preprocessing the resting-state functional magnetic resonance imaging signal data, the brain is divided into several brain regions based on the automatic anatomical labeling template, and the time series of each brain region are extracted respectively, X = [x1, x2, …, x i ,…,x N ]∈R T×N , where x i ∈R T represents the time series of the i-th node, T represents the number of times in the time series, and N represents the number of nodes;

[0012] (3) Based on the extracted time series of each brain region, the Miniformer model is input, first undergoing the learning process of the brain functional network, and then entering the classification process of the brain functional network;

[0013] In the learning process of the brain functional network, the time series is input to the simple encoder module MiniEncoder, which is stacked into multiple layers to gradually extract and fuse the features of the time series; MiniEncoder converts the projection matrix based on Transformer into Simplified to a diagonal matrix W l , where W l The diagonal elements in The weights that correspond exactly to the time points in the fMRI signal; for the projection matrix After simplification, we get the diagonal matrix W l , we get the embedded representation Q l , K l and And through Q l and K l The dot product operation calculates the attention matrix A l ; The resulting matrix is ​​scaled by the Softmax function and then compared with V lMultiplying them together gives us the output of the self-attention mechanism, which is calculated as follows:

[0014]

[0015] SA l (Q l ,K l ,V l )=SA l (Q l ,Q l ,Q l )=softmax(A l )Q l

[0016] The residual connection and layer normalization modules are further used to solve the problem of gradient vanishing or exploding in the output of the self-attention mechanism. The feedforward network module is used to transmit and process the input data of each layer of the neural network, and then passes through the residual connection and layer normalization modules again. Finally, the data output by MiniEncoder will become the input data of the subsequent MiniEncoder, and the whole process is repeated.

[0017] The classification process of re-entering the brain functional network, after learning the functional brain network based on attention, this model directly obtains the functional brain network A L+1 As the input of the classifier; the classic multi-layer perceptron MLP is used as the classifier, and the upper triangular matrix of the functional brain network is extracted to reduce feature redundancy and computational complexity, and then the matrix is ​​flattened and classified through a two-layer fully connected network; the back propagation process uses cross entropy loss, and its formula is as follows:

[0018]

[0019] The final loss of Miniformer is defined as By setting the number of epochs for training updates and defining common indicators for measuring model performance, the classification results of patients and normal people can be output, and the diagonal projection matrix to be learned can be updated at the same time.

[0020] In order to bridge the gap between the existing models and their application scenarios, the present invention provides a simplified Transformer method for brain network learning and classification, namely Miniformer, which simplifies the three projection matrices in the Transformer into a diagonal matrix and directly uses the self-attention matrix as the input of the classification module. This improvement can meet the needs of functional brain network analysis tasks. First, it greatly reduces the number of model parameters, thereby alleviating the risk of overfitting caused by insufficient training data in many medical scenarios. Secondly, it improves the interpretability of model parameters and final results. In particular, each parameter in the model diagonal projection matrix can effectively indicate the importance of the time point in the functional magnetic resonance imaging signal data. By directly using the self-attention matrix (i.e., the functional brain network) as the input of the classification module, brain regions or connections that contribute significantly to the final classification results can be identified. Third, due to the interpretability of the model parameters, it makes the inclusion of priors or hypotheses easier and more natural.

[0021] As a further improvement to this technical solution:

[0022] The simplified Transformer method for brain network learning and classification described above, in MiniEncoder, the projection matrix The simplified diagonal matrix W l , integrate the prior or hypothesis, and design a new variant Miniformer (SM) by considering the contribution of adjacent time points in the functional magnetic resonance imaging signal to the functional brain network learning, that is, the smoothness prior or hypothesis, including the following steps:

[0023] The diagonal matrix W is constrained by the following regularization term l The diagonal elements of :

[0024]

[0025] Among them, w t Represents the weight at the tth time point, that is, the diagonal matrix W l The t-th element on the diagonal, T is the number of time points; the final loss of Miniformer (SM) is defined as Among them, λ is a hyperparameter; then use back propagation to set the number of training updates epochs, define common indicators for measuring model performance, and output the classification results of patients and normal people based on the indicator of the optimal number of updates.

[0026] The simplified Transformer method for brain network learning and classification described above, in MiniEncoder, the projection matrix The simplified diagonal matrix W l, integrate the prior or hypothesis, introduce a regularization term that can measure sparsity into the loss function described in claim 1, and use it to eliminate "messy" time points that may not be beneficial to the learning of functional brain networks, and then design a new variant Miniformer (SP), including the following steps:

[0027] The sparsity is encoded by the following regularization term:

[0028]

[0029] The final loss of Miniformer (SP) is defined as Among them, λ is a hyperparameter; then use back propagation to set the number of training updates epochs, define common indicators for measuring model performance, and output the classification results of patients and normal people based on the indicator of the optimal number of updates.

[0030] The present invention considers temporal smoothness (SM) and sparsity (SP) “prior information” to improve the proposed Miniformer model.

[0031] After adopting the above technical solution, the beneficial technical effects of the present invention are:

[0032] 1. We propose an innovative end-to-end simplified Transformer method, Miniformer, for functional brain network learning and classification. It not only reduces the number of parameters that need to be learned, making training based on limited data more effective, but also serves as a simpler end-to-end method to enhance the interpretability of model parameters and final results.

[0033] 2. Due to the interpretability of model parameters, it is easier to incorporate prior information or assumptions. Two variants of Miniformer are designed, introducing regularization constraints on sparsity and temporal smoothness respectively, verifying that the proposed architecture can flexibly integrate prior information.

[0034] 3. Directly using the attention matrix (functional brain network) as the input of the classification module helps to identify brain regions or connections that contribute significantly to the final classification results, thereby improving the interpretability of the model.

[0035] 4. Comparative experiments were conducted on public datasets to evaluate the simplified Transformer method, namely Miniformer, proposed for brain disease diagnosis tasks. It performed well in all performance evaluation indicators, with overall results higher than those of the comparative methods and more reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is an implementation flow chart of the Miniformer model of the present invention. DETAILED DESCRIPTION

[0037] The present invention will be further described below in conjunction with the accompanying drawings and specific implementation examples.

[0038] like Figure 1 As shown, a simplified Transformer method for brain network learning and classification, namely Miniformer, is shown. The specific implementation example includes the following steps:

[0039] (1) In this example, a public Autism Brain Imaging Data Exchange (ABIDE) dataset and a Depression Dataset (REST-MDD) are used. The data from the largest sites of ABIDE and REST-MDD are selected, namely New York University (NYU), Site 20 (Site20); the NYU dataset has a total of 184 subjects, including 79 autistic patients and 105 healthy controls; the REST-MDD dataset has a total of 533 subjects, including 282 depression patients and 252 healthy controls; the format of the resting-state functional magnetic resonance imaging signal data collected by the magnetic resonance device is read and converted. Except for the first 10 unstable time points of each subject; head movement and time layer correction are performed; the influence of ventricles, white matter, and global signals are regressed out; the template is aligned to the standard space through the Montreal Neurological Institute; and a 0.01-0.1HZ bandpass filter is used to reduce the influence of heartbeat and breathing;

[0040] (2) The DPARSF toolbox was used to perform the following preprocessing on the resting-state functional magnetic resonance imaging signal data collected by the functional magnetic resonance equipment: 1) remove the first p unstable time points of each subject; 2) perform head motion and time layer correction; 3) regress out the influence of ventricles, white matter, and global signals; 4) align to the standard space through the Montreal Neurological Institute template; 5) perform 0.01-0.1 Hz bandpass filtering to reduce the influence of heartbeat and breathing;

[0041] (3) After preprocessing the resting-state functional magnetic resonance imaging signal data, the brain is divided into several brain regions based on the automatic anatomical labeling template, and the average time series of each brain region is extracted, X = [x1, x2, …, x i ,…,x N ]∈R T×N , where x i ∈R T represents the time series of the i-th node, T represents the number of times in the time series, and N represents the number of nodes;

[0042] (4) Based on the extracted time series of each brain region, the Miniformer model is input, which first undergoes the learning process of the brain functional network and then enters the classification process of the brain functional network.

[0043] (5) In the learning process of the brain functional network, the time series is input to the simple encoder module MiniEncoder, which is stacked into multiple layers to gradually extract and fuse the features of the time series; MiniEncoder converts the projection matrix based on the Transformer Simplified to a diagonal matrix W l ,Right now Where l = 1, 2, ..., L is the index of the layer; D Q , D k and D V Represent the query, key and value dimensions respectively; W l The diagonal elements in The projection matrix corresponds exactly to the weight (importance) of the time point of the fMRI signal. After simplification, we get the diagonal matrix W l , and then the corresponding embedding representation Q is obtained l , K l and V l And through Q l and K l The dot product operation calculates the attention matrix A l ; The resulting matrix is ​​scaled by the Softmax function and then compared with V l Multiplying them together gives us the output of the self-attention mechanism, which is calculated as follows:

[0044]

[0045] SA l (Q l ,K l ,V l )=SA l (Q l ,Q l ,Q l )=softmax(A l )Q l

[0046] The residual connection and layer normalization modules are further used to solve the problem of gradient disappearance or explosion. The feedforward network module is used to transmit and process the input data of each layer of the neural network, and then passes through the residual connection and layer normalization modules again. Finally, the data output by MiniEncoder becomes the input data of the subsequent MiniEncoder, and the whole process is repeated.

[0047] To make the model simple and clear, this embodiment only considers single-head self-attention, and the operation of multi-head self-attention is the same. It is worth noting that in this case, the attention matrix A obtained based on the brain region l Corresponding functional brain networks. More importantly, A l At the same time, it encodes the temporal information, and each row (column) represents the correlation between brain regions. The stacked MiniEncoder can gradually extract and integrate multi-layer features, thereby effectively constructing the final functional brain network from the underlying functional brain network. In the experiment, the attention matrix A obtained after L layers of complete MiniEncoder is L+1 It is used as the final functional brain network for the classification of brain diseases.

[0048] (6) The classification process of re-entering the brain functional network is different from the traditional Transformer that inputs the encoder output into the classifier. After learning the attention-based functional brain network, the model of this embodiment directly obtains the functional brain network (A L+1 ) as the input of the classifier, which helps to identify brain regions or connections that have a significant impact on the final results and enhance interpretability. The classifier of this model uses the classic multi-layer perceptron MLP. Given that the constructed functional brain network is symmetrical, the model chooses to reduce feature redundancy and computational complexity by extracting the upper triangular matrix of the functional brain network. The matrix is ​​then flattened and classified through a two-layer fully connected network. The back-propagation process uses cross entropy loss, and its formula is as follows:

[0049]

[0050] The final loss of Miniformer is defined as By setting the number of epochs for training updates and defining common indicators for measuring model performance, the classification results of patients and normal people of the Miniformer model can be output, and the diagonal projection matrix to be learned can be updated at the same time.

[0051] The extremely simplified Transformer method for brain network learning and classification in this embodiment, i.e., Miniformer, simplifies the three projection matrices in the Transformer into a diagonal matrix, and directly uses the self-attention matrix as the input of the classification module. This improvement can meet the needs of functional brain network analysis tasks. First, it greatly reduces the number of model parameters, thereby alleviating the risk of overfitting caused by insufficient training data in many medical scenarios. Secondly, it improves the interpretability of model parameters and final results. In particular, each parameter in the model diagonal projection matrix can effectively indicate the importance of the time point in the functional magnetic resonance imaging signal data, and by directly using the self-attention matrix (i.e., the functional brain network) as the input of the classification module, the brain regions or connections that contribute significantly to the final classification results can be identified. Third, due to the interpretability of the model parameters, it makes the inclusion of priors or hypotheses easier and more natural.

[0052] (7) As a further improvement to the present technical solution, two variants, Miniformer (SM) and Miniformer (SP), are proposed:

[0053] The simplified Transformer method for brain network learning and classification described above, in MiniEncoder, the projection matrix The simplified diagonal matrix W l , integrate the prior or hypothesis, and design a new variant Miniformer (SM) by considering the contribution of adjacent time points in the functional magnetic resonance imaging signal to the functional brain network learning, that is, the smoothness prior or hypothesis, including the following steps:

[0054] The diagonal matrix W is constrained by the following regularization term l The diagonal elements of :

[0055]

[0056] Among them, w t Represents the weight at the tth time point (i.e., the projection matrix W l The tth element on the diagonal), T is the number of time points. The closer the weights of adjacent time points are, The final loss of Miniformer (SM) is defined as Among them, λ is a hyperparameter; then use back propagation to set the number of training updates epochs, define common indicators for measuring model performance, and output the classification results of patients and normal people based on the indicator of the optimal number of updates.

[0057] The simplified Transformer method for brain network learning and classification described above, in MiniEncoder, the projection matrix The simplified diagonal matrix W l , integrate the prior or hypothesis, introduce a regularization term that can measure sparsity into the loss function described in claim 1, and use it to eliminate "messy" time points that may not be beneficial to the learning of functional brain networks, and then design a new variant Miniformer (SP), including the following steps:

[0058] The sparsity is encoded by the following regularization term:

[0059]

[0060] The final loss of Miniformer (SP) is defined as Among them, λ is a hyperparameter; then use back propagation to set the number of training updates epochs, define common indicators for measuring model performance, and output the classification results of patients and normal people based on the indicator of the optimal number of updates.

[0061] Since Miniformer is flexible enough, many other assumptions or priors can be encoded through a simplified projection matrix, thereby more effectively encoding more domain information or knowledge and improving model performance. This embodiment improves on the basis of Miniformer to obtain two variants of Miniformer, namely Miniformer (SM) and Miniformer (SP). The two variants consider the "prior information" of temporal smoothness (SM) and sparsity (SP), respectively, to improve the proposed Miniformer model.

[0062] In order to verify the effectiveness of the model in this embodiment, the training method of the model is to minimize the total loss function In the experiment, a single-layer MiniEncoder, a two-layer fully connected network and a Softmax layer are used for prediction. During the training of MiniEncoder, the batch size is set to 8 and the hyperparameter λ is 0.5.

[0063] In the experiment, two classification tasks were performed: 1) autistic patients vs. normal people on the ABIDE dataset; 2) depression vs. normal people on the REST-MDD dataset. The Miniformer model and its variants were trained in an end-to-end manner and evaluated by 10-fold cross validation. Specifically, all subjects were randomly divided into 10 groups, 9 of which were used for training and the remaining one was used for testing. This process was repeated 10 times and the average was taken to measure the performance of the model. In addition, the specific performance indicators used in this study include accuracy (ACC), sensitivity (SEN), specificity (SPE), balanced accuracy (BAC), positive predictive value (PPV), negative predictive value (NPV), F1score, and area under the curve (AUC). The larger the values ​​of these indicators, the better the performance of the model. These indices are calculated as follows: ACC = (TP + TN) / (TP + FN + FP + TN), SEN = TP / (TP + FN), SPE = TN / (TN + FP), BAC = (SEN + SPE) / 2, PPV = TP / (TP + FP), NPV = TN / (TN + FN), F1score = 2 × PRE × REC / PRE + REC, where TP, TN, FP and FN represent true positive, true negative, false positive and false positive, respectively;

[0064] As shown in Table 1 and Table 2, the performance comparison between the simplified Transformer method based on Miniformer and other comparison methods on the ABIDE and REST-MDD datasets is shown respectively;

[0065] Table 1 Performance comparison of different methods on the ABIDE dataset

[0066] ABIDE AUC ACC LATE SPE BAC PPV NPV F1score PC+SVM 0.6852 0.6088 0.5825 0.6294 0.6060 0.5707 0.6554 0.5483 SR+SVM 0.6309 0.6199 0.4366 0.7824 0.6095 0.5896 0.6701 0.4632 SRiLT+SVM 0.6582 0.6197 0.5063 0.7029 0.6046 0.5830 0.6494 0.5352 GCN 0.6593 0.6196 0.4937 0.7143 0.6040 0.5652 0.6522 0.5270 GATE 0.7219 0.6685 0.6582 0.6762 0.6672 0.6047 0.7245 0.6303 Transformers 0.7379 0.6798 0.6369 0.7174 0.6771 0.6364 0.7176 0.6266 Miniforms 0.7576 0.6953 0.5644 0.7848 0.6746 0.6736 0.7009 0.6005 Miniforms(SP) 0.7612 0.7114 0.5536 0.8079 0.6807 0.7263 0.7088 0.5959 Miniforms(SM) 0.7801 0.7330 0.6102 0.8001 0.7051 0.7280 0.7269 0.6522

[0067] Table 2 Performance comparison of different methods on the REST-MDD dataset

[0068] REST-MDD AUC ACC LATE SPE BAC PPV NPV F1score PC+SVM 0.6087 0.5685 0.5889 0.5411 0.5650 0.5923 0.5380 0.5883 SR+SVM 0.5544 0.5423 0.6077 0.4655 0.5366 0.5581 0.5188 0.5796 SRiLT+SVM 0.5599 0.5463 0.6131 0.4776 0.5454 0.5682 0.5257 0.5848 GCN 0.5258 0.5253 0.5745 0.4701 0.5223 0.5492 0.4958 0.5615 GATE 0.5326 0.5403 0.5603 0.5179 0.5391 0.5663 0.5118 0.5633 Transformers 0.5620 0.5516 0.6024 0.4857 0.5441 0.5693 0.5238 0.5831 Miniforms 0.6101 0.6005 0.6469 0.5525 0.5997 0.6230 0.5829 0.6283 Miniforms(SP) 0.6378 0.6024 0.6282 0.5927 0.6105 0.6365 0.5811 0.6224 Miniforms(SM) 0.6304 0.6079 0.6753 0.5350 0.6052 0.6185 0.6013 0.6419

[0069] Combining Table 1 and Table 2, it can be seen that this embodiment has achieved the best performance in the listed indicators as a whole, especially on the ABIDE dataset, the AUC of Miniformer (SM) is 78.01%, the ACC is 73.30%, and on the REST-MDD dataset, the AUC of Miniformer (SP) is 63.78%, and the ACC of Miniformer (SM) is 60.79%. Compared with other methods, the good performance results are all obtained by the extremely simplified Transformer method proposed in the present invention.

[0070] This example verifies the effectiveness of the Miniformer and its variants proposed for the brain disease diagnosis task through experiments on the above-mentioned public datasets.

[0071] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modification, equivalent replacement and partial supplement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A simplified Transformer method for brain network learning and classification, characterized by The following steps are involved: (1) The DPARSF toolbox was used to perform the following preprocessing on the resting-state functional magnetic resonance imaging signal data collected by the functional magnetic resonance equipment: 1) remove the first p unstable time points of each subject; 2) perform head motion and time layer correction; 3) regress out the influence of ventricles, white matter, and global signals; 4) align to the standard space through the Montreal Neurological Institute template; 5) perform 0.01-0.1 Hz bandpass filtering to reduce the influence of heartbeat and breathing; (2) After preprocessing the resting-state functional magnetic resonance imaging signal data, the brain is divided into several brain regions based on the automatic anatomical labeling template, and the time series of each brain region are extracted respectively, X = [x1, x2, …, x i ,…,x N ]∈R T×N , where x i ∈R T represents the time series of the i-th node, T represents the number of times in the time series, and N represents the number of nodes; (3) Based on the extracted time series of each brain region, the Miniformer model is input, first undergoing the learning process of the brain functional network, and then entering the classification process of the brain functional network; In the learning process of the brain functional network, the time series is input into the MiniEncoder module, which is stacked into multiple layers to gradually extract and fuse the features of the time series; MiniEncoder will be based on the Transformer projection matrix Simplified to a diagonal matrix W l , where W l The diagonal elements in t=1,2…,T corresponds exactly to the weight of the time point in the functional magnetic resonance imaging signal; for the projection matrix After simplification, we get the diagonal matrix W l , we get the embedded representation Q l , K l and And through Q l and K l The dot product operation calculates the attention matrix A l ; The resulting matrix is ​​scaled by the Softmax function and then compared with V l Multiplying them together gives us the output of the self-attention mechanism, which is calculated as follows: IN l (Q l ,K l ,V l )=IN l (Q l ,Q l ,Q l )=softmax(A l )Q l The residual connection and layer normalization modules are further used to solve the problem of gradient vanishing or exploding in the output of the self-attention mechanism. The feedforward network module is used to transmit and process the input data of each layer of the neural network, and then passes through the residual connection and layer normalization modules again. Finally, the data output by MiniEncoder will become the input data of the subsequent MiniEncoder, and the whole process is repeated. The classification process of re-entering the brain functional network, after learning the functional brain network based on attention, this model directly obtains the functional brain network A L+1 As the input of the classifier; the classic multi-layer perceptron MLP is used as the classifier, and the upper triangular matrix of the functional brain network is extracted to reduce feature redundancy and computational complexity, and then the matrix is ​​flattened and classified through a two-layer fully connected network; the back propagation process uses cross entropy loss, and its formula is as follows: The final loss of Miniformer is defined as By setting the number of epochs for training updates and defining common indicators for measuring model performance, the classification results of patients and normal people can be output, and the diagonal projection matrix to be learned can be updated at the same time.

2. According to the simplified Transformer method for brain network learning and classification according to claim 1, in MiniEncoder, the projection matrix The simplified diagonal matrix W l , integrate the prior or hypothesis, and design a new variant Miniformer (SM) by considering the contribution of adjacent time points in the functional magnetic resonance imaging signal to the functional brain network learning, that is, the smoothness prior or hypothesis, including the following steps: The diagonal matrix W is constrained by the following regularization term l The diagonal elements of : Among them, w t Represents the weight at the tth time point, that is, the diagonal matrix W l The tth element on the diagonal, T is the number of time points; The final loss of Miniformer (SM) is defined as Among them, λ is a hyperparameter; then back propagation is used to set the number of training updates epochs, define common indicators for measuring model performance, and output the classification results of patients and normal people based on the indicator of the optimal number of updates.

3. According to the simplified Transformer method for brain network learning and classification according to claim 1, in MiniEncoder, the projection matrix The simplified diagonal matrix W l , integrate the prior or hypothesis, introduce a regularization term that can measure sparsity into the loss function described in claim 1, and use it to eliminate "messy" time points that may not be beneficial to the learning of functional brain networks, and then design a new variant Miniformer (SP), including the following steps: The sparsity is encoded by the following regularization term: The final loss of Miniformer (SP) is defined as Among them, λ is a hyperparameter; then back propagation is used to set the number of training updates epochs, define common indicators for measuring model performance, and output the classification results of patients and normal people based on the indicator of the optimal number of updates.

Images