A brain state classification and recognition method and system based on dynamic graph convolution.

Abstract

This invention relates to the field of biometric recognition technology, specifically to a method and apparatus for classifying and recognizing brain states based on dynamic graph convolution. The method includes acquiring fMRI data of the brain over a time period; acquiring a functional connectivity matrix; acquiring a graph convolution model; and using the graph convolution model to obtain the corresponding brain functional state by applying the functional connectivity matrix of the brain to be identified. This invention effectively utilizes temporal information to model and classify the brain's state at different points in time.

Description

Technical Field

[0001] This invention relates to the field of biometric recognition technology, specifically to a brain state classification and recognition method and system based on dynamic graph convolution. Background Technology

[0002] Blood oxygen-level dependent (BOLD) functional magnetic resonance imaging (fMRI) is one of the most widely used brain imaging techniques. fMRI utilizes the BOLD contrast enhancement principle to image brain functional activity, offering advantages such as non-invasiveness, relatively balanced spatiotemporal resolution, high repeatability, and the ability to image the entire brain. It provides crucial technical means for studying brain cognition, brain development and aging, and major neuropsychiatric disorders. The brain, as an extremely complex system involving numerous connections between neurons, presents a significant challenge in understanding its working mechanisms. Research has shown that the brain can be represented as a network where different regions are interconnected in specific patterns. Some regions are considered influential key nodes in this network, playing crucial roles in neural information processing. For example, highly connected regions in the brain network are called brain network hubs, playing a key role in facilitating information exchange between different parts of the brain. Therefore, in tasks using fMRI for brain state classification and recognition, selecting appropriate classification features and constructing a classifier becomes a critical issue.

[0003] However, due to the high spatial resolution of fMRI data, analyzing and calculating at the voxel level is too complex for classifying and identifying brain states. Existing brain region templates, such as Automated Anatomical Labeling (AAL) and Destrieux Atlas, while dividing the entire brain, still involve a large number of regions and suffer from excessive data volume. Therefore, exploring better methods for classifying and identifying brain states by utilizing influential nodes in the brain and combining them with appropriate classification models has become a worthwhile direction for research. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention primarily targets the issue that while existing technologies divide the whole brain, the large number of regions involved still results in excessively large data volumes. Instead, it provides a brain state classification and recognition method and system based on dynamic graph convolution. This method, acting as a pipeline, includes upstream feature extraction of influential nodes based on fMRI and downstream brain state classification based on dynamic graph convolutional networks.

[0005] The first objective of this invention is to provide a brain state classification and recognition method based on dynamic graph convolution, comprising:

[0006] Acquire fMRI data of the brain over a specific time period;

[0007] The fMRI data at each time point were divided into 148 regions;

[0008] The time series corresponding to the 56 nodes are extracted from 148 regions based on the 56 node region number template;

[0009] The functional connectivity matrix is ​​obtained based on the Pearson correlation between pairwise sequences in the time series of 56 nodes.

[0010] A pre-defined adjacency matrix is ​​used to represent the connectivity of 56 brain node regions, and its values ​​reflect the importance of different node regions to classification.

[0011] The adjacency matrix and functional connectivity matrix are fitted with a graph convolution based on a first-order Chebyshev polynomial to perform feature learning, thereby obtaining a graph convolution model. The input of the graph convolution model is the functional connectivity matrix, and the output is the brain functional state.

[0012] The functional connectivity matrix corresponding to the brain to be identified is used to obtain the corresponding brain functional state through a graph convolution model.

[0013] Preferably, the step of dividing the fMRI data at each time point into 148 regions includes:

[0014] The Destrieux Atlas template was used to divide the brain from fMRI data into 148 regions in the left and right hemispheres.

[0015] Preferably, the preset adjacency matrix includes:

[0016] Predefine an adjacency matrix A, where A∈R 56×56 The matrix is ​​initialized using a Gaussian distribution, and then symmetric and normalized sequentially to obtain the preset adjacency matrix.

[0017] Preferably, when performing feature learning based on fitting the graph convolution with a first-order Chebyshev polynomial, that is, using a first-order Chebyshev polynomial in the spatial domain to fit the graph convolution operation in the frequency domain, as shown in the following formula:

[0018]

[0019] T0(L)=1, T1(L)=L

[0020] In the formula, F* represents the features reconstructed from F after convolution; F represents the functional connectivity matrix; T k () denotes the k-th term of the Chebyshev polynomial, where θ k is the coefficient of the Chebyshev polynomial; L represents the predefined adjacency matrix; · represents matrix multiplication; K represents the order of the Chebyshev polynomial.

[0021] Preferably, feature learning includes:

[0022] The first round of training uses a pre-defined adjacency matrix as the initial value, and the resulting adjacency matrix... It continues to be used as the initial value of the adjacency matrix in the next round of training;

[0023] Train in successive rounds until all preset training rounds are completed;

[0024] In each training round, the current adjacency matrix is ​​used to learn graph convolutional features, and the parameters of the graph convolutional model are updated using the training data.

[0025] Preferably, the training data includes functional connectivity matrices from multiple time points acquired by brain fMRI and corresponding label data. In each round of training, this training data is used as input, wherein the functional connectivity matrices are used for feature learning of the dynamic graph convolutional model, and the label data are used for supervised learning of the model.

[0026] Preferably, the brain functional states include emotion, gambling, language, movement, relationships, social interaction, and working memory.

[0027] The second objective of this invention is to provide a brain state classification and recognition system based on dynamic graph convolution, comprising:

[0028] The data acquisition module is used to acquire fMRI data of the brain over a period of time; the fMRI data at each time point is divided into 148 regions; and the time series corresponding to the 56 nodes are extracted from the 148 regions based on the 56 node region sequence template.

[0029] The data processing module is used to obtain the functional connectivity matrix based on the Pearson correlation between pairs of sequences in the time series of 56 nodes; a preset adjacency matrix is ​​used to represent the connectivity of the 56 brain node regions, and its value reflects the importance of different node regions to the classification; the adjacency matrix and the functional connectivity matrix are fitted with a graph convolution based on a first-order Chebyshev polynomial to perform feature learning and obtain a graph convolution model, wherein the input of the graph convolution model is the functional connectivity matrix and the output is the brain functional state;

[0030] The state recognition module is used to obtain the corresponding brain functional state by using a graph convolution model to obtain the functional connection matrix of the brain to be identified.

[0031] The present invention has at least the following beneficial effects:

[0032] This invention provides a brain state classification and recognition method and system based on dynamic graph convolution. This method, as a pipeline, includes upstream feature extraction of influential nodes based on fMRI and downstream brain state classification based on Dynamic Graph Convolution Neural Networks (DGCNN). For the upstream part, the preprocessed fMRI data is first divided into brain regions using the 56 influential node region templates provided in this invention. Then, the time series of these 56 nodes are extracted, and the Pearson correlations between each pair are calculated to generate their functional connectivity matrices. For the downstream part, a predefined trainable 56*56 adjacency matrix is ​​initialized using a Gaussian distribution and subsequently used in training. Its values, to some extent, characterize the importance of these 56 influential nodes in classification. Then, effective features of the original data are extracted through graph convolution operations to distinguish different brain states, and finally, a multilayer perceptron model is used to learn the category information between features. After testing, the present invention achieved a classification effect similar to or even better than that of all nodes using fewer influential nodes, which is of certain significance for brain state recognition tasks using fMRI high spatial resolution data.

[0033] This invention provides a brain state classification and recognition method that utilizes functional connectivity features of low-dimensional fMRI data, which has practical significance. Experimental results show that the classification performance using 56*56 dimensional features not only does not decrease compared to 148*148 dimensional features, but even shows a slight improvement in some cases.

[0034] The DGCNN model used in this invention utilizes a dynamic adjacency matrix as a parameter, which can, to some extent, reflect the nodes that play a significant role in the classification task. Since the connection patterns between node regions found in each training iteration generally fall into local optima, the resulting adjacency matrices are often not entirely consistent. However, by performing statistical analysis on the adjacency matrices obtained from 30 training sets, a relatively stable ranking of node region classification contributions can still be obtained. This ranking result reflects, to some extent, the contribution of different nodes in a specific brain state classification task, providing new analytical approaches for further research. Attached Figure Description

[0035] Figure 1 This is a flowchart illustrating a brain state classification and recognition method based on dynamic graph convolution provided by the present invention.

[0036] Figure 2 This is a schematic diagram of the process for converting fMRI data signals into a functional connectivity matrix feature representation.

[0037] Figure 3 This is a schematic diagram illustrating the structure of a classification method using the DGCNN model with a functional connectivity matrix. Detailed Implementation

[0038] In order to illustrate the technical means and effects adopted by the present invention to achieve the intended purpose, the following detailed description is provided in conjunction with the embodiments.

[0039] In order to achieve concise and accurate brain state classification and recognition by utilizing the features of influential nodes in the brain, this invention proposes a brain state classification and recognition method and system based on dynamic graph convolution.

[0040] See Figure 1 As shown, a brain state classification and recognition method based on dynamic graph convolution includes:

[0041] S1. Obtain fMRI data of the brain over a period of time;

[0042] The fMRI data at each time point were divided into 148 regions;

[0043] Based on a template of 56 node region numbers, time series corresponding to the 56 nodes are extracted from 148 regions; the process of dividing the fMRI data at each time point into 148 regions includes:

[0044] The Destrieux Atlas template was used to divide the brain from fMRI data into 148 regions in the left and right hemispheres.

[0045] It should be noted that the acquired fMRI data needs to be preprocessed; see [link / reference]. Figure 2 As shown, preprocessing mainly includes basic processing steps after fMRI data acquisition, such as registration, head motion correction, and spatial normalization. Most publicly available datasets have already undergone this basic preprocessing upon provision. In this embodiment, the Destrieux Atlas template is used to divide the fMRI data into 148 regions in the left and right hemispheres.

[0046] S2. Obtain the functional connectivity matrix based on the Pearson correlation between pairs of sequences in the time series of 56 nodes;

[0047] See Figure 2As shown, the preprocessed fMRI signal is processed in three steps: ① The brain region of the fMRI data is divided into 148 regions using Destrieux Atlas; ② The time series of the corresponding nodes are extracted using the provided 56 node region indices; ③ The Pearson correlation between each pair of the extracted time series is calculated to obtain a 56*56 functional connectivity matrix.

[0048] It should be noted that, based on experience, 56 influential node region number templates were set up. Time series data of the corresponding node regions were extracted from the fMRI data, and Pearson correlations among these 56 node time series were calculated to generate a functional connectivity matrix. The formula for calculating Pearson correlations is as follows:

[0049]

[0050] In the formula, r represents the Pearson correlation coefficient, which measures the linear relationship between two variables X and Y; X i Y represents the i-th observation of the first variable; i This represents the i-th observation of the second variable; This represents the mean of the first variable; The second variable represents the mean; n represents the functional connectivity matrix F∈R for each fMRI dataset. 56×56 The 56 rows identify the 56 different influential brain nodes proposed. The 56-dimensional vector corresponding to each row is the functional connectivity of each region. The functional connectivity of each node is used as its feature input into the model for subsequent training and classification.

[0051] In a functional connectivity matrix, each row and column represents a specific nodal region of the brain. The elements in the matrix represent the strength of functional connectivity between corresponding nodal regions, typically quantified using their time-series Pearson correlation coefficients. Specifically, each element F in the matrix... ij It represents the functional connectivity strength between the i-th node region and the j-th node region, that is, it measures the similarity between the two regions in the time series of fMRI data.

[0052] S3. Preset adjacency matrix, which represents the connectivity of 56 brain node regions. Its value reflects the importance of different node regions to classification.

[0053] The preset adjacency matrix includes:

[0054] Predefine an adjacency matrix A, where A∈R 56×56 The matrix is ​​initialized using a Gaussian distribution, and then symmetric and normalized sequentially to obtain the preset adjacency matrix.

[0055] In this embodiment, an adjacency matrix A∈R is predefined. 56×56 As training parameters, they are initialized using a Gaussian distribution. This adjacency matrix A characterizes the connectivity of the 56 brain node regions to some extent; therefore, as the model trains, its final value also reflects, to some extent, the importance of different brain regions for specific classification tasks. Based on its geometric meaning, A is set as a symmetric matrix and then symmetricized.

[0056] A′=relu(A)

[0057] A sym =A′+(A′) T

[0058] The ReLU function discards all negative numbers in A. sym It is a symmetric adjacency matrix.

[0059] For A sym Normalize:

[0060] L=DA sym D

[0061] in,

[0062]

[0063] In the formula, d represents a vector containing all di, used to normalize the adjacency matrix A. sym ;

[0064] d i The degree of node i is represented by the adjacency matrix A. sym The sum of all elements related to node i, expressed as: This indicates that it reflects the degree of connectivity of node i in the graph;

[0065] ε represents a very small positive number used to prevent the denominator from being zero and to ensure the stability of normalized calculations;

[0066] i, j represent matrix A sym The row and column indices, i.e. the numbers of brain node regions, range from 1, 2, ..., 56.

[0067] Where L is the normalized adjacency matrix, and d can be derived from A sym Calculations show that D is a diagonal matrix.

[0068] Therefore, the normalized adjacency matrix L is used as the preset adjacency matrix.

[0069] S4. Fit the adjacency matrix and the functional connectivity matrix to a graph convolution based on a first-order Chebyshev polynomial to perform feature learning and obtain a graph convolution model.

[0070] The graph convolution model takes a functional connectivity matrix as input and outputs brain functional states as output.

[0071] When performing feature learning based on fitting graph convolution with a first-order Chebyshev polynomial, that is, using a first-order Chebyshev polynomial in the spatial domain to fit the graph convolution operation in the frequency domain, as shown in the following formula:

[0072]

[0073] T0(L)=1, T1(L)=L

[0074] In the formula, F * T represents the features reconstructed from F after convolution; F represents the functional connectivity matrix; T k () denotes the k-th term of the Chebyshev polynomial, where θ k is the coefficient of the Chebyshev polynomial; L represents the predefined adjacency matrix; · represents matrix multiplication; K represents the order of the Chebyshev polynomial.

[0075] Specifically, feature learning includes:

[0076] The first round of training uses a preset adjacency matrix L as the initial value, and the resulting adjacency matrix... It continues to be used as the initial value of the adjacency matrix in the next round of training;

[0077] Train in successive rounds until all preset training rounds are completed;

[0078] In each training round, the current adjacency matrix is ​​used to learn graph convolutional features, and the parameters of the graph convolutional model are updated using the training data.

[0079] In this embodiment, the iterative training and model update process includes:

[0080] In the first round of training, an initial functional connectivity matrix F is used as input. Simultaneously, an initial adjacency matrix L is pre-calculated and used as both input and parameter. During training, the graph convolutional model learns how to adjust the adjacency matrix to better capture the connectivity of brain node regions. After one round of training, an updated adjacency matrix is ​​obtained. It will continue to be used as the initial value of the adjacency matrix in the next round of training.

[0081] Repeat steps S3-S4 until all preset training rounds are completed. In each round, graph convolutional features are learned using the current adjacency matrix, and the model parameters are updated using training data from different brain states. Finally, the trained DGCNN model can be used to classify and identify brain states from new brain fMRI data.

[0082] The training data includes functional connectivity matrices from multiple time points acquired by fMRI of the brain, along with corresponding label data. In each round of training, this training data is used as input, with the functional connectivity matrices used for feature learning of the dynamic graph convolutional model and the label data used for supervised learning of the model.

[0083] S5. Obtain the corresponding brain functional state by using the graph convolution model to obtain the functional connection matrix of the brain to be identified.

[0084] See Figure 3 The diagram illustrates the structure of a classification method using the DGCNN model with a functional connectivity matrix. The obtained functional connectivity matrix is ​​used as the model input. The input data first passes through a graph convolutional layer to obtain reconstructed features. The convolutional kernel of this layer is fitted using a first-order Chebyshev polynomial constructed from the adjacency matrix L, flattening the reconstructed features into a one-dimensional vector. Finally, the data passes through two fully connected layers to obtain the classification result, which represents the brain functional state category indicated by the current input data.

[0085] Testing phase:

[0086] After processing the dataset in steps S1 and S2, the data is input into steps S3 and S4 to train the DGCNN model. Step S5 yields the final classification result. The dataset is selected from the Human Connectome Project (HCP) HCP 900 dataset, containing data on seven different brain functional states: Emotion, Gambling, Language, Motor, Relational, Social, and Working Memory (WM). These functional states encompass complex human cognitive and emotional experiences, and the data in this dataset has undergone basic preprocessing. This method has achieved excellent results in various experiments on this dataset. The accuracy for seven-class classification reaches 87.78%, while the accuracy for seven-class classification using the method but extracting 148*148 dimensional (high-dimensional) features from the complete Destriux Atlas is 84.16%. Furthermore, to verify whether this effect depends solely on the specificity of the 56 node regions provided by the method, 56 random nodes were selected from the difference between the complete template and the 56 node regions. The accuracy of the seven-class classification task using these 56 random nodes was only 61.73%, significantly lower than the two experiments mentioned above. In addition, two to six types of brain functional state data were selected from these seven types of data for binary to six-class classification comparison experiments, using the same experimental methods as the seven-class classification, all yielding similar results. This demonstrates the effectiveness of the method for classifying and recognizing human brain states, and also fully verifies the effectiveness of the 56 node region templates provided by this method in further reducing the feature dimension of fMRI data used in this classification task.

[0087] This invention proposes a brain state classification and recognition method based on dynamic graph convolution. First, by designing influential node partitioning and constructing a functional connectivity matrix, this method can extract key brain region connectivity information from raw fMRI data. This refined feature extraction strategy not only reduces computational complexity but also captures the most representative and influential nodes in the brain network, thereby improving the accuracy and stability of classification and recognition. Second, by introducing a dynamic graph convolutional network, this method can effectively utilize temporal information to model and classify the brain's state at different time points. This temporal modeling capability gives the method an advantage in capturing the temporal changes and dynamic features of brain states, contributing to more accurate classification and recognition of complex neural activity patterns. This method also introduces a trainable adjacency matrix, using graph convolution operations to extract features from the raw data. This adaptive network structure can adjust the connection weights between nodes according to specific tasks and data characteristics, thus better adapting to the needs of different brain state classification tasks. Furthermore, it can be used to further analyze the importance of corresponding brain regions for specific state classification tasks, which is of research significance. Furthermore, accurate classification and identification of brain states can provide deeper understanding for neuroscience research, provide more accurate brain state assessment and decision support for clinical medical diagnosis and treatment, and also bring new possibilities for the development of brain-computer interfaces, human-computer interaction and other fields.

[0088] This invention provides a brain state classification and recognition system based on dynamic graph convolution, comprising:

[0089] The data acquisition module is used to acquire fMRI data of the brain over a period of time; the fMRI data at each time point is divided into 148 regions; and the time series corresponding to the 56 nodes are extracted from the 148 regions based on the 56 node region sequence template.

[0090] The data processing module is used to obtain the functional connectivity matrix based on the Pearson correlation between pairs of sequences in the time series of 56 nodes; a preset adjacency matrix is ​​used to represent the connectivity of the 56 brain node regions, and its value reflects the importance of different node regions to the classification; the adjacency matrix and the functional connectivity matrix are fitted with a graph convolution based on a first-order Chebyshev polynomial to perform feature learning and obtain a graph convolution model, wherein the input of the graph convolution model is the functional connectivity matrix and the output is the brain functional state;

[0091] The state recognition module is used to obtain the corresponding brain functional state by using a graph convolution model to obtain the functional connection matrix of the brain to be identified.

[0092] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for brain state classification based on dynamic graph convolution, characterized in that, include: Acquire fMRI data of the brain over a specific time period; The fMRI data at each time point were divided into 148 regions; The time series corresponding to the 56 nodes are extracted from 148 regions based on the 56 node region number template; The functional connectivity matrix is ​​obtained based on the Pearson correlation between pairwise sequences in the time series of 56 nodes. A pre-defined adjacency matrix is ​​used to represent the connectivity of 56 brain node regions, and its values ​​reflect the importance of different node regions to classification. The adjacency matrix and functional connectivity matrix are fitted with a graph convolution based on a first-order Chebyshev polynomial to perform feature learning, thereby obtaining a graph convolution model. The input of the graph convolution model is the functional connectivity matrix, and the output is the brain functional state. The functional connectivity matrix corresponding to the brain to be identified is used to obtain the corresponding brain functional state through a graph convolution model; When performing feature learning based on fitting graph convolution with a first-order Chebyshev polynomial, that is, using a first-order Chebyshev polynomial in the spatial domain to fit the graph convolution operation in the frequency domain, as shown in the following formula: In the formula, denotes reconstructed features after convolution; denotes a functional connection matrix; denotes the first term of the Chebyshev polynomial, wherein is a Chebyshev polynomial coefficient; denotes a preset adjacency matrix; denotes matrix multiplication; denotes the order of the Chebyshev polynomial; Feature learning includes: The first round of training uses a pre-defined adjacency matrix as the initial value, and the resulting adjacency matrix... It continues to be used as the initial value of the adjacency matrix in the next round of training; Train in successive rounds until all preset training rounds are completed; In each training round, the current adjacency matrix is ​​used to learn graph convolutional features, and the parameters of the graph convolutional model are updated using the training data.

2. The brain state classification and recognition method based on dynamic graph convolution according to claim 1, characterized in that, The fMRI data at each time point is divided into 148 regions, including: The Destrieux Atlas template was used to divide the brain from fMRI data into 148 regions in the left and right hemispheres.

3. The brain state classification and recognition method based on dynamic graph convolution according to claim 1, characterized in that, The preset adjacency matrix includes: Predefine an adjacency matrix A, where A The matrix is ​​initialized using a Gaussian distribution, and then symmetric and normalized sequentially to obtain the preset adjacency matrix.

4. The brain state classification and recognition method based on dynamic graph convolution according to claim 1, characterized in that, The training data includes functional connectivity matrices from multiple time points acquired by brain fMRI and corresponding label data. In each round of training, this training data is used as input, where the functional connectivity matrices are used for feature learning of the dynamic graph convolutional model, and the label data are used for supervised learning of the model.

5. The brain state classification and recognition method based on dynamic graph convolution according to claim 1, characterized in that, The brain functional states mentioned include emotion, gambling, language, movement, relationships, social interaction, and working memory.

6. A system for the brain state classification and recognition method based on dynamic graph convolution as described in claim 1, characterized in that, include: The data acquisition module is used to acquire fMRI data of the brain over a period of time. The fMRI data at each time point were divided into 148 regions; the time series corresponding to the 56 nodes were extracted from the 148 regions based on the 56 node region number template. The data processing module is used to obtain the functional connectivity matrix based on the Pearson correlation between pairs of sequences in the time series of 56 nodes; A pre-defined adjacency matrix is ​​used to represent the connectivity of 56 brain node regions, and its value reflects the importance of different node regions to classification. The adjacency matrix and the functional connectivity matrix are fitted with a graph convolution based on a first-order Chebyshev polynomial to perform feature learning and obtain a graph convolution model. The input of the graph convolution model is the functional connectivity matrix, and the output is the brain functional state. The state recognition module is used to obtain the corresponding brain functional state by using a graph convolution model to obtain the functional connection matrix of the brain to be identified.

Images