ADHD graph convolution model construction method based on fMRI spatial-temporal characteristics

Abstract

The invention belongs to the technical field of deep learning and brain science, and particularly relates to an ADHD graph convolution model construction method based on fMRI spatial-temporal characteristics, and the method comprises the following steps: processing resting state functional magnetic resonance data; fMRI sequence processing: randomly cutting fMRI sequences from different sites to obtain fMRI sequences with consistent sequence lengths, and obtaining a functional connection matrix according to Pearson's correlation; model input data: the model input data is composed of an fMRI sequence, ADHD phenotype information and a functional connection matrix, and the input data is divided to obtain a training set, a verification set and a test set; constructing a model; and model training: model evaluation. According to the method, the gated feature fusion module is proposed to fuse the captured fMRI features to obtain the spatial-temporal features, the phenotypic information of the subject is considered, the ADHD graph convolution model based on the fMRI spatial-temporal features is constructed, and the classification accuracy of ADHD diseases is improved.

Description

Technical Field

[0001] The present invention belongs to the technical fields of deep learning and brain science, and particularly relates to a method for constructing an ADHD graph convolutional model based on fMRI spatio-temporal features. Background Art

[0002] Attention Deficit and Hyperactivity Disorder (ADHD) is a neuropsychiatric disorder characterized by inattention, hyperactivity, or impulsivity. At present, the pathological mechanism of ADHD has not been fully clarified, which leads to many challenges in the clinical diagnosis process. The continuous maturity of medical imaging technology provides a new way to explore the pathological mechanism of ADHD. Among them, the Resting State Functional Magnetic Resonance Imaging (rs-fMRI) imaging technology has the advantages of high temporal resolution and high spatial resolution, and has been widely used in the auxiliary diagnosis research of ADHD brain diseases.

[0003] Data-driven deep learning technology can fully explore the features of fMRI time series and has been widely used in the auxiliary diagnosis research of brain diseases. The current mainstream deep learning algorithm applied to ADHD disease research is the Convolutional Neural Networks (CNN) model, which is suitable for exploring data features with Euclidean space characteristics. Since the brain functional connection network has non-Euclidean features, the classification accuracy of using the CNN model for ADHD diseases is often not high.

[0004] In recent years, the Graph Convolutional Network (GCN) has attracted wide attention due to its powerful function of analyzing non-Euclidean structure data and has been successfully used in ADHD disease diagnosis. However, the GCN does not fully consider the spatio-temporal features simultaneously possessed by the fMRI sequence itself, resulting in the unsatisfactory performance of some models represented by the GCN. How to use the GCN model to simultaneously capture the spatio-temporal features of the fMRI sequence for ADHD classification diagnosis is the main research content at present. Summary of the Invention

[0005] Aiming at the technical problem that the existing GCN model does not fully explore the spatio-temporal features of the fMRI sequence, the present invention provides a method for constructing an ADHD graph convolutional model based on fMRI spatio-temporal features.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0007] A method for constructing an ADHD graph convolutional model based on fMRI spatio-temporal features, comprising the following steps:

[0008] S1. Resting-state functional magnetic resonance data processing: Obtain the resting-state functional magnetic resonance data of the subjects, perform preprocessing operations on the data to obtain the fMRI sequence;

[0009] S2. fMRI sequence processing: Randomly crop the fMRI sequences from different sites to obtain fMRI sequences with the same sequence length, and obtain the functional connectivity matrix according to Pearson correlation;

[0010] S3. Model input data: The model input data consists of the fMRI sequence, ADHD phenotype information, and the functional connectivity matrix. Divide the input data to obtain the training set, validation set, and test set;

[0011] S4. Model construction: Construct a graph convolutional model based on the extraction of fMRI spatio-temporal features; this model integrates multiple neural network modules: a multi-scale interaction convolutional module for simultaneously learning the local information and long-range dependence relationship of the fMRI sequence; a time feature extraction module based on the multi-head self-attention mechanism for obtaining the attention of the time window and the time features of the fMRI sequence; a Chebyshev graph convolutional network module based on the time window for extracting the spatial features of the interaction between brain regions in the fMRI sequence under different time windows; a gated feature fusion module for fusing time and spatial features;

[0012] S5. Model training: Use the binary cross-entropy loss function to measure the difference between the predicted label and the true label. Input the training set and validation set obtained in S3 into the model constructed in S4 for training until the model converges;

[0013] S6. Model evaluation: Use the test set divided in S3 to verify and evaluate the model after convergence in S5. Calculate the accuracy, precision, recall, and F1 score of the prediction results as evaluation indicators, and compare the results with existing models in this field.

[0014] The data preprocessing operation in S1 uses the AFNI preprocessing tool to preprocess the original image data of the subjects, and obtain the fMRI sequence of the brain regions based on the Craddock 200 (CC200) functional segmentation atlas.

[0015] The random cropping method for the fMRI sequence in S2 is: randomly set the cropping points for the complete fMRI sequence for cropping to make its sequence length consistent.

[0016] The generation method of the functional connectivity matrix in S2 is: use Pearson correlation analysis for the fMRI sequences of each ROI to obtain the functional connectivity matrix.

[0017] In step S3, the ratio of the input data divided into the training set, validation set, and test set is 7:2:1.

[0018] The multi-scale interactive convolution module in step S4 includes: two one-dimensional convolutional layers with convolutional kernel sizes of 1 and 3 respectively, which are used to learn the local information and long-range dependencies of the fMRI sequence.

[0019] The time feature extraction module based on the multi-head self-attention mechanism in step S4 includes: first, windowing the fMRI sequence in the time dimension after passing through the multi-scale interactive convolution module; second, applying the multi-head self-attention mechanism to each time window for calculation to obtain attention values and attention probabilities; finally, performing a projection operation on the attention scores and the fMRI sequence to obtain the time features after attention processing.

[0020] The Chebyshev graph convolutional network module based on time windows in step S4 includes: first, performing Chebyshev graph convolution on the functional connectivity matrix and performing a dimension rearrangement operation to obtain a feature map, then performing a Hadamard product of the feature map and the attention scores to obtain a time attention-based feature map, and finally obtaining spatial features through residual connection.

[0021] The gated feature fusion module in step S4 includes: first, initializing a trainable weight vector in the range [0,1], and fusing the time features and spatial features according to this weight vector to obtain the fused spatio-temporal features.

[0022] The formula for the binary cross-entropy loss function in step S5 is:

[0023]

[0024] where y i is the binary label 0 or 1, and p(y i ) is the probability that the output belongs to the label.

[0025] Compared with the prior art, the beneficial effects of the present invention are:

[0026] 1. The present invention proposes a multi-scale interactive convolution module, which captures both the local information and long-range dependencies of the fMRI sequence.

[0027] 2. The present invention proposes a time feature extraction module based on the multi-head self-attention mechanism and a Chebyshev graph convolutional network module based on time window attention, which fully explores the spatio-temporal features of the fMRI sequence from different dimensions.

[0028] 3. The present invention proposes a gating feature fusion module to fuse the captured fMRI features to obtain spatio-temporal features, and considers the subject phenotype information to construct an ADHD graph convolutional model based on fMRI spatio-temporal features, improving the classification accuracy of ADHD disease. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only exemplary, and for those of ordinary skill in the art, without creative efforts, other implementation drawings can be obtained according to the provided drawings.

[0030] The structures, proportions, sizes, etc. shown in this specification are only used to cooperate with the content disclosed in the specification for those familiar with this technology to understand and read, and are not used to limit the limiting conditions for the implementation of the present invention. Therefore, they do not have technical essence. Any modification of the structure, change of the proportional relationship or adjustment of the size, without affecting the effects that the present invention can produce and the purposes that can be achieved, should still fall within the scope covered by the technical content disclosed in the present invention.

[0031] Figure 1 It is a flowchart of the implementation steps of the method of the present invention;

[0032] Figure 2 It is a framework diagram of the method of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0033] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. These descriptions are only for further explaining the features and advantages of the present invention, rather than limiting the claims of the present invention; based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope protected by the present application.

[0034] The following will further describe in detail the specific implementation manners of the present invention in conjunction with the drawings and embodiments. The following embodiments are used to illustrate the present invention, but are not used to limit the scope of the present invention.

[0035] As Figures 1 to 2 shown, this embodiment proposes a method for constructing an ADHD graph convolutional model based on fMRI spatio-temporal features, which mainly includes the following steps:

[0036] As Figure 1 shown, according to Figure 1The following is a detailed description of the process of constructing an ADHD graph convolutional model based on fMRI spatio-temporal features:

[0037] Step 1: Obtain the resting-state functional magnetic resonance data of the subjects in the ADHD-200 dataset, perform preprocessing operations on the data, and obtain the fMRI sequence.

[0038] The specific implementation steps are as follows:

[0039] 1-1) First, obtain the resting-state functional magnetic resonance imaging data of the subjects in the ADHD-200 dataset, and use the processing tool AFNI to preprocess the original image data, successively completing skull stripping, segmentation, registration, and spatial smoothing operations.

[0040] 1-2) Remove the influence of the cerebellum, use the Craddock 200 (CC200) functional segmentation atlas to generate a preprocessed average time series matrix with 190 ROIs, calculate the normalized value of its blood oxygenation level-dependent contrast (BOLD) signal as the time series feature of the ROI, so that each sample obtains time series signal data containing 190 ROIs, and the time series feature of each subject is where N is the number of ROIs and M is the number of sampling time points.

[0041] Step 2: Randomly crop the fMRI sequences from different sites to obtain fMRI sequences with the same sequence length, and obtain the functional connectivity matrix according to Pearson correlation.

[0042] The specific implementation steps are as follows:

[0043] 2-1) Randomly set the cropping points for the complete time series for cropping. Here, the number of randomly set cropping points is 5, that is, a new dataset five times the size of the original dataset is obtained.

[0044] 2-2) Set the cropped sequence length to 120. The finally obtained samples with the same time series length can be expressed as where each cropped sample is regularized to 120×190, that is, sequence length × number of ROIs.

[0045] 2-3) Calculate the correlation between any brain regions of each sample to construct the functional connectivity matrix. For N brain regions, connections are obtained. Use the Pearson correlation coefficient to measure the correlation between brain regions. For sample X i the fMRI sequence of brain region A and the fMRI sequence of brain region B, its PCC calculation method is shown in formula (1):

[0046]

[0047] In formula (1), μ A , μ B , σ A , σ B are the mean and standard deviation of the fMRI sequences on brain regions A and B respectively, and E(.) represents the expected value of a random variable.

[0048] 3-2) Use the Fisher Z-transform to transform the Pearson correlation coefficient, improve the consistency of the variance and stabilize the distribution of the correlation coefficient, and finally represent it as a resting-state functional connectivity matrix.

[0049] Step 3: The model input data consists of fMRI sequences, ADHD phenotype information, and a functional connectivity matrix, and the input data is divided into a training set, a validation set, and a test set.

[0050] The specific implementation steps are as follows:

[0051] 3-1) The fMRI sequence data is where N is the number of ROIs and M is the number of sampling time points.

[0052] 3-2) The functional connectivity matrix is an N×N-dimensional matrix composed of the Pearson correlation coefficient ρ between any brain regions.

[0053] 3-3) The phenotype information includes: gender, handedness, age, IQ, and the concatenated phenotype features are where n is the number of subjects.

[0054] 3-4) Use the functional connectivity matrix as the subject space information, the fMRI sequence data as the subject time series information, and the corresponding subject phenotype information as the input of the subsequent model.

[0055] 3-5) Divide the input data into a training set, a validation set, and a test set according to the ratio of 7:2:1.

[0056] Step 4: Construct a graph convolutional model based on fMRI spatio-temporal feature extraction; this model integrates multiple neural network modules: a multi-scale interaction convolutional module for simultaneously learning the local information and long-range dependencies of fMRI sequences; a time feature extraction module based on a multi-head self-attention mechanism for obtaining the attention of time windows and the time features of fMRI sequences; a Chebyshev graph convolutional network module based on time windows for extracting the spatial features of the interaction between brain regions of fMRI sequences under different time windows; and a gated feature fusion module for fusing time and spatial features.

[0057] The specific implementation steps are as follows:

[0058] 4-1) Perform positional embedding on the time series features through a learnable embedding layer to generate a unique positional encoding for each time step in the sequence. Set the maximum number of positional embeddings to 512. The size of the hidden layer is set to the number of nodes in the network, ensuring that the dimension of the embedding matches the input requirements of the subsequent layers. Additionally, traditional word embeddings are not used at this stage, but rather focus on encoding the time series positional information.

[0059] 4-2) Perform multi-scale convolution on the encoded tensor. First, pass the tensor through one-dimensional convolutional layers with kernel sizes of 1 and 3 respectively to obtain tensors S 1 and tensor S 2 , and through the GELU activation function and the Dropout layer to obtain the corresponding tensors S ′ 1 and S ′ 2 . Perform the Hadamard product operation on tensors S 1 and S ′ 2 to obtain tensor T 1 , and at the same time perform the same operation on tensors S 2 and S ′ 1 to obtain tensor T 2 . Finally, add T 1 and T 2 to obtain the final output tensor. The formula for the GELU activation function is as follows:

[0060]

[0061] where x is the tensor in the transfer process.

[0062] The formula for the Hadamard product is as follows:

[0063] (C⊙D) ij =c ij *d ij #(3)

[0064] where c ij , d ij are the elements at the corresponding positions of the original two tensors C and D.

[0065] 4-3) Divide the fMRI sequence after passing through the multi-scale interaction convolution module into different time windows, and apply the multi-head self-attention mechanism to each time window for calculation to obtain attention values and attention probabilities; finally, perform a projection operation on the attention scores and the original input data to obtain the time features after attention processing.

[0066] Let be the input tensor corresponding to each time window, They are the learned d-dimensional query, key, and value vectors respectively, and h is the number of heads. Specifically, assume the input sequence is z 1 , z 2 ,..., z n , then the calculation of multi-head attention is as follows:

[0067] MultiHead(z) = Concat(head 1 , …, head h )W O #(4)

[0068] where Q, K, V represent the queries, keys, and values of the input sequence respectively, head i represents the i-th attention head, and W O is the weight parameter of the linear projection. Each attention head is obtained by performing self-attention calculation on Q, K, V, and the calculation formula is as follows:

[0069]

[0070] where d k is the dimension size of Q and K, used to scale the size of attention.

[0071] head i = Attention(ZQ i , ZK i , ZV i ), i = 1, …, h#(6)

[0072]

[0073] where are the learned projection matrices respectively, and q, k, and v are the dimensions of the query, key, and value vectors.

[0074] 4-4) Use the Chebyshev graph convolutional network module based on time window attention to process the functional connectivity matrix: First, perform graph convolution operation on the functional connectivity matrix of each subject to obtain the feature map. The corresponding graph convolution formula is as follows:

[0075] g * x = U(U T gU T x) = Ug θ U T x#(8)

[0076] where g θ is a learnable filter, U is the feature matrix, and x is the feature vector of the node.

[0077] Approximate calculations are carried out using Chebyshev polynomials, and the Chebyshev polynomials are expressed as follows:

[0078]

[0079] where θ k is a learnable coefficient, is the eigenvalue matrix of the normalized Laplacian matrix.

[0080] The Chebyshev graph convolutional layer corresponds to the graph convolution formula as follows:

[0081]

[0082] where is the normalized Laplacian matrix.

[0083] The feature map and the attention scores are subjected to Hadamard product to obtain a feature map focused on the time window, and finally spatial features are obtained through residual connection.

[0084] 4-5) Change the time window scale, and repeat steps (4-3) and (4-4) for the sequence passing through the Chebyshev graph convolutional network module based on the time window.

[0085] 4-6) Initialize a learnable parameter, fuse the time features and spatial features according to this weight vector, and finally perform batch normalization to obtain the fused spatio-temporal features.

[0086] The formula of the gated fusion mechanism adopted is as follows:

[0087] G ST =(1 - θ)G S +θG T #(11)

[0088] where the learnable parameter θ∈[0,1] controls the fusion weight, G S represents the spatial features, and G T represents the time features.

[0089] Step 5: Model training: Use the binary cross-entropy loss function to measure the difference between the predicted labels and the true labels, and input the training set and validation set obtained in S3 into the model constructed in S4 for training until the model converges.

[0090] The specific implementation steps are as follows:

[0091] 5-1) Use the binary cross-entropy loss function to measure the difference between the predicted labels and the true labels, and input the training set and validation set obtained in S3 into the model constructed in S4 for training and validation. The formula of the binary cross-entropy loss function is as follows:

[0092]

[0093] Among them, y i is a binary label 0 or 1, and p(y i ) is the probability that the output belongs to the label.

[0094] Step 6: Model evaluation: Use the test set of S2 to verify and evaluate the model after convergence of S4, calculate the accuracy (ACC), precision, recall, and F1 score of the prediction results as evaluation indicators, and compare the results with existing models in this field.

[0095] The specific implementation steps are as follows:

[0096] 6-1) Calculate the accuracy (ACC), precision, recall, and F1 score indicators of the prediction results to evaluate the performance of the model. The calculation formulas are as follows:

[0097]

[0098] Among them, TP, TN, FP, and FN represent true positive, true negative, false positive, and false negative respectively. Among them, TP represents the number of ADHD patients correctly classified, TN represents the number of normal subjects correctly classified, FP represents the number of normal subjects misclassified as ADHD patients, and FN represents the number of ADHD patients misclassified as normal subjects.

[0099] The comparison of the model evaluation results with existing models in this field is shown in Table 1:

[0100] Table 1 Comparison table of experimental results between the model in this paper and existing models

[0101]

[0102] It can be seen that the model in this embodiment performs best in terms of the classification accuracy (ACC) index, and also performs relatively well in terms of precision, recall, and F1 score and other indicators.

[0103] The above results prove that the model in this embodiment fully explores the features of fMRI sequences from different dimensions, fuses spatio-temporal features, and considers the phenotypic information of subjects, improving the classification accuracy of ADHD diseases and providing a scientific method for ADHD diagnosis.

[0104] The above only describes in detail the preferred embodiments of the present invention. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those of ordinary skill in the art, various changes can be made without departing from the gist of the present invention, and all such changes should be included within the scope of protection of the present invention.

Claims

1. A method for constructing an ADHD graph convolution model based on fMRI spatiotemporal features, characterized in that: The following steps are involved: S1. Resting-state functional MRI data processing: Obtain the resting-state functional MRI data of the subject, perform preprocessing operations on the data, and obtain the fMRI sequence; S2, fMRI sequence processing: fMRI sequences from different sites were randomly cropped to obtain fMRI sequences with consistent sequence lengths, and the functional connectivity matrix was obtained based on Pearson correlation; S3, model input data: The model input data consists of fMRI sequences, ADHD phenotype information and functional connectivity matrix. The input data is divided into training set, validation set and test set. S4. Model construction: Construct a graph convolution model based on fMRI spatiotemporal feature extraction; the model integrates multiple neural network modules: a multi-scale interactive convolution module for simultaneously learning local information and long-range dependencies of fMRI sequences; a temporal feature extraction module based on a multi-head self-attention mechanism for acquiring the attention of time windows and temporal features of fMRI sequences; a Chebyshev graph convolution network module based on time windows for extracting spatial features of brain region interactions in fMRI sequences under different time windows; and a gated feature fusion module for fusing temporal and spatial features. S5, model training: Use the binary cross entropy loss function to measure the difference between the predicted label and the true label, and input the training set and validation set obtained in S3 into the model constructed in S4 for training until the model converges; S6, model evaluation: Use the test set divided by S3 to verify and evaluate the model after S5 convergence, calculate the accuracy, precision, recall rate and F1 score of the prediction results as evaluation indicators, and compare the results with existing models in this field.

2. The method for constructing an ADHD graph convolution model based on fMRI spatiotemporal features according to claim 1, characterized in that: The data preprocessing operation in S1 uses the AFNI preprocessing tool to preprocess the original image data of the subjects, and obtains the fMRI sequence of the brain region based on the Craddock 200 (CC200) functional segmentation atlas.

3. The method for constructing an ADHD graph convolution model based on fMRI spatiotemporal features according to claim 1, characterized in that: The random cropping method for the fMRI sequence in S2 is: randomly setting cropping points for the complete fMRI sequence to crop the sequence so that the sequence length is consistent.

4. The method for constructing an ADHD graph convolution model based on fMRI spatiotemporal features according to claim 1, characterized in that: The method for generating the functional connection matrix in S2 is: using Pearson correlation analysis on the fMRI sequence of each ROI to obtain the functional connection matrix.

5. The method for constructing an ADHD graph convolution model based on fMRI spatiotemporal features according to claim 1, characterized in that: In S3, the input data is divided into a training set, a validation set, and a test set in a ratio of 7:2:

1.

6. The method for constructing an ADHD graph convolution model based on fMRI spatiotemporal features according to claim 1, characterized in that: The multi-scale interactive convolutional interaction module in S4 includes: two one-dimensional convolutional layers with convolution kernel sizes of 1 and 3, which are used to learn the local information and long-range dependency of fMRI sequences respectively.

7. The method for constructing an ADHD graph convolution model based on fMRI spatiotemporal features according to claim 1, characterized in that: The temporal feature extraction module based on the multi-head self-attention mechanism in S4 includes: first, windowing the fMRI sequence after the multi-scale interactive convolution module in the time dimension; second, applying the multi-head self-attention mechanism to each time window to calculate the attention value and attention probability; finally, performing a projection operation on the attention score and the fMRI sequence to obtain the temporal feature after attention processing.

8. The method for constructing an ADHD graph convolution model based on fMRI spatiotemporal features according to claim 1, characterized in that: The time window-based Chebyshev graph convolution network module in S4 includes: first, performing Chebyshev graph convolution on the functional connection matrix and performing dimension rearrangement operation to obtain a feature map, then performing Hadamard product between the feature map and the attention score to obtain a feature map based on time attention, and finally obtaining spatial features through residual connection.

9. The method for constructing an ADHD graph convolution model based on fMRI spatiotemporal features according to claim 1, characterized in that: The gated feature fusion module in S4 includes: firstly initializing a trainable weight vector in the range of [0, 1], fusing the temporal features and the spatial features according to the weight vector, and obtaining the fused spatiotemporal features.

10. The method for constructing an ADHD graph convolution model based on fMRI spatiotemporal features according to claim 1, characterized in that: The binary cross entropy loss function formula in S5 is: Among them, y i is a binary label 0 or 1, p(y i ) is the probability that the output belongs to the label.

Images