A method for constructing an ADHD graph convolutional model based on fMRI spatiotemporal features

Abstract

The present invention belongs to the fields of deep learning and brain science technology, specifically relating to a method for constructing an ADHD graph convolutional model based on fMRI spatiotemporal features. The method comprises the following steps: resting-state functional MRI data processing; fMRI sequence processing: randomly cropping fMRI sequences from different sites to obtain fMRI sequences of consistent length, and obtaining a functional connectivity matrix based on Pearson correlation; model input data: the model input data consists of fMRI sequences, ADHD phenotypic information, and a functional connectivity matrix, and the input data is divided into training, validation, and test sets; model construction; model training; and model evaluation. The present invention proposes a gated feature fusion module to fuse captured fMRI features to obtain spatiotemporal features. Furthermore, the method considers subject phenotypic information to construct an ADHD graph convolutional model based on fMRI spatiotemporal features, thereby improving the classification accuracy of ADHD.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning and brain science technology, and specifically relates to a method for constructing an ADHD graph convolution model based on fMRI spatiotemporal features. Background Art

[0002] Attention Deficit and Hyperactivity Disorder (ADHD) is a neurological or psychiatric disorder characterized by inattention, hyperactivity, or impulsivity. The pathological mechanisms of ADHD are not yet fully understood, leading to numerous challenges in clinical diagnosis. The continued maturation of medical imaging technology has provided new avenues for exploring the pathological mechanisms of ADHD. Resting-state functional Magnetic Resonance Imaging (rs-fMRI) imaging technology, with its advantages of high temporal and spatial resolution, has been widely used in auxiliary diagnostic research for ADHD brain disorders.

[0003] Data-driven deep learning techniques can fully exploit the characteristics of fMRI time series and have been widely used in aiding the diagnosis of brain diseases. The mainstream deep learning algorithm currently used in ADHD research is the Convolutional Neural Network (CNN) model, which is suitable for mining data features with Euclidean spatial properties. Due to the non-Euclidean characteristics of brain functional connectivity networks, the accuracy of ADHD classification using CNN models is often low.

[0004] In recent years, graph convolutional networks (GCNs) have attracted widespread attention for their powerful ability to analyze non-Euclidean data and have been successfully used for ADHD diagnosis. However, GCNs fail to fully consider the spatiotemporal characteristics of fMRI sequences, leading to suboptimal performance of some models, particularly GCNs. Current research focuses on how to leverage GCNs to simultaneously capture the spatiotemporal characteristics of fMRI sequences for ADHD classification and diagnosis. Summary of the Invention

[0005] In response to the technical problem that the existing GCN model does not sufficiently mine the spatiotemporal features of fMRI sequences, the present invention provides a method for constructing an ADHD graph convolution model based on fMRI spatiotemporal features.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0007] A method for constructing an ADHD graph convolutional model based on fMRI spatiotemporal features includes the following steps:

[0008] S1. Resting-state functional MRI data processing: Obtain the resting-state functional MRI data of the subjects, perform preprocessing operations on the data, and obtain fMRI sequences;

[0009] S2. fMRI sequence processing: fMRI sequences from different sites were randomly cropped to obtain fMRI sequences with consistent sequence lengths, and the functional connectivity matrix was obtained based on Pearson correlation;

[0010] S3. Model input data: The model input data consists of fMRI sequences, ADHD phenotype information, and functional connectivity matrices. The input data is divided into training, validation, and test sets.

[0011] S4. Model construction: Construct a graph convolution model based on fMRI spatiotemporal feature extraction; this model integrates multiple neural network modules: a multi-scale interactive convolution module for simultaneously learning local information and long-range dependencies in fMRI sequences; a temporal feature extraction module based on a multi-head self-attention mechanism for acquiring attention in time windows and temporal features of fMRI sequences; a Chebyshev graph convolution network module based on time windows for extracting spatial features of brain region interactions in fMRI sequences under different time windows; and a gated feature fusion module for fusing temporal and spatial features.

[0012] S5, model training: Use the binary cross entropy loss function to measure the difference between the predicted label and the true label. Input the training set and validation set obtained in S3 into the model constructed in S4 for training until the model converges.

[0013] S6. Model evaluation: Use the test set divided in S3 to verify and evaluate the model after convergence in S5, calculate the accuracy, precision, recall rate and F1 score of the prediction results as evaluation indicators, and compare the results with existing models in this field.

[0014] The data preprocessing operation in S1 uses the AFNI preprocessing tool to preprocess the original image data of the subjects, and obtains the fMRI sequence of the brain region based on the Craddock 200 (CC200) functional segmentation atlas.

[0015] The random cropping method for the fMRI sequence in S2 is: randomly setting cropping points on the complete fMRI sequence to crop it so that the sequence length is consistent.

[0016] The functional connectivity matrix in S2 is generated by performing Pearson correlation analysis on the fMRI sequence of each ROI to obtain the functional connectivity matrix.

[0017] In S3, the input data is divided into a training set, a validation set, and a test set in a ratio of 7:2:1.

[0018] The multi-scale interactive convolution module in S4 includes two one-dimensional convolution layers with convolution kernel sizes of 1 and 3, which are used to learn local information and long-range dependencies of fMRI sequences respectively.

[0019] The temporal feature extraction module based on the multi-head self-attention mechanism in S4 includes: first, windowing the fMRI sequence after the multi-scale interactive convolution module in the time dimension; second, applying the multi-head self-attention mechanism to each time window to calculate the attention value and attention probability; finally, performing a projection operation on the attention score and the fMRI sequence to obtain the temporal feature after attention processing.

[0020] The time window-based Chebyshev graph convolutional network module in S4 includes: first, performing Chebyshev graph convolution on the functional connection matrix and performing dimensionality rearrangement operation to obtain a feature map, then performing Hadamard product between the feature map and the attention score to obtain a feature map based on time attention, and finally obtaining spatial features through residual connection.

[0021] The gated feature fusion module in S4 includes: first initializing a trainable weight vector in the range of [0, 1], fusing the temporal features and the spatial features according to the weight vector, and obtaining the fused spatiotemporal features.

[0022] The binary cross entropy loss function formula in S5 is:

[0023]

[0024] Among them, y i is a binary label 0 or 1, p(y i ) is the probability that the output belongs to the label.

[0025] Compared with the prior art, the present invention has the following beneficial effects:

[0026] 1. This paper proposes a multi-scale interactive convolution module to simultaneously capture the local information and long-range dependencies of fMRI sequences.

[0027] 2. This paper proposes a temporal feature extraction module based on a multi-head self-attention mechanism and a Chebyshev graph convolutional network module based on time window attention, which fully explores the spatiotemporal characteristics of fMRI sequences from different dimensions.

[0028] 3. The present invention proposes a gated feature fusion module to fuse the captured fMRI features to obtain spatiotemporal features, and considers the subject's phenotypic information to construct an ADHD graph convolution model based on fMRI spatiotemporal features, thereby improving the classification accuracy of ADHD diseases. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the embodiments or the description of the prior art. Obviously, the drawings described below are merely exemplary, and those skilled in the art can, without inventive effort, derive other implementation drawings based on the provided drawings.

[0030] The structures, proportions, sizes, etc. illustrated in this specification are intended solely to complement the contents disclosed herein and to facilitate understanding and reading by persons skilled in the art. They are not intended to limit the conditions under which the present invention may be implemented and therefore have no substantive technical significance. Any structural modifications, changes in proportions, or adjustments in sizes, without affecting the efficacy and objectives of the present invention, shall remain within the scope of the technical contents disclosed herein.

[0031] Figure 1 Schematic diagram of the implementation steps of the method of the present invention;

[0032] Figure 2 It is a framework diagram of the method of the present invention. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only part of the embodiments of this application, not all the embodiments. These descriptions are only to further illustrate the features and advantages of the present invention, rather than to limit the claims of the present invention. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0034] The following embodiments of the present invention are described in further detail with reference to the accompanying drawings and examples. The following embodiments are used to illustrate the present invention but are not intended to limit the scope of the present invention.

[0035] like Figures 1 to 2 As shown, this embodiment proposes a method for constructing an ADHD graph convolution model based on fMRI spatiotemporal features, which mainly includes the following steps:

[0036] like Figure 1 As shown, according to Figure 1The following is a detailed description of the process of constructing an ADHD graph convolution model based on fMRI spatiotemporal features:

[0037] Step 1: Obtain resting-state functional magnetic resonance imaging (fMRI) data of subjects in the ADHD-200 dataset, perform preprocessing on the data, and obtain fMRI sequences.

[0038] The specific implementation steps are as follows:

[0039] 1-1) First, resting-state functional magnetic resonance imaging data of subjects in the ADHD-200 dataset were obtained. The raw image data were preprocessed using the processing tool AFNI, completing skull removal, segmentation, registration, and spatial smoothing operations in sequence.

[0040] 1-2) Remove the influence of the cerebellum and use the Craddock 200 (CC200) functional segmentation atlas to generate a preprocessed average time series matrix with 190 ROIs. Calculate the normalized value of the blood oxygen concentration-dependent contrast (BOLD) signal as the time series feature of the ROI, so that each sample obtains time series signal data containing 190 ROIs. The time series feature of each subject is Where N is the number of ROIs and M is the number of sampling time points.

[0041] Step 2: Randomly crop the fMRI sequences from different sites to obtain fMRI sequences with consistent sequence length, and obtain the functional connectivity matrix based on Pearson correlation.

[0042] The specific implementation steps are as follows:

[0043] 2-1) The complete time series is cropped by randomly setting cropping points. Here, the number of random cropping points is set to 5, which means that a new dataset is obtained after the original dataset is expanded five times.

[0044] 2-2) Set the length of the cropped sequence to 120, and the samples with the same time series length can be expressed as Among them, each cropped sample is normalized to 120×190, that is, sequence length×number of ROIs.

[0045] 2-3) Calculate the correlation between any brain regions of each sample to construct a functional connectivity matrix. For N brain regions, we get The Pearson correlation coefficient is used to measure the correlation between brain regions. For sample X i fMRI sequence of brain area A and fMRI sequences of brain area B The PCC calculation method is shown in formula (1):

[0046]

[0047] In formula (1), μ A 、μ B , σ A , σ B are the mean and standard deviation of the fMRI sequences in brain area A and brain area B, respectively, and E(.) represents the expected value of the random variable.

[0048] 3-2) The Pearson correlation coefficient was transformed using Fisher Z transformation to improve the consistency of variance and stabilize the distribution of the correlation coefficient, and finally expressed as a resting state functional connectivity matrix.

[0049] Step 3: The model input data consists of fMRI sequences, ADHD phenotype information, and functional connectivity matrix. The input data is divided into training set, validation set, and test set.

[0050] The specific implementation steps are as follows:

[0051] 3-1) fMRI sequence data is Where N is the number of ROIs and M is the number of sampling time points.

[0052] 3-2) The functional connectivity matrix is ​​an N×N dimensional matrix composed of the Pearson correlation coefficient ρ between any brain regions.

[0053] 3-3) Phenotypic information includes: gender, handedness, age, IQ, and the spliced ​​phenotypic characteristics are Where n is the number of subjects.

[0054] 3-4) The functional connectivity matrix is ​​used as the spatial information of the subject, the fMRI sequence data is used as the time series information of the subject, and the corresponding phenotypic information of the subject is used as the input of the subsequent model.

[0055] 3-5) The input data is divided into training set, validation set and test set in a ratio of 7:2:1.

[0056] Step 4: Construct a graph convolution model based on fMRI spatiotemporal feature extraction; this model integrates multiple neural network modules: a multi-scale interactive convolution module for simultaneously learning local information and long-range dependencies of fMRI sequences; a temporal feature extraction module based on a multi-head self-attention mechanism for acquiring the attention of time windows and the temporal features of fMRI sequences; a Chebyshev graph convolution network module based on time windows for extracting spatial features of brain region interactions in fMRI sequences under different time windows; and a gated feature fusion module for fusing temporal and spatial features.

[0057] The specific implementation steps are as follows:

[0058] 4-1) Positional embedding of time series features is performed using a learnable embedding layer, generating a unique positional encoding for each time step in the sequence. The maximum number of positional embeddings is set to 512. The size of the hidden layer is set to the number of nodes in the network, ensuring that the embedding dimension matches the input requirements of subsequent layers. Furthermore, traditional word embeddings are not used at this stage, focusing instead on encoding the positional information of the time series.

[0059] 4-2) Perform multi-scale convolution on the encoded tensor. First, pass the tensor through a one-dimensional convolution layer with a convolution kernel size of 1 and 3 to obtain tensors S1 and S2, respectively. Then, pass the GELU activation function and the Dropout layer to obtain the corresponding tensor S. ′ 1 and S ′ 2. For tensors S1 and S ′ 2 Perform Hadamard product operation to obtain tensor T1, and at the same time perform Hadamard product operation on tensors S2 and S ′ 1 performs the same operation to obtain tensor T2, and finally adds T1 and T2 to obtain the final output tensor. The GELU activation function formula is as follows:

[0060]

[0061] Among them, x is the tensor in the transfer process.

[0062] The Hadamard product calculation formula is as follows:

[0063] (C⊙D) ij =c ij *d ij #(3)

[0064] where c ij , d ij are the elements of the corresponding positions of the original two tensors C and D.

[0065] 4-3) The fMRI sequence after the multi-scale interactive convolution module is divided into different time windows, and the multi-head self-attention mechanism is applied to each time window to calculate the attention value and attention probability; finally, the attention score and the original input data are projected to obtain the time features after attention processing.

[0066] set up is the input tensor corresponding to each time window, are the learned d-dimensional query, key, and value vectors, respectively, and h is the number of heads. Specifically, assuming the input sequence is z1,z2,...,z n , then the calculation of multi-head attention is as follows:

[0067] MultiHead(z)=Concat(head1,…,headh )W O #(4)

[0068] Among them, Q, K, V represent the queries, keys and values ​​of the input sequence respectively, head i represents the i-th attention head, W O is the weight parameter of the linear projection. Each attention head is obtained by performing self-attention calculation on Q, K, and V. The calculation formula is as follows:

[0069]

[0070] Among them, d k is the dimension size of Q, K, which is used to scale the size of attention.

[0071] head i =Attention(ZQ i ,ZK i ,ZV i ),i=1,…,h#(6)

[0072]

[0073] in, are the learned projection matrices, q, k, and v are the dimensions of the query, key, and value vectors, respectively.

[0074] 4-4) The functional connectivity matrix is ​​processed using a Chebyshev graph convolutional network module based on time window attention: First, a graph convolution operation is performed on the functional connectivity matrix of each subject to obtain a feature map. The corresponding graph convolution formula is as follows:

[0075] g*x=U(U T Ug T x)=Ug θ U T x#(8)

[0076] Among them, g θ is a learnable filter, U is the feature matrix, and x is the feature vector of the node.

[0077] The Chebyshev polynomials are used for approximate calculations. The Chebyshev polynomials are expressed as follows:

[0078]

[0079] Among them, θ k are the learnable coefficients, is the eigenvalue matrix of the normalized Laplacian matrix.

[0080] The corresponding graph convolution formula of the Chebyshev graph convolution layer is as follows:

[0081]

[0082] in, is the normalized Laplacian matrix.

[0083] The feature map is Hadamard-producted with the attention score to obtain a feature map based on time window attention, and finally the spatial feature is obtained through residual connection.

[0084] 4-5) Change the time window scale and repeat steps (4-3) and (4-4) for the sequence that passes through the time window-based Chebyshev graph convolutional network module.

[0085] 4-6) Initialize a learnable parameter, fuse the temporal features and spatial features according to the weight vector, and finally perform batch normalization to obtain the fused spatiotemporal features.

[0086] The gated fusion mechanism formula used is as follows:

[0087] G ST =(1-θ)G S +θG T #(11)

[0088] The learnable parameter θ∈[0,1] controls the fusion weight, G S Represents the spatial feature, G T Represents time characteristics.

[0089] Step 5: Model training: Use the binary cross entropy loss function to measure the difference between the predicted label and the true label. Input the training set and validation set obtained in S3 into the model constructed in S4 for training until the model converges.

[0090] The specific implementation steps are as follows:

[0091] 5-1) Use the binary cross entropy loss function to measure the difference between the predicted label and the true label. Input the training set and validation set obtained in S3 into the model constructed in S4 for training and validation. The binary cross entropy loss function formula is as follows:

[0092]

[0093] Among them, y i is a binary label 0 or 1, p(y i ) is the probability that the output belongs to the label.

[0094] Step 6: Model evaluation: Use the test set of S2 to verify and evaluate the model after S4 convergence, calculate the accuracy (ACC), precision (Precision), recall (Recall) and F1 score of the prediction results as evaluation indicators, and compare the results with existing models in this field.

[0095] The specific implementation steps are as follows:

[0096] 6-1) Calculate the accuracy (ACC), precision (Precision), and recall (F1) score of the prediction results to evaluate the performance of the model. The calculation formula is as follows:

[0097]

[0098] TP, TN, FP, and FN represent true positives, true negatives, false positives, and false negatives, respectively. TP represents the number of correctly classified ADHD patients, TN represents the number of correctly classified normal subjects, FP represents the number of normal subjects misclassified as ADHD patients, and FN represents the number of ADHD patients misclassified as normal subjects.

[0099] The model evaluation results are compared with existing models in this field as shown in Table 1:

[0100] Table 1 Comparison of experimental results between this model and existing models

[0101]

[0102] It can be seen that the model of this embodiment performs best in the classification accuracy (ACC) indicator, and also performs relatively well in indicators such as precision, recall, and F1 score.

[0103] The above results demonstrate that the model of this embodiment fully exploits the characteristics of fMRI sequences from different dimensions, integrates spatiotemporal features, and considers subject phenotypic information, thereby improving the classification accuracy of ADHD and providing a scientific approach for ADHD diagnosis.

[0104] The above only describes in detail the preferred embodiments of the present invention, but the present invention is not limited to the above embodiments. Various changes can be made within the knowledge of ordinary technicians in this field without departing from the purpose of the present invention, and various changes should be included in the scope of protection of the present invention.

Claims

1. A method for constructing an ADHD graph convolution model based on fMRI spatiotemporal features, characterized by: The following steps are involved: S1. Resting-state functional MRI data processing: Obtain the resting-state functional MRI data of the subjects, perform preprocessing operations on the data, and obtain fMRI sequences; S2. fMRI sequence processing: fMRI sequences from different sites were randomly cropped to obtain fMRI sequences with consistent sequence lengths, and the functional connectivity matrix was obtained based on Pearson correlation; S3. Model input data: The model input data consists of fMRI sequences, ADHD phenotype information, and functional connectivity matrices. The input data is divided into training, validation, and test sets. S4. Model construction: Construct a graph convolution model based on fMRI spatiotemporal feature extraction; this model integrates multiple neural network modules: a multi-scale interactive convolution module for simultaneously learning local information and long-range dependencies in fMRI sequences; a temporal feature extraction module based on a multi-head self-attention mechanism for acquiring attention in time windows and temporal features of fMRI sequences; a Chebyshev graph convolution network module based on time windows for extracting spatial features of brain region interactions in fMRI sequences under different time windows; and a gated feature fusion module for fusing temporal and spatial features. S5, model training: Use the binary cross entropy loss function to measure the difference between the predicted label and the true label. Input the training set and validation set obtained in S3 into the model constructed in S4 for training until the model converges. S6. Model evaluation: Use the test set divided in S3 to verify and evaluate the model after convergence in S5, calculate the accuracy, precision, recall rate and F1 score of the prediction results as evaluation indicators, and compare the results with existing models in this field.

2. The method for constructing an ADHD graph convolutional model based on fMRI spatiotemporal features according to claim 1, characterized in that: The data preprocessing operation in S1 uses the AFNI preprocessing tool to preprocess the original image data of the subjects, and obtains the fMRI sequence of the brain region based on the Craddock 200 (CC200) functional segmentation atlas.

3. The method for constructing an ADHD graph convolution model based on fMRI spatiotemporal features according to claim 1, characterized in that: The random cropping method for the fMRI sequence in S2 is: randomly setting cropping points on the complete fMRI sequence to crop it so that the sequence length is consistent.

4. The method for constructing an ADHD graph convolution model based on fMRI spatiotemporal features according to claim 1, characterized in that: The functional connectivity matrix in S2 is generated by performing Pearson correlation analysis on the fMRI sequence of each ROI to obtain the functional connectivity matrix.

5. The method for constructing an ADHD graph convolutional model based on fMRI spatiotemporal features according to claim 1, characterized in that: In S3, the input data is divided into a training set, a validation set, and a test set in a ratio of 7:2:

1.

6. The method for constructing an ADHD graph convolution model based on fMRI spatiotemporal features according to claim 1, characterized in that: The multi-scale interactive convolution module in S4 includes two one-dimensional convolution layers with convolution kernel sizes of 1 and 3, which are used to learn local information and long-range dependencies of fMRI sequences respectively.

7. The method for constructing an ADHD graph convolution model based on fMRI spatiotemporal features according to claim 1, characterized in that: The temporal feature extraction module based on the multi-head self-attention mechanism in S4 includes: first, windowing the fMRI sequence after the multi-scale interactive convolution module in the time dimension; second, applying the multi-head self-attention mechanism to each time window to calculate the attention value and attention probability; finally, performing a projection operation on the attention score and the fMRI sequence to obtain the temporal feature after attention processing.

8. The method for constructing an ADHD graph convolution model based on fMRI spatiotemporal features according to claim 1, characterized in that: The time window-based Chebyshev graph convolutional network module in S4 includes: first, performing Chebyshev graph convolution on the functional connection matrix and performing dimensionality rearrangement operation to obtain a feature map, then performing Hadamard product between the feature map and the attention score to obtain a feature map based on time attention, and finally obtaining spatial features through residual connection.

9. The method for constructing an ADHD graph convolution model based on fMRI spatiotemporal features according to claim 1, characterized in that: The gated feature fusion module in S4 includes: first initializing a trainable weight vector in the range of [0, 1], fusing the temporal features and the spatial features according to the weight vector, and obtaining the fused spatiotemporal features.

10. The method for constructing an ADHD graph convolution model based on fMRI spatiotemporal features according to claim 1, characterized in that: The binary cross entropy loss function formula in S5 is: Among them, y i is a binary label 0 or 1, p(y i ) is the probability that the output belongs to the label.

Images