Autism identification method based on functional partition and graph convolutional neural network

Abstract

The invention belongs to the field of autism recognition, and particularly relates to an autism recognition method based on functional partition and a graph convolutional neural network. Comprising the following steps: acquiring infantile autism brain imaging data, partitioning a brain by adopting a DMN mode, and constructing a local functional adjacency matrix and a global dynamic adjacency matrix according to partitions; processing the autism brain imaging data by adopting a spatial-temporal feature extractor to obtain spatial-temporal features; sending the spatial-temporal features to a frequency feature extractor to obtain frequency features; processing the frequency features and the local function adjacent matrix by adopting a local function feature extractor to obtain local function features; a global dynamic feature extractor is adopted to process the frequency features and the global dynamic adjacency matrix, and global dynamic features are obtained; carrying out feature fusion on the local function features and the global dynamic features; inputting the fused features into a classifier to obtain an autism recognition result; according to the invention, the data difference between different autism patients is reduced, and the recognition effect is improved.

Description

Technical Field

[0001] The present invention belongs to the field of autism recognition, and specifically relates to an autism recognition method based on functional partitioning and graph convolutional neural network. Background Art

[0002] Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder characterized by difficulties in social interaction, verbal and nonverbal communication, as well as narrow interests, repetitive behaviors and behavioral stereotyping. However, defining and diagnosing ASD is a very difficult process due to the overlapping nature of symptoms and the lack of biological tests that can serve as a clear and quantifiable gold standard. So far, the diagnosis of ASD patients is basically based on symptom-based behavioral descriptions, and the diagnosis is made through joint interviews between ASD patients and their families by experienced doctors. However, during the doctor's interview process, personal observations and subjective decisions often lead to misdiagnosis or overdiagnosis of mild cases. Therefore, there is an urgent need to implement effective computer-assisted diagnosis technology to help doctors diagnose ASD.

[0003] Existing technologies usually use traditional machine learning methods such as K-NN (k-Nearest Neighbors) and artificial neural networks (ANN) for autism identification. However, traditional machine learning methods usually rely on manual selection and preprocessing of features. The performance of ASD varies depending on individual differences, task types, environmental influences and other factors. Therefore, the generalization ability of traditional machine learning methods in different sample sets or different task conditions may be poor. The interpretability of traditional methods is relatively poor.

[0004] In order to solve the above problems, researchers have introduced deep learning and graph network technologies in recent years to improve the recognition ability of ASD. Deep learning models can automatically extract high-dimensional feature representations and capture complex temporal and spatial relationships in fMRI signals. And graph networks are naturally suitable for unstructured data such as the brain. Therefore, it is natural to use graph neural networks for research. However, existing methods only focus on a single graph structure when extracting features, while ignoring the potential features of local functions and global dynamic changes. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention proposes an autism recognition method based on functional partitioning and graph convolutional neural network, the method comprising: obtaining brain imaging data and inputting it into a trained autism recognition model to obtain an autism recognition result;

[0006] The training process of the autism recognition model includes:

[0007] S1: Obtain autism brain imaging data, use the DMN model to partition the brain, and construct a local functional adjacency matrix and a global dynamic adjacency matrix based on the partitions;

[0008] S2: Use the spatiotemporal feature extractor to process autism brain imaging data to obtain spatiotemporal features;

[0009] S3: Send the spatiotemporal features to the frequency feature extractor to obtain frequency features;

[0010] S4: using a local function feature extractor to process the frequency feature and the local function adjacency matrix to obtain a local function feature;

[0011] S5: using a global dynamic feature extractor to process the frequency features and the global dynamic adjacency matrix to obtain global dynamic features;

[0012] S6: Fusing local functional features and global dynamic features to obtain fused features;

[0013] S7: Input the fused features into the classifier to obtain autism recognition results;

[0014] S8: Calculate the total model loss and adjust the model parameters according to the total model loss. When the pre-set number of training times is reached, the model training is completed.

[0015] Preferably, the spatiotemporal feature extractor comprises a channel attention layer and a spatial attention layer;

[0016] The calculation formula of the channel attention layer is:

[0017] T (x) =σ(W1(W0(x ag ))+W1(W0(x max )))

[0018] The calculation formula of the spatial attention layer is:

[0019] S (x) =σ(f 7*7 ([x ag :x max ]))

[0020] Among them, T (x) represents the output of the channel attention layer, S (x) represents the output of the spatial attention layer, σ represents the sigmoid activation function, W1 and W0 represent the first and second training weight matrices, and x ag and x max Represents average pooling and maximum pooling, respectively, and f 7*7 Indicates a convolution with a convolution kernel size of 7*7.

[0021] Preferably, the frequency feature extractor is a wavelet convolution layer, which utilizes a convolution recursive operation to simulate a wavelet function to extract frequency features from spatiotemporal features to obtain frequency features.

[0022] Preferably, the local function feature extractor includes a graph convolution layer, a RELU activation layer and a Dropout layer.

[0023] Preferably, the global dynamic feature extractor includes a graph dynamic convolution layer, a RELU activation layer and a Dropout layer.

[0024] Preferably, the process of fusing the local functional features with the global dynamic features includes concatenating the local functional features with the global dynamic features to obtain fused features.

[0025] Preferably, the process of classifier fusing features includes:

[0026] The fused features are input into the first fully connected layer for dimensionality reduction to obtain the dimensionality reduction result; the dimensionality reduction result is input into the RELU activation layer to obtain the intermediate features; the intermediate features are input into the second fully connected layer for dimensionality reduction to obtain autism-related features; the autism-related features are input into the Sofmax layer for normalization to obtain the autism recognition results.

[0027] Preferably, the formula for calculating the total loss of the model is:

[0028]

[0029] Among them, L cls Represents the total loss of the model, N t represents the number of training samples, C represents the total number of classification categories, represents the true label of the i-th sample, Represents the predicted value of the i-th sample.

[0030] The beneficial effects of the present invention are as follows: the present invention uses the DMN model to partition the brain into functional areas, obtains the general brain function distribution pattern of a person, and ensures the independence of individual characteristics. By combining frequency features with graph neural networks, different brain activity states between autistic patients and normal people are obtained, which improves the generalization performance of the model and the recognition accuracy of the model, and can assist doctors in making the diagnosis of autistic patients more accurate and scientific. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a structural schematic diagram of the autism recognition model in the present invention. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] The present invention proposes an autism recognition method based on functional partitioning and graph convolutional neural network, and the method comprises the following contents:

[0034] Obtain brain imaging data and input it into the trained autism recognition model to obtain autism recognition results.

[0035] like Figure 1 As shown, the training process of the autism recognition model includes:

[0036] S1: Obtain autism brain imaging data, use the DMN model to partition the brain, and construct a local functional adjacency matrix and a global dynamic adjacency matrix based on the partitions.

[0037] Obtain autism brain imaging data. Preferably, the ABIDE 1 public dataset can be used as training data. The ABIDE1 dataset contains data from 17 research institutions and about 1,112 participants, including 539 autistic patients (ASD) and 573 healthy controls (TD). All subjects are divided according to the leave-one-out method, and all data are cross-validated to obtain training sets, validation sets, and test sets.

[0038] The data set is collected by cc200, with a total of 200 regions. The corresponding region on the data set is regarded as a functional area through the DMN partitioning mode. A functional area has only internal connections, so a local functional adjacency matrix is ​​constructed. At the same time, the global dynamic graph structure represents the connection between different functional areas. The internal connections of the graph structure only exist between different functional areas, so a global dynamic adjacency matrix is ​​constructed.

[0039] S2: Use the spatiotemporal feature extractor to process autism brain imaging data to obtain spatiotemporal features.

[0040] Autism brain imaging data is a time series data and has not been processed. Therefore, the original signal needs to be subjected to corresponding noise reduction processing; in addition, there is relevant temporal feature information and preliminary spatial information in the original signal that needs to be obtained. Therefore, the present invention uses a spatiotemporal feature extractor to input the original signal into the spatiotemporal attention layer to achieve the above two purposes, so as to ensure that more accurate feature information can be obtained in the subsequent frequency feature extraction.

[0041] The spatiotemporal feature extractor includes a channel attention layer and a spatial attention layer; the calculation formula of the channel attention layer is:

[0042] T (x) =σ(W1(W0(x ag ))+W1(W0(x max )))

[0043] The calculation formula of the spatial attention layer is:

[0044] S (x) =σ(f 7*7 ([x ag :x max ]))

[0045] Among them, T (x) represents the output of the channel attention layer, S (x) represents the output of the spatial attention layer, σ represents the sigmoid activation function, W1 and W0 represent the first and second training weight matrices, and x ag and x max Represents average pooling and maximum pooling, respectively, and f 7*7 Indicates a convolution with a convolution kernel size of 7*7.

[0046] S3: Send the spatiotemporal features to the frequency feature extractor to obtain frequency features.

[0047] Frequency feature information is more suitable for reflecting the movement state of the brain. Therefore, the spatiotemporal feature data obtained by the spatiotemporal attention layer is sent to the frequency feature extractor for feature extraction. The extractor mainly includes a wavelet convolution layer, which uses a continuous recursive convolution layer network instead of implementing the wavelet convolution algorithm. Through this layer, the frequency features of the five frequency bands of δ (1-3HZ), θ (4-7HZ), α (8-13HZ), β (14-30HZ) and γ (31-50HZ) are extracted respectively.

[0048] S4: Use a local function feature extractor to process the frequency features and the local function adjacency matrix to obtain local function features.

[0049] The extracted frequency features and local function adjacency matrix are sent to the local function feature extractor to ensure that the information on the impact of different functional areas on autism is correctly extracted. The local function feature extractor includes a graph convolution layer, a RELU activation layer, and a Dropout layer. The graph convolution layer uses the local function adjacency matrix to aggregate the output of the frequency feature extractor, and then performs a node information dimension reduction operation; the Dropout layer is used to process the reduced data to avoid the problems of gradient explosion and gradient disappearance; the RELU activation layer is used to enhance the nonlinear relationship of the model. The calculation process of graph convolution is as follows:

[0050] First, the Laplacian matrix L is calculated for the local function adjacency matrix:

[0051] L=DD -1 / 2 (I n +A)D -1 / 2

[0052] Where D represents the degree matrix, I n represents the identity matrix, and A is the local functional adjacency matrix. Using the Laplacian matrix for graph convolution operations involves eigenvalue decomposition, which is computationally intensive. Therefore, the Chebyshev polynomial is used to approximate the solution:

[0053] The Chebyshev convolution operation is as follows:

[0054]

[0055] Among them, σ represents the Sigmoid activation function, H (l) is the feature matrix of the lth layer, T k (L) is the approximation of Chebyshev polynomials, W k is the weight matrix of the kth polynomial coefficient, K is the polynomial coefficient, which controls the accuracy of the approximation.

[0056] S5: Use a global dynamic feature extractor to process the frequency features and the global dynamic adjacency matrix to obtain global dynamic features.

[0057] The global dynamic adjacency matrix is ​​set to a trainable weight matrix with a random initial value, and the adjacency matrix is ​​updated according to the back-propagation algorithm to dynamically adjust the connection relationship between each functional area. In this way, the global dynamic adjacency matrix can change according to the change of feature information, thereby ensuring the correct reflection of the global dynamics of the brain between different functional areas of autistic patients and normal people. The global dynamic feature extractor includes a graph dynamic convolution layer, a RELU activation layer, and a Dropout layer. The graph dynamic convolution layer uses the global dynamic adjacency matrix to aggregate the output of the frequency feature extractor, and then performs a node information dimension reduction operation; the Dropout layer is used to process the reduced data to avoid the problem of gradient explosion and gradient disappearance; the RELU activation layer is used to enhance the nonlinear relationship of the model. The graph dynamic convolution process is the same as the graph convolution process.

[0058] S6: Fuse the local functional features and the global dynamic features to obtain fused features.

[0059] Concatenating the local functional information with the global dynamic information can more comprehensively reflect the difference in brain information between autistic patients and normal people, and increase the performance of the model. Preferably, the present invention uses a Concate layer to perform feature fusion on the local functional features and the global dynamic features to obtain fused features.

[0060] S7: Input the fused features into the classifier to obtain the autism recognition result.

[0061] The process of the classifier on fusion features includes: inputting the fusion features into the first fully connected layer for dimensionality reduction to obtain the dimensionality reduction result; inputting the dimensionality reduction result into the RELU activation layer to obtain the intermediate features; inputting the intermediate features into the second fully connected layer for dimensionality reduction to obtain autism-related features; inputting the autism-related features into the Sofmax layer for normalization to obtain the probability value of whether the patient has autism, that is, the autism recognition result.

[0062] S8: Calculate the total model loss and adjust the model parameters according to the total model loss. When the pre-set number of training times is reached, the model training is completed.

[0063] The probability values ​​output by the model are classified to obtain the predicted labels, and the loss value is calculated by cross entropy with the true category, and back propagation is performed to update the model parameters; the classification loss, that is, the total model loss, is:

[0064]

[0065] Among them, L cls Represents the total loss of the model, N t represents the number of training samples, C represents the total number of classification categories, Indicates that the i-th sample is the true label of class c, Indicates the predicted value of the i-th sample being class c.

[0066] In summary, the present invention partitions the brain through the DMN model, constructs local functions and global dynamic graph structures; inputs the original data into the spatiotemporal attention extractor to obtain preliminary spatiotemporal features; inputs the spatiotemporal features into the wavelet convolution layer to obtain different frequency information; the frequency information and the local functions and the global dynamic graph structure are respectively sent to the graph convolution structure to obtain the local feature information and global dynamic feature information of the brain; the obtained features are fused and classified. The present invention reduces the data differences between different autistic patients by combining the brain partition method with the graph neural network, which can greatly improve the recognition effect.

[0067] The above embodiments further illustrate the purpose, technical solutions and advantages of the present invention in detail. It should be understood that the above embodiments are only preferred implementation modes of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made to the present invention within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for identifying autism based on functional partitioning and graph convolutional neural network, characterized in that: include: Obtain brain imaging data and input it into the trained autism recognition model to obtain autism recognition results; The training process of the autism recognition model includes: S1: Obtain autism brain imaging data, use the DMN model to partition the brain, and construct a local functional adjacency matrix and a global dynamic adjacency matrix based on the partitions; S2: Use the spatiotemporal feature extractor to process autism brain imaging data to obtain spatiotemporal features; S3: Send the spatiotemporal features to the frequency feature extractor to obtain frequency features; S4: using a local function feature extractor to process the frequency feature and the local function adjacency matrix to obtain a local function feature; S5: using a global dynamic feature extractor to process the frequency features and the global dynamic adjacency matrix to obtain global dynamic features; S6: Fusing local functional features and global dynamic features to obtain fused features; S7: Input the fused features into the classifier to obtain autism recognition results; S8: Calculate the total model loss and adjust the model parameters according to the total model loss. When the pre-set number of training times is reached, the model training is completed.

2. The autism identification method based on functional partitioning and graph convolutional neural network according to claim 1, characterized in that: The spatiotemporal feature extractor includes a channel attention layer and a spatial attention layer; The calculation formula of the channel attention layer is: T (x) =σ(W1(W0(x ag ))+W1(W0(x max ))) The calculation formula of the spatial attention layer is: S (x) =σ(f 7*7 ([x ag :x max ])) Among them, T (x) represents the output of the channel attention layer, S (x) represents the output of the spatial attention layer, σ represents the sigmoid activation function, W1 and W0 represent the first and second training weight matrices, and x ag and x max Represents average pooling and maximum pooling, respectively, and f 7*7 Indicates a convolution with a convolution kernel size of 7*7.

3. The autism identification method based on functional partitioning and graph convolutional neural network according to claim 1, characterized in that: The frequency feature extractor is a wavelet convolution layer, which utilizes a convolution recursive operation to simulate a wavelet function to extract frequency features from spatiotemporal features to obtain frequency features.

4. The autism identification method based on functional partitioning and graph convolutional neural network according to claim 1, characterized in that: The local function feature extractor includes a graph convolution layer, a RELU activation layer and a Dropout layer.

5. The autism identification method based on functional partitioning and graph convolutional neural network according to claim 1, characterized in that: The global dynamic feature extractor includes a graph dynamic convolution layer, a RELU activation layer and a Dropout layer.

6. The autism identification method based on functional partitioning and graph convolutional neural network according to claim 1, characterized in that: The process of fusing local functional features with global dynamic features includes concatenating local functional features with global dynamic features to obtain fused features.

7. The autism identification method based on functional partitioning and graph convolutional neural network according to claim 1, characterized in that: The process of classifier fusion features includes: The fused features are input into the first fully connected layer for dimensionality reduction to obtain the dimensionality reduction result; the dimensionality reduction result is input into the RELU activation layer to obtain the intermediate features; the intermediate features are input into the second fully connected layer for dimensionality reduction to obtain autism-related features; the autism-related features are input into the Sofmax layer for normalization to obtain the autism recognition results.

8. The autism identification method based on functional partitioning and graph convolutional neural network according to claim 1, characterized in that: The formula for calculating the total loss of the model is: Among them, L cls Represents the total loss of the model, N t represents the number of training samples, C represents the total number of classification categories, represents the true label of the i-th sample, Represents the predicted value of the i-th sample.

Images