ARLSTM-GCNs-based autism diagnosis method

Abstract

The invention discloses an autism diagnosis method based on ARLSTM-GCNs, and relates to a medical mode recognition diagnosis method.The autism spectrum disorder diagnosis method based on an improved AR-LSTM and a fused attention graph convolutional network-ARLSTM-GCNs comprises the steps that collected fMRI images are segmented into a plurality of ROIs through AAL, and BOLD sequence groups of all the ROIs are extracted; carrying out linear normalization processing on a time-frequency graph generated by carrying out wavelet transformation on the BOLD sequence, and estimating FCNs between sample ROIs for obtained two-dimensional sequence data by using Pearson's correlation; dividing the processed data set into a training set, a verification set and a test set; building an ARLSTM-GCNs diagnosis model, wherein the ARLSTM-GCNs diagnosis model comprises an improved AR-LSTM embedded model and an improved GCN-SAGPool model; and based on the constructed ARLSTM-GCNs model, executing a staged training task. According to the method, the autism can be accurately predicted.

Description

Technical Field

[0001] The present invention relates to a medical pattern recognition diagnosis method, and in particular to an autism diagnosis method based on ARLSTM-GCNs. Technical Background

[0002] Autism spectrum disorder (ASD) is a severe mental disorder characterized by social impairments, language impairments, and repetitive, stereotyped behaviors, with lifelong consequences. The average prevalence of ASD in Chinese children is 1.8% and is increasing annually. Internationally, the diagnosis of autism in children relies on behavioral symptoms, which is time-consuming and prone to misdiagnosis and missed diagnosis. Resting-state functional magnetic resonance imaging (rs-fMRI), as an objective neurological marker, has made significant contributions to the study of brain function and disease diagnosis. rs-fMRI measures blood oxygenation level-dependent (BOLD) changes in brain structures and analyzes BOLD changes and synchronized activity levels within regions of interest (ROIs) within brain networks, revealing crucial information about brain activity. Unlike organic neurological damage, the causes of ASD involve a complex interplay of multiple factors, including genetics and the environment. Consequently, the pathological factors displayed in imaging vary significantly between patients, posing a diagnostic challenge for medical professionals.

[0003] The importance of machine learning in ASD research is becoming increasingly prominent. It provides a powerful tool for processing and analyzing complex brain imaging data. The diagnosis of brain diseases based on fMRI usually adopts two technical routes: based on ROIs information or functional connection networks (FCNs) information of the brain. This single-mode analysis cannot fully capture the complex changes in brain activity in ASD patients. In recent years, researchers have attempted to fuse data from these two modalities. In order to obtain a more comprehensive understanding, GCNs provide a new method for the fusion analysis of local structure and functional connection features of the brain, and have achieved great success in the field of brain disease diagnosis. However, the performance of modal fusion classification models using GCNs in existing studies is generally lower than that of single-modal models. The reason is that GCN has complex modeling of time series data in ROIs, which makes it easy to learn noisy time series data during training, resulting in gradient explosion and overfitting problems. Summary of the Invention

[0004] The purpose of this invention is to propose an autism diagnosis method based on ARLSTM-GCNs, which is based on an improved LSTM and a self-attention graph convolutional network. This method combines an improved adaptive residual long short-term memory network (AR-LSTM) with an attention-integrated graph convolutional network (GCN-SAGPool) for diagnosing autism spectrum disorder (ASD)—ARLSTM-GCNs—and the method's segmented training technique. This method aims to improve the accuracy and efficiency of autism diagnosis by analyzing patients' multimodal physiological data.

[0005] The purpose of the present invention is achieved through the following technical solutions:

[0006] A method for diagnosing autism based on ARLSTM-GCNs, the method specifically comprising the following steps:

[0007] 1) Collect and screen the fMRI data required for the experiment;

[0008] 2) The fMRI images were segmented into multiple ROIs using the AAL automated anatomical template, and the blood oxygen level-dependent (BOLD) signal sequence groups of all ROIs were extracted;

[0009] 3) Perform wavelet transform on the BOLD sequence using the continuous wavelet transform method, and perform linear normalization on the generated time-frequency map to obtain the required two-dimensional sequence data. Then, use Pearson correlation to estimate the FCN between each sample ROI based on the two-dimensional sequence data;

[0010] 4) The processed sample ROI group and the corresponding FCN are used as the dataset and divided into training set, validation set and test set;

[0011] 5) Build an ARLSTM-GCNs network algorithm model as the main classification model, which consists of a multi-step training method;

[0012] 6) Based on the classification model, the subjects were diagnosed with ASD.

[0013] The autism diagnosis method based on ARLSTM-GCNs described herein collects and screens the fMRI data required for the experiment, collects sufficient and accurate data, and then screens out subjects with similar characteristics through the subject phenotypic information.

[0014] The ARLSTM-GCNs-based autism diagnosis method obtains the information required for diagnosis through data preprocessing;

[0015] The correlation coefficient between ROIs is estimated by Pearson correlation analysis method, and the correlation coefficient matrix R of the corresponding sample is obtained, which contains the correlation coefficients between all pairs of ROIs; for any pair of ROI two-dimensional sequences X and Y, the correlation coefficient r X,Y The calculation is as shown in Formula 1. Then, the obtained correlation coefficient matrix R is thresholded to obtain the adjacency matrix A, as shown in Formula 2:

[0016]

[0017]

[0018] in: and is the expectation of sequences X and Y, n is the sequence length, i and j are the sequence numbers of a pair of brain ROIs R(i,j), and A(i,j) is the connection information and correlation coefficient between the ROIs;

[0019] Use Morlet continuous wavelet transform on the ROI, and calculate the time-frequency information of the corresponding sequence f(t) according to the scale factor a, translation factor τ and Morlet wavelet basis function ψ(t), as shown in Formula 3;

[0020]

[0021] The time-frequency graph will be injected into AR-LSTM as a dataset for learning.

[0022] The method for diagnosing autism based on ARLSTM-GCNs constructs an ARLSTM-GCNs classification algorithm framework, which is composed of a cascade of AR-LSTM and GCNs networks; the input model data is a ROI wavelet time-frequency sequence group of a batch of samples and the FCNs information of the corresponding samples in this batch, and the output is the ARLSTM branch classification result and the GCNs branch classification result of the sample;

[0023] The AR-LSTM model based on the improved adaptive residual long short-term memory network is built. The model includes a layer normalization layer, an ARLSTM layer, and a self-normalization activation layer. The three layers are cascaded to form a basic AR-LSTM unit. The unit forward propagation process is shown in Formula 4-9.

[0024]

[0025]

[0026]

[0027]

[0028] Where: X is the linear normalized wavelet time-frequency map, F is the time-frequency matrix after layer normalization, γ and β are the layer normalization weight parameters; Equations 7 and 8 are the forward propagation formula of the AR-LSTM time unit and the definition of the output parameter matrix, where, is the output hidden vector of t time unit, f t is the hidden vector of F at time t, which is used as input parameter, h t-1 is the hidden parameter at time t-1, C t is the state parameter at time t, W R 、b R With W O 、b O is the weight bias parameter of ARLSTM adaptive residual connection and the weight bias parameter of output gate; Formula 9 is the self-normalization activation function, O is The activation output of , the scaling factor λ≈1.0507, the negative slope constant α≈1.6733;

[0029] Build a graph convolutional network GCN-SAGPool model that integrates the graph attention pooling mechanism. The model is a multi-layer cascade unit, each unit includes GCN and SAGPooling algorithms; the forward propagation formula for the first unit is shown in Figures 10-12:

[0030]

[0031]

[0032]

[0033] Among them: Among them, H (l) represents the node feature group of the lth layer, W (l) is the weight matrix of the lth layer, is the adjacency matrix with self-loop added in Equation 3, yes The degree matrix, σ(·) is the sigmoid activation function; the attention score matrix Corresponding node feature H( l) For each node in is the weight of the SAGPool layer, in which the attention score is calculated by the attention weight Calculated; based on this score, use Topk pooling to retain k node combinations with higher scores Connection with it Reduce the complexity of the graph.

[0034] The method for diagnosing autism based on ARLSTM-GCNs is based on the constructed ARLSTM-GCNs model and performs the following steps: Figure 2 The algorithm segmented training method and application method shown in the figure are completed through two-step training of ARLSTM training and GCNs training. The training adopts 5-fold cross validation. The learning rate change in the ARLSTM training process uses the cosine annealing strategy, and ASD diagnosis is achieved through forward propagation calculation in the diagnostic application.

[0035] The advantages and effects of the present invention are:

[0036] The present invention solves the problem that traditional GCN cannot express time series, and the gradient smoothing problem caused by traditional LSTM after multi-layer cascade. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 ARLSTM-GCNs algorithm framework;

[0038] Figure 2 Improve the AR-LSTM algorithm structure;

[0039] Figure 3 Model training and diagnostic application methods;

[0040] Figure 4 GCN-SAGPool model structure. DETAILED DESCRIPTION

[0041] To clearly illustrate the purpose, technical details, and effective applications of the present invention and to facilitate understanding and implementation by those skilled in the art, the present invention will be further described in detail below in conjunction with the embodiments and accompanying drawings. Obviously, the embodiments described herein are only for the purpose of illustrating and explaining the present invention and are not intended to limit the present invention.

[0042] The present invention provides an autism diagnosis method based on ARLSTM-GCNs, which involves an overall algorithm framework, an improved AR-LSTM algorithm structure, and an appendix. Figure 1 、 2 Model training and diagnostic application methods Figure 3 The following steps are included:

[0043] Collect and screen fMRI data required for the experiment;

[0044] The fMRI images were segmented into multiple ROIs using the AAL automatic anatomical template, and the BOLD sequence groups of all ROIs were extracted.

[0045] The BOLD sequence is subjected to wavelet transform by continuous wavelet transform method, and the generated time-frequency graph is linearly normalized to obtain the required two-dimensional sequence data. Based on the two-dimensional sequence data, the Pearson correlation is used to estimate the FCN between each sample ROI; the data set is divided into training set, validation set and test set, and a 5-fold cross-validation training strategy is used for the training set. The AR-LSTM model is trained, and the model output encoding is classified by a multi-layer perceptron. The optimization step size is adjusted using cosine annealing, and the output loss is evaluated using the cross-entropy loss function. After the AR-LSTM model is trained, the AR-LSTM network parameters are frozen and the multi-layer perceptron is removed. The time-frequency graph node encoding is output. The graph node encoding is combined with the FCN data into graph data and injected into the GCN-SAGPool model for training (the GCN-SAGPool model structure is attached). Figure 4 ), the output codes are classified using a multilayer perceptron, and the cross-entropy loss function is used for loss evaluation. GCN-SAGPool training is now complete, and ARLSTM-GCNs training is now complete. A trained ARLSTM-GCNs autism diagnosis model is obtained. This model can be used for autism diagnosis and risk screening in suspected patients or subjects.

[0046] Example

[0047] The present invention provides an autism diagnosis method based on an ARLSTM-GCNs network, which includes the following steps:

[0048] 1. Collect and screen the fMRI data required for the experiment;

[0049] 2. Segment the fMRI image into multiple ROIs using the AAL automated anatomical template and extract the BOLD sequence groups of all ROIs

[0050] 3. Perform wavelet transform on the BOLD sequence using the continuous wavelet transform method, and perform linear normalization on the generated time-frequency map to obtain the required two-dimensional sequence data. Then, use Pearson correlation to estimate the FCN between each sample ROI based on the two-dimensional sequence data;

[0051] 4. The processed sample ROI group and the corresponding FCN are used as the data set and divided into training set, validation set and test set;

[0052] 5. Build an ARLSTM-GCNs network algorithm model as the main classification model. The algorithm model consists of a multi-step training method;

[0053] 6. Based on the classification model, the subjects were diagnosed with ASD.

[0054] The specific implementation steps of the ASD diagnosis method provided by the present invention are as follows:

[0055] Step 1:

[0056] The collected data comes from the open database ABIDE, which contains raw fMRI data.

[0057] Step 2:

[0058] The fMRI images were segmented into 116 ROIs using the AAL automated anatomical template, and the BOLD signal sequence of each ROI was extracted.

[0059] Step 3:

[0060] The correlation coefficient between ROIs is estimated by Pearson correlation analysis method, and the correlation coefficient matrix R of the corresponding sample is obtained, which contains the correlation coefficients between all pairs of ROIs. For any pair of ROI two-dimensional sequences X and Y, the correlation coefficient r X,Y The calculation is as shown in Formula 1. Then the obtained correlation coefficient matrix R is thresholded to obtain the adjacency matrix A, as shown in Formula 2:

[0061]

[0062]

[0063] in: and is the expectation of sequences X and Y, n is the sequence length, i and j are the sequence numbers of a pair of brain ROIs R(i,j), and A(i,j) is the connection information and correlation coefficient between the ROIs.

[0064] The Morlet continuous wavelet transform is used on the ROI. According to the scale factor a, the translation factor τ and the Morlet wavelet basis function ψ(t), the time-frequency information of the corresponding sequence f(t) is calculated, as shown in Formula 3.

[0065]

[0066] The time-frequency graph will be injected into AR-LSTM as a dataset for learning.

[0067] Step 4:

[0068] Construct an ARLSTM-GCNs classification algorithm framework, consisting of a cascade of AR-LSTM and GCNs networks. The input model data is a batch of ROI wavelet time-frequency sequences and the FCNs information of the corresponding samples in this batch. The output is the classification results of the ARLSTM branch and the GCNs branch for that sample.

[0069] The AR-LSTM model is based on an improved adaptive residual long short-term memory network. The model includes a layer normalization layer, an ARLSTM layer, and a self-normalization activation layer. These three layers are cascaded to form a basic AR-LSTM unit. The forward propagation process of the unit is shown in Figure 4-9.

[0070]

[0071]

[0072]

[0073]

[0074] Where: X is the linear normalized wavelet time-frequency map, F is the time-frequency matrix after layer normalization, γ and β are the layer normalization weight parameters; Equations 7 and 8 are the forward propagation formula of the AR-LSTM time unit and the definition of the output parameter matrix, where, is the output hidden vector of t time unit, f t is the hidden vector of F at time t, which is used as input parameter, h t-1 is the hidden parameter at time t-1, C t is the state parameter at time t, W R 、b R With W O 、b O is the weight bias parameter of ARLSTM adaptive residual connection and the weight bias parameter of output gate; Formula 9 is the self-normalization activation function, O is The activation output of , the scaling factor λ≈1.0507, and the negative slope constant α≈1.6733.

[0075] Build a graph convolutional network (GCN)-SAGPoool model that integrates a graph attention pooling mechanism. The model consists of multiple cascaded units, each of which includes the GCN and SAGPooling algorithms. The forward propagation formula for the first unit is shown in Figures 10-12:

[0076]

[0077]

[0078]

[0079] Among them: Among them, H (l) represents the node feature group of the lth layer, W (l) is the weight matrix of the lth layer, is the adjacency matrix with self-loop added in Equation 3, yes The degree matrix of , σ(·) is the sigmoid activation function. The attention score matrix Corresponding node feature H (l) For each node in is the weight of the SAGPool layer, in which the attention score is calculated by the attention weight Based on this score, Topk pooling is used to retain k node combinations with higher scores Connection with it Reduce the complexity of the graph.

[0080] Step 5:

[0081] Based on the constructed ARLSTM-GCNs model, perform phased training tasks.

Claims

1. An autism diagnosis method based on ARLSTM-GCNs, characterized in that: The method specifically comprises the following steps: 1) Collect and screen the fMRI data required for the experiment; 2) The fMRI images were segmented into multiple ROIs using the AAL automatic anatomical template, and the blood oxygen level dependent (BOLD) signal sequence groups of all ROIs were extracted; 3) The BOLD sequence is subjected to wavelet transformation by continuous wavelet transform method, and the generated time-frequency diagram is subjected to linear normalization to obtain the required two-dimensional sequence data; then, the FCN between each sample ROI is estimated using Pearson correlation based on the two-dimensional sequence data; 4) The processed sample ROI group and the corresponding FCN are used as the data set and divided into training set, validation set and test set; 5) Build an ARLSTM-GCNs network algorithm model as the main classification model, which consists of a multi-step training method; 6) Based on the classification model, the subjects were diagnosed with ASD.

2. The autism diagnosis method based on ARLSTM-GCNs according to claim 1, characterized in that: The fMRI data required for the experiment should be collected and screened, and sufficient and accurate data should be collected, and then subjects with similar characteristics should be screened out through the subject phenotypic information.

3. The autism diagnosis method based on ARLSTM-GCNs according to claim 1, characterized in that: The method obtains information required for diagnosis through data preprocessing; The correlation coefficient between ROIs is estimated by Pearson correlation analysis method, and the correlation coefficient matrix R of the corresponding samples is obtained, which contains the correlation coefficients between all pairs of ROIs; for any pair of ROI two-dimensional sequences X and Y, the correlation coefficient r X,Y The calculation is as shown in Formula 1; then the obtained correlation coefficient matrix R is thresholded to obtain the adjacency matrix A, as shown in Formula 2: in: and is the expectation of the sequence X, Y, n is the sequence length, where i, j is a pair of brain ROI numbers R(i,j), A(i,j) is the connection information and correlation coefficient between this pair of ROIs; Use Morlet continuous wavelet transform on ROI, and calculate the time-frequency information of the corresponding sequence f(t) according to the scale factor a, translation factor τ and morlet wavelet basis function ψ(t), as shown in formula 3; The time-frequency graph will be injected into AR-LSTM as a dataset for learning.

4. The autism diagnosis method based on ARLSTM-GCNs according to claim 1, characterized in that: The method constructs an ARLSTM-GCNs classification algorithm framework, which is composed of a cascade of AR-LSTM and GCNs networks; the input model data is a ROI wavelet time-frequency sequence group of a batch of samples and the FCNs information of the corresponding samples in this batch, and the output is the ARLSTM branch classification result and the GCNs branch classification result of the sample; Build an AR-LSTM model based on the improved adaptive residual long short-term memory network. The model includes a layer normalization layer, an ARLSTM layer, and a self-normalization activation layer. The three layers are cascaded to form a basic AR-LSTM unit. The unit forward propagation process is shown in Formula 4-9. Where: X is the linear normalized wavelet time-frequency diagram, F is the time-frequency matrix after layer normalization, γ and β are the layer normalization weight parameters; Formula 7 and 8 are the forward propagation formula of AR-LSTM time unit and the definition of output parameter matrix, where, is the output hidden vector of the t-time unit, f t is the hidden vector of F at time t, which is used as an input parameter, h t-1 is the hidden parameter at time t-1, C t is the state parameter at time t, W R , b R With W O , b O is the weight bias parameter of the ARLSTM adaptive residual connection and the weight bias parameter of the output gate; Formula 9 is the self-normalization activation function, and O is The activation output of , the scaling factor λ≈1.0507, the negative slope constant α≈1.6733; Build a graph convolutional network GCN-SAGPool model that integrates the graph attention pooling mechanism. The model is a multi-layer cascade unit, each unit includes GCN and SAGPooling algorithms; the forward propagation formula for the lth unit is shown in 10-12: Among them: Among them, H (l) represents the node feature group of the lth layer, W (l) is the weight matrix of the lth layer, is the adjacency matrix with self-loop added to Equation 3, yes The degree matrix, σ(·) is the sigmoid activation function; the attention score matrix Corresponding node feature H (l) For each node in is the weight of the SAGPool layer, in which the attention score is calculated by the attention weight Calculated; based on this score, use Topk pooling to retain k node combinations with higher scores Connect with it Reduce the complexity of the graph.

5. The autism diagnosis method based on ARLSTM-GCNs according to claim 1, characterized in that: The method is based on the constructed ARLSTM-GCNs model, and executes the algorithm segmented training method and application method as shown in Figure 2; the method is completed through two-step training of ARLSTM training and GCNs training, and the training adopts 5-fold cross validation, in which the learning rate change in the ARLSTM training process uses a cosine annealing strategy, and ASD diagnosis is achieved through forward propagation calculation in diagnostic applications.

Images