Data classification method and system based on deep multipath attention adaptive graph convolutional network

Abstract

The invention discloses a data classification method and system based on a deep multi-path attention adaptive graph convolutional network, and relates to the technical field of medical image processing. Rs-fMRI data is acquired and preprocessed to obtain a BOLD sequence, a function connection feature vector is constructed and input into a DMAGCN model, and finally an optimal model is acquired through five-fold cross validation for classification. The data quality is guaranteed through data preprocessing, and a foundation is laid for accurate analysis. Functional connection feature vectors effectively represent data, after a model is input, a Transform backbone network and an MLP branch network can extract multi-source domain features, non-imaging data are utilized in combination with a graph network, the model can learn rich features, and generalization and adaptability are enhanced. According to the method, the classification accuracy is improved, the performance is excellent when uncertain data classification such as autism spectrum disorder is processed, related disease research is promoted, and an efficient and reliable method is provided for medical data classification.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing technology, and more specifically to a data classification method and system based on a deep multi-path attention adaptive graph convolutional network. Background Art

[0002] At present, Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder in children, with clinical manifestations of stereotyped behavior, narrow interests, and communication disorders. If timely intervention can be carried out in the early stages of autism, the effect will be better. However, some ASD patients are not discovered in childhood, but symptoms are observed in adolescence. Therefore, early prevention of autism patients is very necessary.

[0003] Over the past few decades, and in the past decade, advances in neuroimaging technology have provided a critical step forward and made it possible to measure functional and structural changes associated with ASD. Several methods based on resting-state functional magnetic resonance imaging (rs-fMRI) have been proposed for the diagnosis of various brain disorders and have gained widespread attention in the discovery and classification of ASD biomarkers. It is a key tool that can reveal brain dysfunction based on the blood oxygen level-dependent (BOLD) signal when the subject is at rest. The Pearson correlation coefficient is calculated as functional connectivity by using the average BOLD signal of two brain regions in rs-fMR. It has been widely used in computer-assisted ASD diagnosis due to its effectiveness in brain functional organization and biomarkers of neuropsychiatric diseases. Most rs-fMRI-based ASD classification methods are developed using functional connectivity as features.

[0004] Deep learning has been successfully applied to disease diagnosis and has achieved amazing results. However, human neural activity is characterized by uncertainty. This is because the process of collecting rs-fMRI data in clinical practice is interfered by the noise of the human body and the equipment, making it difficult to reflect the real neural activity. Secondly, ASD is highly heterogeneous, and there are obvious individual differences in the neural activity of different ASD patients. These factors make rs-fMRI data contain obvious uncertainty, which is not conducive to building a robust ASD auxiliary diagnosis model. At the same time, clinical neural image datasets often face the problem of small dataset size due to their expensive acquisition and time-consuming labeling. Therefore, in some studies, such as ASD diagnosis, multi-site rs-fMRI data are often merged to expand the dataset, which leads to the second problem: in most cases, samples from different scanners or acquisition protocols do not follow the same distribution. Sun et al. minimized domain shift by aligning the second-order statistics of the source and target domain distributions; Liu et al. alleviated the marginal distribution differences between domains by adjusting the global structure of the predicted multi-site data; Wang et al. aimed to reduce the differences in data distribution by determining a common low-rank representation of data from multiple sites; Eslami et al. used autoencoders and single-layer perceptrons (SLP) to improve the quality of extracted features and optimize the parameters of the model. These methods are effective in solving the problem of inconsistent feature distribution of data sets, but the processing of features is not detailed enough, which may lead to misdiagnosis, which is very serious in the medical field.

[0005] Therefore, how to handle the classification of uncertain data is an urgent problem that technical personnel in this field need to solve. Summary of the invention

[0006] In view of this, the present invention provides a data classification method and system based on a deep multi-path attention adaptive graph convolutional network, and proposes a new deep neural network - DMAGCN, which can handle uncertain data classification; it includes a backbone network and multiple branch networks, the backbone network is constructed by Transformer, and common features are extracted for different source domains, the branch networks are constructed by MLP and GCN, and the number is the same as the number of source domains, this part extracts the unique features of the source domain, and GCN combines image data and non-image data by constructing a population map, and extracts multimodal information for model training; in the feature classification stage, the features obtained from each branch are connected, and the final classification result is obtained using the softmax function, thereby realizing the classification of uncertain data.

[0007] In order to achieve the above object, the present invention adopts the following technical solution:

[0008] A data classification method based on a deep multi-path attention adaptive graph convolutional network, comprising:

[0009] Resting-state functional magnetic resonance imaging data were obtained from the Autism Brain Imaging Data Exchange database and preprocessed using a configurable pipeline for connectome analysis to obtain BOLD sequences;

[0010] The functional feature matrix was constructed using the BOLD sequence, and the upper triangular part of the functional feature matrix was straightened to obtain the functional connectivity feature vector as the input of the DMAGCN model.

[0011] Construct a DMAGCN model, which consists of a backbone network and multiple branch networks. The backbone network is built with Transformer, and the branch networks are composed of MLP and graph networks respectively. The backbone network is used to extract the common features of all source domains, and the branch network is used to extract the unique features of a single source domain.

[0012] The DMAGCN model was verified based on the five-fold cross-validation method to obtain the optimal DMAGCN model;

[0013] Input the data to be classified into the optimal DMAGCN model to obtain the data classification results.

[0014] Optionally, the preprocessing involves skull delineation, slice timing correction, motion correction, global mean intensity normalization, noise signal regression, bandpass filtering 0.01–0.1 Hz, and registration of resting-state functional magnetic resonance imaging data to a standard anatomical space.

[0015] Optionally, the calculation formula of the functional connectivity feature vector is as follows:

[0016]

[0017] Among them, X i and X j is the average time series of the i-th ROI region and the j-th ROI region; X i,t ,X j,t Yes X i and X j BOLD intensity at time instant t; and represents the mean of the average BOLD time series of the ith brain region and the jth brain region, respectively, T represents the total number of time points of the average BOLD series, and corr represents the Pearson correlation coefficient.

[0018] Optionally, the backbone network built with MLP and Transformer extracts features from imaging data. In addition to imaging features, the graph network uses non-imaging data as edges between nodes to provide multimodal information to train the DMAGCN model. When the training is completed, the graph network will be discarded and other networks will jointly make decisions on the target domain.

[0019] Optionally, the backbone network uses Transformer as the backbone network, and the overall structure of Transformer includes a multi-head self-attention module, a feedforward network, a residual connectivity layer and a normalization layer;

[0020] The self-attention mechanism is the core of the Transformer-encoder and is calculated from the Query, Key, and Value matrices:

[0021]

[0022] in N and M represent the length of Query and Key, D k and D v Represents the dimensions of Key and Value; Softmax is an activation function that converts attention scores into probabilities; the multi-head attention mechanism is applied in Transformer:

[0023]

[0024] in, are the parameter matrices corresponding to Q, K, and V respectively; W 0 is the parameter matrix for multi-head attention calculation; the feedforward network then applies two linear transformations with Gelu activation function to the output of the multi-head self-attention:

[0025] X=FFN(x)=Gelu(xW1+b1)W2+b2;

[0026] Among them, x is the output of the previous layer, W1, W2, b1 and b2 represent the training parameter matrix and bias value.

[0027] Optionally, in order to complete the feature extraction of multiple tasks, multiple branch networks are set up to extract the unique features of each source domain. Represents the input features, which consists of fully connected layers. Each layer has multiple nodes. Each node receives the output of the previous layer node as input. The output of the kth node in the lth layer is:

[0028]

[0029] Among them, k (l)is the output of the lth layer, l=1,2,...,L;k=1,2,...,K;j=1,2,...,J;k≠j; is the feature of the kth node in the l-1th layer; θ jk (l) represents the connection weight between the kth nodes in the lth layer; α l represents the activation function of the lth layer;

[0030] Use the backbone network to extract common features from all source domains:

[0031]

[0032] Use the branch network to extract the unique features of the i-th source domain:

[0033]

[0034] Where i=1,2,3...I represents the i-th source domain sample or target domain sample; s is the source domain dataset; f c,i (·) is the extractor of common features; f s,i (·) is the extractor of unique features; Θ s,i Parameters of the i-th source domain-specific feature extractor; Θ c,i represents the parameters of the i-th common feature extractor; represents the common features of the i-th source domain sample in the source domain; represents the unique features of the i-th source domain sample.

[0035] Optionally, the maximum mean difference is used to measure the distribution distance between the common features and unique features related to the source domain, specifically:

[0036]

[0037] Where n is the number of samples in the source domain; i and j represent the sample numbers, i≠j; represents the Gaussian kernel Hilbert space.

[0038] The domain alignment loss is:

[0039] L domain =L com +L s +L cs .

[0040] Optionally, the graph network is specifically an edge variational graph convolutional network, which uses the spatial perception of the brain network and the population relationship of the data set to train the optimization model;

[0041] Given N subjects’ data consisting of imaging and non-imaging data, we construct a population graph: G = (V, E, W), where |V| = N represents the set of vertices. is a set of edges, where the weight of the edge is W; define the node characteristics As the C-dimensional feature vector extracted from the imaging data of the i-th subject; (x i ,x j ) i,j ∈W is defined as a learnable function that represents the information of non-imaging data: φ:(x i ,x j ), which is modeled and trained by the pairwise association encoder PAE:

[0042] h i =φ(x i ,Ω);

[0043] h j =φ(x j ,Ω);

[0044]

[0045] Among them, x% i is the normalized input; τ is the ReLU function; h i and h j is the input feature x i and x j Mapping in the same feature space; Ω represents the parameters trained in PAE;

[0046] The uncertainty-aware prediction loss function is:

[0047]

[0048] Among them, P(x i ) represents the predicted value of the i-th sample, P%(x i ) is the true value of the i-th sample;

[0049] Therefore, the total loss function of the DMAGCN model is:

[0050]

[0051] L=λL domain +L ev ;

[0052] Where λ changes from 0 to 1 over time, γ is a hyperparameter, and ρ represents the number of iterations.

[0053] Optionally, the convolutional layer of the graph convolution is composed of Chebyshev convolution, and the recurrence relationship of Chebyshev polynomials is:

[0054] T0(L)=1,T1(L)=L;

[0055] T k (L)=2LT k-1 (L)-T k-2 (L);

[0056] The formula for Chebyshev convolution is:

[0057]

[0058] Among them, T k (L) represents the topological structure L of the graph G after being calculated by Chebyshev polynomials in the kth term; H l represents the node feature vector of the lth layer; θ k l is the convolution kernel parameter.

[0059] A data classification system based on deep multi-path attention adaptive graph convolutional network, comprising:

[0060] The data acquisition and processing module obtains resting-state functional magnetic resonance imaging data from the Autism Brain Imaging Data Exchange database and preprocesses them using a configurable pipeline for connectome analysis to obtain BOLD sequences;

[0061] The feature extraction module uses the BOLD sequence to construct a functional feature matrix, and straightens the upper triangular part of the functional feature matrix to obtain the functional connection feature vector as the input of the DMAGCN model;

[0062] Model building module, building DMAGCN model, the model consists of a backbone network and multiple branch networks. The backbone network is built with Transformer, and the branch networks are composed of MLP and graph networks respectively. The backbone network is used to extract the common features of all source domains, and the branch network is used to extract the unique features of a single source domain.

[0063] Model validation module, which validates the DMAGCN model based on the five-fold cross validation method to obtain the optimal DMAGCN model;

[0064] The result output module inputs the data to be classified into the optimal DMAGCN model to obtain the data classification results.

[0065] It can be known from the above technical solutions that compared with the prior art, the present invention discloses a data classification method and system based on a deep multi-path attention adaptive graph convolutional network. First, rs-fMRI data is obtained and pre-processed from the ABIDE database to obtain a BOLD sequence, and a functional connection feature vector is constructed to input into the DMAGCN model, which is composed of a specific structure network. Finally, the optimal model is obtained through five-fold cross validation for classification. The present invention ensures data quality through data preprocessing and lays the foundation for accurate analysis. The functional connection feature vector effectively represents the data. After inputting the model, the Transformer backbone network and the MLP branch network can extract multi-source domain features. Combined with the graph network to utilize non-imaging data, the model can learn rich features and enhance generalization and adaptability. Five-fold cross validation ensures model reliability and improves classification accuracy, especially when dealing with uncertain data classification such as autism spectrum disorder, promotes related disease research, and provides an efficient and reliable method for medical data classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0067] Figure 1 A schematic diagram of the process of constructing a functional connection feature vector provided by the present invention;

[0068] Figure 2 The overall structure diagram of the DMAGCN model provided by the present invention;

[0069] Figure 3 This is a diagram of the trunk and branch network structure provided by the present invention. DETAILED DESCRIPTION

[0070] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0071] The embodiment of the present invention discloses a data classification method based on a deep multi-path attention adaptive graph convolutional network, comprising:

[0072] Resting-state functional magnetic resonance imaging data were obtained from the Autism Brain Imaging Data Exchange database and preprocessed using a configurable pipeline for connectome analysis to obtain BOLD sequences;

[0073] The functional feature matrix was constructed using the BOLD sequence, and the upper triangular part of the functional feature matrix was straightened to obtain the functional connectivity feature vector as the input of the DMAGCN model.

[0074] Construct a DMAGCN model, which consists of a backbone network and multiple branch networks. The backbone network is built with Transformer, and the branch networks are composed of MLP and graph networks respectively. The backbone network is used to extract the common features of all source domains, and the branch network is used to extract the unique features of a single source domain.

[0075] The DMAGCN model was verified based on the five-fold cross-validation method to obtain the optimal DMAGCN model;

[0076] Input the data to be classified into the optimal DMAGCN model to obtain the data classification results.

[0077] In a specific embodiment, the data set is provided by the Autism Brain Imaging Data Exchange (ABIDE) and has been preprocessed using a configurable pipeline for connectome analysis (C-PAC). The acquisition device is Siemens, and the process involves skull striping, slice timing correction, motion correction, global mean intensity normalization, interference signal regression, bandpass filtering (0.01-0.1Hz), and fMRI image registration to standard anatomical space (MNI152). The data set used comes from six acquisition points including N=1735 subjects. Specific statistical information is shown in the following table:

[0078] Table 1 Dataset information

[0079]

[0080]

[0081] Explanation of parameters in the table:

[0082] Repetition Time (TR): The time interval between successive pulse trains applied to the same slice.

[0083] Echo Time (TE): is the time between the peak of the transmitted and received pulses.

[0084] Short TR and short TE enhance T1-weighted contrast, making tissue with short T1 delay times (such as fat) appear bright, while tissue with longer T1 times (such as fluid) appear dim.

[0085] Longer TR and TE values ​​produce T2-weighted images in which tissues with longer T2 relaxation times (e.g., fluids) appear bright. Longer TR ensures complete longitudinal relaxation, while longer TE allows for adequate lateral dephasing of T2 contrast.

[0086] Voxel size: affects image resolution. The smaller the voxel, the higher the resolution. It also affects the results of data analysis. Too large a voxel may not be able to capture subtle differences between brain regions.

[0087] In a specific embodiment, the original feature extraction is as follows Figure 1 As shown in the figure, for each sample, the Pearson correlation coefficient between any pair of ROI regions of each brain is calculated to obtain the final functional connectivity feature matrix. Due to the symmetry of the matrix, only the upper triangle of the matrix is ​​selected, and this part is stretched into a one-dimensional vector as the functional connectivity feature vector. The calculation formula of the functional connectivity feature vector is as follows:

[0088] The calculation formula of functional connectivity feature vector is as follows:

[0089]

[0090] Among them, X i and X j is the average time series of the i-th ROI region and the j-th ROI region; X i,t ,X j,t Yes X i and X j BOLD intensity at time instant t; and represents the mean of the average BOLD time series of the ith brain region and the jth brain region, respectively, T represents the total number of time points of the average BOLD series, and corr represents the Pearson correlation coefficient.

[0091] In a specific embodiment, the whole process flow is as follows Figure 2As shown in the figure, it includes four parts: data preprocessing, functional connection feature extraction, extraction of mixed features and classification. Specifically, the rs-fMRI data is first preprocessed to obtain the BOLD sequence, and then the functional feature matrix is ​​constructed using the BOLD sequence, and the upper triangular part of the matrix is ​​straightened to obtain the functional connection feature vector. Because the data sets involved are from different imaging centers, this is a multi-source domain problem. The model is composed of a backbone network plus multiple branch networks. The backbone network is built with Transformer, and the branch networks are composed of MLP and graph networks respectively. The network built by MLP and Transformer mainly extracts features from imaging data. In addition to imaging features, the graph network uses non-imaging information as edges between nodes to provide multimodal information to train the model. The backbone network is used to extract common features of all source domains, and the branch network is used to extract unique features of a single source domain. When the training is completed, the graph network will be discarded, and other networks will jointly make decisions on the target domain.

[0092] The branch network structure is as follows Figure 3 As shown in the figure, it consists of a fully connected layer. The input features are the output of the Transformer structure. The data will be reduced in dimension in the deep neural network. The neural network will learn the feature information from it and extract key features for subsequent model optimization and classification.

[0093] In a specific embodiment, the backbone network uses Transformer as the backbone network, and the overall structure of Transformer includes a multi-head self-attention module, a feedforward network, a residual connectivity layer and a normalization layer;

[0094] The self-attention mechanism is the core of the Transformer-encoder and is calculated from the Query, Key, and Value matrices:

[0095]

[0096] in N and M represent the length of Query and Key, D k and D v Represents the dimensions of Key and Value; Softmax is an activation function that converts attention scores into probabilities; the multi-head attention mechanism is applied in Transformer:

[0097]

[0098] in, are the parameter matrices corresponding to Q, K, and V respectively; W 0is the parameter matrix for multi-head attention calculation; the feedforward network then applies two linear transformations with Gelu activation function to the output of the multi-head self-attention:

[0099] X=FFN(x)=Gelu(xW1+b1)W2+b2;

[0100] Among them, x is the output of the previous layer, W1, W2, b1 and b2 represent the training parameter matrix and bias value.

[0101] In order to complete the feature extraction of multiple tasks, multiple branch networks are set up to extract the unique features of each source domain. Represents the input features, which consists of fully connected layers. Each layer has multiple nodes. Each node receives the output of the previous layer node as input. The output of the kth node in the lth layer is:

[0102]

[0103] Among them, k (l) is the output of the lth layer, l=1,2,...,L;k=1,2,...,K;j=1,2,...,J;k≠j;oj ( l- 1) is the feature of the kth node in the l-1th layer; θ jk (l) represents the connection weight between the kth nodes in the lth layer; α l represents the activation function of the lth layer;

[0104] Use the backbone network to extract common features from all source domains:

[0105]

[0106] Use the branch network to extract the unique features of the i-th source domain:

[0107]

[0108] Where i=1,2,3...I represents the i-th source domain sample or target domain sample; s is the source domain dataset; f c,i (·) is the extractor of common features; f s,i (·) is the extractor of unique features; Θ s,i Parameters of the i-th source domain-specific feature extractor; Θ c,i represents the parameters of the i-th common feature extractor; represents the common features of the i-th source domain sample in the source domain; represents the unique features of the i-th source domain sample.

[0109] Maximum mean difference (MMD) is the most widely used loss function in transfer learning, especially in domain adaptation. It is mainly used to measure the distance between the distributions of two different but related random variables. Here, MMD is used to measure the distance between the distributions of common and unique features related to the source domain. Specifically:

[0110]

[0111] Where n is the number of samples in the source domain; i and j represent the sample numbers, i≠j; represents the Gaussian kernel Hilbert space.

[0112] The domain alignment loss is:

[0113] L domain =L com +L s +L cs .

[0114] In a specific embodiment, uncertainty-aware population graph construction, the graph network is specifically an edge variational graph convolutional network, which provides a method for constructing an adaptive population graph with partially labeled nodes and variational edges, integrating imaging data with non-imaging data in the population for uncertainty-aware disease prediction, and has proven its effectiveness. This model uses it as one of its branches, aiming to use the spatial perception of the brain network and the population relationship of the data set to train the optimization model.

[0115] Given N subjects’ data consisting of imaging and non-imaging data, we construct a population graph: G = (V, E, W), where |V| = N represents the set of vertices. is a set of edges, where the weight of the edge is W; define the node characteristics As the C-dimensional feature vector extracted from the imaging data of the i-th subject; (x i ,x j ) i,j ∈W is defined as a learnable function that represents the information of non-imaging data: φ:(x i ,x j ), which is modeled and trained by the pairwise association encoder PAE:

[0116] h i =φ(x i ,Ω);

[0117] h j =φ(x j ,Ω);

[0118]

[0119] Among them, x% i is the normalized input; τ is the ReLU function; h i and h j is the input feature x i and x j Mapping in the same feature space; Ω represents the parameters of the mapping network φ(·);

[0120] The convolution layer of graph convolution is composed of Chebyshev convolution, and the recurrence relationship of Chebyshev polynomial is:

[0121] T0(L)=1,T1(L)=L;

[0122] T k (L)=2LT k-1 (L)-T k-2 (L);

[0123] The formula for Chebyshev convolution is:

[0124]

[0125] Among them, T k (L) represents the topological structure L of the graph G after being calculated by Chebyshev polynomials in the kth term; H l Represents the node feature vector of the lth layer; is the convolution kernel parameter.

[0126] The uncertainty-aware prediction loss function is:

[0127]

[0128] Among them, P(x i ) represents the predicted value of the i-th sample, is the true value of the i-th sample;

[0129] Therefore, the total loss function of the DMAGCN model is:

[0130]

[0131] Where λ changes from 0 to 1 over time, γ is a hyperparameter, and ρ represents the number of iterations.

[0132] In a specific embodiment, experimental results and analysis are also included;

[0133] Experimental setup

[0134] The experimental data comes from the ABIDE database. A new dataset is constructed using data from six sites, including NYU, USM, UM, Leuven, YALE, and UCLA. The experiment is based on five-fold cross validation, which is a validation scheme commonly used in many studies. In the experiment, one of the six imaging centers is selected as the target domain, and the rest are selected as the source domain. Each source domain is divided into five subsets (the number of samples in each subset is similar, and the number of ASD patients and healthy controls in the subset is also roughly the same). In each fold of the cross-validation, four subsets are selected from each independent source domain as labeled source domain training sets, and the target domain data of all five subsets are selected as unlabeled target domain training sets. The target domain label is not used in training, and the target domain label is only used in testing to evaluate the classification performance of the model on the target domain.

[0135] The experiment ensures that each comparison algorithm maintains the same data partition as the proposed algorithm. The above process is repeated ten times and the average value is taken to evaluate each method. ACC, SEN, SPE, and AUC are used as evaluation indicators to quantitatively evaluate the classification performance of all methods.

[0136] Performance Index Evaluation

[0137] Accuracy (ACC), sensitivity (SEN), specificity (SPE), and AUC were used to measure the classification performance of all relevant methods.

[0138] ACC indicates the proportion of correctly classified samples to the total number of samples, SEN indicates the proportion of correctly classified samples with ASD as the real samples, and SPE indicates the proportion of correctly classified samples with healthy controls as the real samples. The higher the values ​​of these three indicators, the better the classification performance of the model. The calculation method of ACC, SEN and SPE is shown in the following formula:

[0139]

[0140] In the above formula, TP is the number of true positive examples, i.e., the number of samples with the true label ASD that are predicted correctly, FN is the number of false negative examples, i.e., the number of samples with the true label ASD that are predicted incorrectly, TN is the number of true negative examples, i.e., the number of samples with the true label healthy control that are predicted correctly, and FP is the number of false positive examples, i.e., the number of samples with the true label healthy control that are predicted incorrectly.

[0141] Experimental Summary

[0142] Comparison with state-of-the-art methods and baselines

[0143] In order to fully verify the effectiveness of the method proposed in this invention, its results are compared with those of the latest methods.

[0144] ST-Transformer: A linear spatiotemporal multi-head attention unit is proposed to obtain spatial and temporal representations of fMRI data. In addition, a data balancing method based on Gaussian GAN is introduced to solve the data imbalance problem in real-world ASD datasets for ASD subtype diagnosis.

[0145] ST-ASDNet: Two modules are proposed: Bidirectional Long Short-Term Memory Transformer (BLSTM-Transformer) and Fully Convolutional Network Transformer (FCN-Transformer), which respectively obtain the spatial and temporal features of fMRI data.

[0146] BrainNETTF: proposes an orthogonal clustering readout operation based on self-supervised soft clustering and orthogonal projection. This design considers the underlying functional modules that determine the similar behaviors between ROI groups, resulting in distinguishable cluster-aware node embeddings and informative graph embeddings.

[0147] RGTNet: A residual graph transformer network for FC learning, namely RGTNet, is proposed. A graph encoder is designed to extract time-dependent features with long-range dependencies, from which an interpretable FC matrix can be modeled. In addition, the residual technique is introduced to deepen the GCN architecture to learn higher-level information.

[0148] AIMAFE: A multi-graph deep feature representation method based on stacked denoising autoencoders (SDA) is proposed. A multi-layer perceptron (MLP) and ensemble learning method are proposed to perform the final ASD recognition task.

[0149] MDANN: Captures the relationships in multimodal data (functional neuroimaging data and PC data) by integrating multi-layer neural networks, attention mechanisms, and feature fusion.

[0150] PLSNet: A time series encoder is designed for context-rich feature extraction, followed by a functional connection generator for modeling correlations with long-range dependencies. Position embedding technology is used to give each graph region a unique identity. A sparse method is embedded to filter significant nodes during message diffusion, which is also conducive to reducing dimensionality complexity.

[0151] MVS-GCN: A graph-structured learning algorithm that adaptively builds clean brain networks via a supervised learning scheme. Compared with whole-brain networks, coarse graph representations are beneficial for brain network embedding learning and disease diagnosis. In addition, graph-structured learning considers group-level consistency of subjects from multiple sites by highlighting indicative edges, thereby eliminating noisy correlations in brain networks.

[0152] LRCDRl: Adopting low-rank representation, it alleviates the marginal distribution difference between domains by aligning the global structure of projected multi-site data. In order to reduce the conditional distribution difference of all site data, LRCDR learns class-discriminative representations of data from multiple source and target domains to enhance the intra-class compactness and inter-class separability of projected data.

[0153] This embodiment uses a support vector machine (SVM), a random forest (RF), and a naive Bayes classifier (NB). In neuroimaging research, RF, SVM, and NB are often used as baselines.

[0154] Support vector machine: A linear classifier that uses the maximum interval in the feature space as a learning strategy. The kernel method is introduced to map the original features to a high-dimensional space, which can turn it into a de facto nonlinear classifier. Among them, the penalty parameter C = 1, the kernel function kernel = "linear", and gamma = 1.

[0155] Random forest: With decision trees as the basic unit, random forest integrates multiple decision trees through the idea of ​​ensemble learning. Each decision tree randomly extracts data from the training set with replacement and then randomly selects a part of the features as training data to ensure the irrelevance between different trees and improve the noise resistance of random forest. The number of decision trees in random forest is n = 100.

[0156] Naive Bayes classifier: Based on Bayesian theory, it assumes that all features have conditionally independent Gaussian distribution. The Naive Bayes algorithm learns the joint probability distribution of input and output data, and then uses Bayesian theorem to infer the label with the largest posterior probability as the prediction.

[0157] Table 2 Comparison of experimental results with various methods on ABIDE

[0158]

[0159]

[0160] Table 3 Comparison with machine learning methods

[0161]

[0162] Tables 2 and 3 show the average performance of 10 repeated 5-fold cross validations. The proposed method outperforms all the compared methods in terms of accuracy, and its accuracy is verified on three atlases: 75.6% on the AAL atlas, 73.6 on CC200, and 79.4% on dosenbatch160. Compared with the Transformer-related models, the comparison on AAL is 2.2% higher than the accuracy of the RGTNet model among the other models; compared with the GCN-related models, the comparison on AAL and cc200 is improved, up to 6.7%; compared with the domain adaptation-related methods, the accuracy is improved by 2.5%. Comprehensive analysis shows that although other methods have proved that they do have good performance in classification problems, they are not as good as the present invention in the face of the problem of inconsistent feature space distribution of rs-fMRI. The reasons for the analysis are: 1. The feature extraction is not detailed enough. The present invention divides the features into common features and unique features, which takes a wider range of considerations. 2. The present invention adopts a multi-network approach, setting up a network for each sample, reducing the forgetting caused by too much knowledge in the same network. 3. The present invention adopts the method of constructing an uncertainty population map and integrates it into a multi-branch network. Compared with other methods, it considers more non-imaging data and the relationship between samples, making the model more generalized.

[0163] Ablation experiment

[0164] This example verifies the effectiveness of each part of the model, and the results are shown in Table 4:

[0165] Table 4 Validity verification results

[0166]

[0167]

[0168] Ablation experiments were conducted at six sites: UM, NYU, USM, Leuven, UCLA, and YALE. The results are analyzed as follows:

[0169] 1. Compared with the whole, removing the DANN part will result in a 4.5% decrease in accuracy. DANN integrates the feature extractor and the classifier, achieves domain alignment and classification in a simple way, and achieves domain alignment in an adversarial way. At the same time, removing the branch network alignment L s The accuracy dropped by 2.3% after removing the alignment between the common features and the unique features. cs After that, the accuracy dropped by 4.5%, proving the effectiveness of feature refinement.

[0170] 2. Remove the population map L evAfter that, the accuracy dropped by 7.8%, the highest among all losses. This proves that using EV-GCN as a branch network and introducing the relationship between samples and non-imaging data is very effective for the diagnosis and classification of ASD, which enhances the generalization of the model.

[0171] The present invention proposes a new multi-center domain adaptive neural network, which is developed based on Transformer and population graph. Transformer helps to model the long-term dependencies between extracted data on time series data. The GCN network uses the form of population graphs, combines imaging data and non-imaging data to optimize the model, and enhances the generalization of the model. In addition, this paper sets up a backbone network and multiple branch networks in the process of feature learning to extract common features and unique features of the source domain. This method attempts to introduce population graphs in multi-task ASD. The present invention is verified on the datasets of six imaging centers UM, NYU, USM, Leuven, YALE and UCLA in the three atlases of ABIDEI, and the accuracy can be increased to 79.4% at most.

[0172] A data classification system based on deep multi-path attention adaptive graph convolutional network, comprising:

[0173] The data acquisition and processing module obtains resting-state functional magnetic resonance imaging data from the Autism Brain Imaging Data Exchange database and preprocesses them using a configurable pipeline for connectome analysis to obtain BOLD sequences;

[0174] The feature extraction module uses the BOLD sequence to construct a functional feature matrix, and straightens the upper triangular part of the functional feature matrix to obtain the functional connection feature vector as the input of the DMAGCN model;

[0175] Model building module, building DMAGCN model, the model consists of a backbone network and multiple branch networks. The backbone network is built with Transformer, and the branch networks are composed of MLP and graph networks respectively. The backbone network is used to extract the common features of all source domains, and the branch network is used to extract the unique features of a single source domain.

[0176] Model validation module, which validates the DMAGCN model based on the five-fold cross validation method to obtain the optimal DMAGCN model;

[0177] The result output module inputs the data to be classified into the optimal DMAGCN model to obtain the data classification results.

[0178] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0179] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data classification method based on deep multi-path attention adaptive graph convolutional network, characterized in that: include: Resting-state functional magnetic resonance imaging data were obtained from the Autism Brain Imaging Data Exchange database and preprocessed using a configurable pipeline for connectome analysis to obtain BOLD sequences; The functional feature matrix was constructed using the BOLD sequence, and the upper triangular part of the functional feature matrix was straightened to obtain the functional connectivity feature vector as the input of the DMAGCN model. Construct a DMAGCN model, which consists of a backbone network and multiple branch networks. The backbone network is built with Transformer, and the branch networks are composed of MLP and graph networks respectively. The backbone network is used to extract the common features of all source domains, and the branch network is used to extract the unique features of a single source domain. The DMAGCN model was verified based on the five-fold cross-validation method to obtain the optimal DMAGCN model; Input the data to be classified into the optimal DMAGCN model to obtain the data classification results.

2. According to claim 1, a data classification method based on deep multi-path attention adaptive graph convolutional network is characterized in that: The preprocessing involved skull striping, slice timing correction, motion correction, global mean intensity normalization, noise signal regression, bandpass filtering 0.01–0.1 Hz, and registration of resting-state functional MRI data to standard anatomical space.

3. According to claim 1, a data classification method based on deep multi-path attention adaptive graph convolutional network is characterized in that: The calculation formula of the functional connection feature vector is as follows: Among them, X i and X j is the average time series of the i-th ROI region and the j-th ROI region; X i,t ,X j,t Yes X i and X j BOLD intensity at time instant t; and represents the mean of the average BOLD time series of the ith brain region and the jth brain region, respectively, T represents the total number of time points of the average BOLD series, and corr represents the Pearson correlation coefficient.

4. According to claim 1, a data classification method based on deep multi-path attention adaptive graph convolutional network is characterized in that: The backbone network built with MLP and Transformer extracts features from imaging data. In addition to imaging features, the graph network uses non-imaging data as edges between nodes to provide multimodal information to train the DMAGCN model. When the training is completed, the graph network will be discarded, and other networks will jointly make decisions on the target domain.

5. According to claim 1, a data classification method based on deep multi-path attention adaptive graph convolutional network is characterized in that: The backbone network adopts Transformer as the backbone network, and the overall structure of Transformer includes a multi-head self-attention module, a feedforward network, a residual connectivity layer and a normalization layer; The self-attention mechanism is the core of the Transformer-encoder and is calculated from the Query, Key, and Value matrices: in N and M represent the length of Query and Key, D k and D v Represents the dimensions of Key and Value; Softmax is an activation function that converts attention scores into probabilities; the multi-head attention mechanism is applied in Transformer: in, are the parameter matrices corresponding to Q, K, and V respectively; W 0 is the parameter matrix for multi-head attention calculation; the feedforward network then applies two linear transformations with Gelu activation function to the output of the multi-head self-attention: X=FFN(x)=Gelu(xW1+b1)W2+b2; Among them, x is the output of the previous layer, W1, W2, b1 and b2 represent the training parameter matrix and bias value.

6. According to claim 1, a data classification method based on deep multi-path attention adaptive graph convolutional network is characterized in that: In order to complete the feature extraction of multiple tasks, multiple branch networks are set up to extract the unique features of each source domain. Represents the input features, which consists of fully connected layers. Each layer has multiple nodes. Each node receives the output of the previous layer node as input. The output of the kth node in the lth layer is: Among them, k (l) is the output of the lth layer, l=1,2,...,L;k=1,2,...,K;j=1,2,...,J;k≠j;o j (l-1) is the feature of the kth node in the l-1th layer; θ jk (l) represents the connection weight between the kth node and the jth node in the lth layer; α l represents the activation function of the lth layer; Use the backbone network to extract common features from all source domains: Use the branch network to extract the unique features of the i-th source domain: Where i=1,2,3...I represents the i-th source domain sample or target domain sample; s is the source domain dataset; f c,i (·) is the extractor of common features; f s,i (·) is the extractor of unique features; Θ s,i Parameters of the i-th source domain-specific feature extractor; Θ c,i represents the parameters of the i-th common feature extractor; represents the common features of the i-th source domain sample in the source domain; represents the unique features of the i-th source domain sample.

7. According to claim 6, a data classification method based on deep multi-path attention adaptive graph convolutional network is characterized in that: The maximum mean difference is used to measure the distribution distance between the common features and unique features related to the source domain, specifically: Where n is the number of samples in the source domain; i and j represent the sample numbers, i≠j; represents the Gaussian kernel Hilbert space; The domain alignment loss is: L domain =L com +L s +L cs .

8. According to claim 1, a data classification method based on deep multi-path attention adaptive graph convolutional network is characterized in that: The graph network is specifically an edge variational graph convolutional network, which uses the spatial perception of the brain network and the population relationship of the data set to train and optimize the model; Given N subjects’ data consisting of imaging and non-imaging data, we construct a population graph: G = (V, E, W), where |V| = N represents the set of vertices. is a set of edges, where the weight of the edge is W; define the node characteristics As the C-dimensional feature vector extracted from the imaging data of the i-th subject; (x i ,x j ) i,j ∈W is defined as a learnable function that represents the information of non-imaging data: φ:(x i ,x j ), which is modeled and trained by the pairwise association encoder PAE: h i =φ(x i ,Oh); h j =φ(x j ,Oh); in, is the normalized input; τ is the ReLU function; h i and h j is the input feature x i and x j Mapping in the same feature space; Ω represents the parameters of the mapping network φ(·); The uncertainty-aware prediction loss function is: Among them, P(x i ) represents the predicted value of the i-th sample, is the true value of the i-th sample; Therefore, the total loss function of the DMAGCN model is: L=λL domain +L ev ; Where λ changes from 0 to 1 over time, γ is a hyperparameter, and ρ represents the number of iterations.

9. The data classification method based on deep multi-path attention adaptive graph convolutional network according to claim 8 is characterized in that: The convolution layer of graph convolution is composed of Chebyshev convolution, and the recurrence relationship of Chebyshev polynomial is: T0(L)=1,T1(L)=L; T k (L)=2LT k-1 (L)-T k-2 (L); The formula for Chebyshev convolution is: Among them, T k (L) represents the topological structure L of the graph G after being calculated by Chebyshev polynomials in the kth term; H l Represents the node feature vector of the lth layer; is the convolution kernel parameter.

10. A data classification system based on deep multi-path attention adaptive graph convolutional network, characterized in that: A data classification method based on a deep multi-path attention adaptive graph convolutional network as described in any one of claims 1 to 9 is applied, comprising: The data acquisition and processing module obtains resting-state functional magnetic resonance imaging data from the Autism Brain Imaging Data Exchange database and preprocesses them using a configurable pipeline for connectome analysis to obtain BOLD sequences; The feature extraction module uses the BOLD sequence to construct a functional feature matrix, and straightens the upper triangular part of the functional feature matrix to obtain the functional connection feature vector as the input of the DMAGCN model; Model building module, building DMAGCN model, the model consists of a backbone network and multiple branch networks. The backbone network is built with Transformer, and the branch networks are composed of MLP and graph networks respectively. The backbone network is used to extract the common features of all source domains, and the branch network is used to extract the unique features of a single source domain. Model validation module, which validates the DMAGCN model based on the five-fold cross validation method to obtain the optimal DMAGCN model; The result output module inputs the data to be classified into the optimal DMAGCN model to obtain the data classification results.

Images