Multi-mode graph convolutional network-based autism spectrum disorder identification method

Abstract

The invention belongs to the field of autism spectrum disorder recognition, and particularly discloses an autism spectrum disorder recognition method based on a multi-mode graph convolutional network, and the method comprises the steps: collecting patient data, preprocessing the data, and constructing the graph convolutional network. The MPPCA method based on the threshold value is designed, prior information related to noise variance and the PCA denoising threshold value are combined, signals with dominant noise can be better distinguished, the function of the prior information related to the noise variance and the like is fully played, and a better data basis is provided for follow-up phenotype prediction; according to the method, a multi-modal graph convolutional network is used, phenotypes are predicted according to brain imaging data, behavior data and gene data of a patient, advantage complementation among different data types is utilized, and specific parameterization design is performed on the weight of a filter, so that the recognition accuracy of the autism spectrum disorder is improved.

Description

Technical Field

[0001] The present invention relates to the field of autism spectrum disorder recognition, and specifically refers to an autism spectrum disorder recognition method based on a multi-modal graph convolutional network. Background Art

[0002] The identification of autism spectrum disorder is mainly through observation, assessment and professional diagnosis to find out whether an individual has the characteristics and manifestations of autism spectrum disorder. Early identification can allow children to start targeted rehabilitation training and education as early as possible, which helps to improve their prognosis and development. The identification of autism spectrum disorder not only relies on behavioral observation, but also requires imaging examinations such as brain MRI as auxiliary means. Among the existing denoising methods, MPPCA (Marchenko-PasturPrincipal Component Analysis) is an effective denoising method. However, MPPCA is seriously affected by spatially correlated noise and cannot denoise spatially correlated data. When identifying autism spectrum disorder, it is necessary to observe the children in many aspects, including abnormal social interaction, communication barriers, repetitive stereotyped behaviors and limited interests, etc. There are certain limitations and it is difficult to make accurate judgments. Summary of the invention

[0003] In view of the above situation, in order to overcome the defects of the prior art, the present invention provides an autism spectrum disorder identification method based on a multimodal graph convolutional network. The identification of autism spectrum disorder not only relies on behavioral observation, but also requires imaging examinations such as brain nuclear magnetic resonance as auxiliary means. In view of the technical problem that MPPCA is an effective denoising method in the existing denoising methods, but MPPCA is seriously affected by spatially correlated noise and cannot denoise spatially correlated data, the present invention designs a threshold-based MPPCA method, which combines the prior information about the noise variance with the PCA denoising threshold, can better distinguish the signal dominated by noise, and give full play to the role of prior information such as noise variance. It provides a better data basis for subsequent phenotype prediction; in order to solve the technical problem that it is difficult to make accurate judgments, it is necessary to observe children in many aspects, including abnormal social interactions, communication disorders, repetitive stereotyped behaviors and limited interests, etc., when identifying autism spectrum disorders, the present invention uses a multimodal graph convolutional network to predict phenotypes based on the patient's brain imaging data, behavioral data and genetic data, and uses the complementary advantages of different data types. By performing specific parameterized design on the filter weights, it can better adapt to the characteristics and relationships of these different types of data, making the model more in line with actual needs, and can more comprehensively understand the characteristics and patterns related to the disease, thereby improving the accuracy of autism spectrum disorder identification.

[0004] The technical solution adopted by the present invention is as follows: The present invention provides an autism spectrum disorder identification method based on a multi-modal graph convolutional network, and the autism spectrum disorder identification method based on a multi-modal graph convolutional network mainly includes the following steps:

[0005] Step S1: patient data collection;

[0006] Step S2: data preprocessing;

[0007] Step S3: construct a graph convolutional network;

[0008] The patient data collection collects brain imaging data, behavioral data, and genetic data of the patient;

[0009] The data preprocessing uses a threshold-based MPPCA method to denoise the patient's brain imaging data, delete the part of the behavioral data containing blank values, manually correct and mark the obvious erroneous data, map the behavioral data to the interval [0, 1], remove the part of the genetic data with a high sequencing error rate, and delete duplicate data;

[0010] The graph convolution network is constructed, and the importance weights of brain imaging data, behavioral data, and genetic data are determined and integrated using the attention mechanism, and a graph convolution framework is designed to predict the patient phenotype;

[0011] Furthermore, step S2: data preprocessing, using a threshold-based MPPCA method to denoise the patient's brain imaging data, the threshold-based MPPCA method specifically comprising the following steps:

[0012] Step S21: Create and initialize the MPPCA model as a denoising model, input the patient's brain imaging data into the denoising model, define a U×V matrix X as a sliding window, where U represents the number of rows in the sliding window, rows represent adjacent voxels, V represents the number of columns in the sliding window, and columns represent redundant measurements. Calculate the column mean of each column of the matrix X, subtract the corresponding column mean from each redundant measurement in the matrix X, and perform principal component analysis in the eigendecomposition of the covariance matrix of the matrix X. The formula used is as follows: ;

[0013] In the formula, is a V×V matrix containing all PCA eigenvectors, is a V×V diagonal matrix containing the principal component analysis eigenvalues, Indicates transposition;

[0014] Step S22: Fill the matrix X with pure Gaussian random noise, and propagate the pure Gaussian random noise to the principal component analysis eigenvalues ​​in the form of probability distribution. The formula used is as follows: ;

[0015] In the formula, is a function of the probability distribution, , is the upper bound of the MP distribution, is the lower bound of the MP distribution, is the principal component analysis eigenvalue, ;

[0016] Step S23: Calculate the noise variance estimate based on the MP distribution bandwidth, using the following formula: ;

[0017] In the formula, is the noise variance estimate, is the noise-related Quantity, for Principal component analysis eigenvalues ​​on components;

[0018] Step S24: Classify the eigenvalues ​​related to noise into The maximum number of the smallest eigenvalues ​​that best matches the probability distribution of , and the largest eigenvalues ​​are iteratively removed until the average of the eigenvalues ​​is higher than the noise variance estimate calculated according to the MP distribution;

[0019] Step S25: Calculate the prior noise variance estimate, using the following formula: ;

[0020] In the formula, is the prior noise variance estimate, is the diffusion weight, yes The number of is the voxel position, is In the position The signal obtained when is the average signal;

[0021] Step S26: define the characteristic value threshold criterion of the denoising model, the formula used is as follows: ;

[0022] In the formula, is the eigenvalue threshold of the denoising model;

[0023] Step S27: The eigenvalues ​​below the eigenvalue threshold are reset to zero, the denoised signal of each sliding window is reconstructed through the eigenvector, and the intermediate voxels of each sliding window are completely reconstructed to obtain denoised brain imaging data.

[0024] Further, step S3: constructing a graph convolution network, and using a multimodal graph convolution method to predict the patient phenotype, the multimodal graph convolution method specifically includes the following steps:

[0025] Step S31: collecting existing brain rs-fMRI data and DTI data of autistic patients, and annotating the patient phenotypes to form a brain data set;

[0026] Step S32: creating and initializing a graph convolution framework, wherein the graph convolution framework is composed of a learning module and an artificial neural network, assuming that N is the number of patients and P is the number of brain regions;

[0027] Step S33: Extract the structural connectivity graph from the DTI data and use It indicates that, are nodes, which are segmented brain regions, It is an edge, indicating that there is at least one fiber bundle between the segmented brain regions. The brain rs-fMRI data and DTI data are input into the graph convolution framework. The graph convolution framework aggregates information at each node based on the topological structure of the structural connectivity graph and performs graph filtering on the input brain rs-fMRI data and DTI data. The formula used is as follows: ; ; ;

[0028] In the formula, is the structural connectivity graph of the input, is the structural connectivity graph after filtering, are the filter weights, is the adjacency matrix of the structural connectivity graph, is a dimensional identity matrix, is the reparameterized adjacency matrix and the depth matrix The Laplacian graph of and represents the coordinates of the node, represents a graph filtering function;

[0029] Step S34: define a graph filtering operation to act on the correlation matrix of the rs-fMRI data to generate a connectivity embedding matrix. The formula used is as follows: ;

[0030] In the formula, Compute Node The weighted sum of the functional connectivity of is the number of channels, each channel has learnable parameters, is a row filter, is a column filter, is a bias term, each parameter is parameterized by row and column filters, is the Laplacian matrix of the DTI data, is the correlation matrix of rs-fMRI data;

[0031] Step S35: Use two graph convolutional layers with pooling to calculate a node-level representation and full-graph embedding. The formula used is as follows: ;

[0032] In the formula, and They are node-level representation and full-graph embedding, and is the bias term, is the channel vector, is the graph embedding matrix;

[0033] Step S36: Use channel vector, graph embedding matrix, bias term and The weights of the filters are parameterized and the full image embedding is input into an artificial neural network to predict the patient phenotype;

[0034] Step S37: define the objective function of the artificial neural network, and use the L2 loss function and the L1 loss function to train the artificial neural network. The formula used is as follows: ;

[0035] In the formula, is the objective function of training, is the total number of categories of patient phenotypes, is the annotated patient phenotype, is the predicted patient phenotype, is the L2 loss function, is the L1 loss function;

[0036] Step S38: extract the patient's rs-fMRI data and DTI data from the denoised brain imaging data, input them into the trained graph convolution framework, identify the patient's phenotype and output the results.

[0037] The beneficial effects achieved by the present invention using the above scheme are as follows:

[0038] (1) The identification of autism spectrum disorders not only relies on behavioral observation, but also requires imaging examinations such as brain magnetic resonance imaging as auxiliary means. Among the existing denoising methods, MPPCA is an effective denoising method, but MPPCA is seriously affected by spatially correlated noise and cannot denoise spatially correlated data. The present invention designs a threshold-based MPPCA method that combines the prior information about the noise variance with the PCA denoising threshold, which can better distinguish the signal dominated by noise, give full play to the role of prior information such as noise variance, and provide a better data basis for subsequent phenotypic prediction;

[0039] (2) In view of the technical problem that it is difficult to make accurate judgments when observing children in many aspects, including abnormal social interactions, communication disorders, repetitive stereotyped behaviors and restricted interests, the present invention uses a multimodal graph convolutional network to predict phenotypes based on the patient's brain imaging data, behavioral data and genetic data, and utilizes the complementary advantages of different data types. By performing specific parameterized design on the filter weights, it can better adapt to the characteristics and relationships of these different types of data, making the model more in line with actual needs, and can more comprehensively understand the characteristics and patterns related to the disease, thereby improving the accuracy of autism spectrum disorder identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A flowchart of a method for identifying autism spectrum disorders based on a multi-modal graph convolutional network provided by the present invention;

[0041] Figure 2 FIG. 4 is a flowchart of the threshold-based MPPCA method described in step S2.

[0042] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION

[0043] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0044] Example 1: See Figure 1This embodiment provides an autism spectrum disorder identification method based on a multi-modal graph convolutional network. The autism spectrum disorder identification method based on a multi-modal graph convolutional network mainly includes the following steps:

[0045] Step S1: patient data collection;

[0046] Step S2: data preprocessing;

[0047] Step S3: construct a graph convolutional network;

[0048] The patient data collection collects brain imaging data, behavioral data, and genetic data of the patient;

[0049] The data preprocessing uses a threshold-based MPPCA method to denoise the patient's brain imaging data, delete the part of the behavioral data containing blank values, manually correct and mark the obvious erroneous data, map the behavioral data to the interval [0, 1], remove the part of the genetic data with a high sequencing error rate, and delete duplicate data;

[0050] The graph convolutional network is constructed, and the attention mechanism is used to determine the importance weights of brain imaging data, behavioral data, and genetic data and fuse them, and a graph convolution framework is designed to predict the patient phenotype.

[0051] Embodiment 2: This embodiment is based on the above embodiment. Step S2: Data preprocessing, using a threshold-based MPPCA method to denoise the patient's brain imaging data. The threshold-based MPPCA method specifically includes the following steps:

[0052] Step S21: Create and initialize the MPPCA model as a denoising model, input the patient's brain imaging data into the denoising model, define a U×V matrix X as a sliding window, where U represents the number of rows in the sliding window, rows represent adjacent voxels, V represents the number of columns in the sliding window, and columns represent redundant measurements. Calculate the column mean of each column of the matrix X, subtract the corresponding column mean from each redundant measurement in the matrix X, and perform principal component analysis in the eigendecomposition of the covariance matrix of the matrix X. The formula used is as follows: ;

[0053] In the formula, is a V×V matrix containing all PCA eigenvectors, is a V×V diagonal matrix containing the principal component analysis eigenvalues, Indicates transposition;

[0054] Step S22: Fill the matrix X with pure Gaussian random noise, and propagate the pure Gaussian random noise to the principal component analysis eigenvalues ​​in the form of probability distribution. The formula used is as follows: ;

[0055] In the formula, is a function of the probability distribution, , is the upper bound of the MP distribution, is the lower bound of the MP distribution, is the principal component analysis eigenvalue, ;

[0056] Step S23: Calculate the noise variance estimate based on the MP distribution bandwidth, using the following formula: ;

[0057] In the formula, is the noise variance estimate, is the noise-related Quantity, for Principal component analysis eigenvalues ​​on components;

[0058] Step S24: Classify the eigenvalues ​​related to noise into The maximum number of the smallest eigenvalues ​​that best matches the probability distribution of , and the largest eigenvalues ​​are iteratively removed until the average of the eigenvalues ​​is higher than the noise variance estimate calculated according to the MP distribution;

[0059] Step S25: Calculate the prior noise variance estimate using the following formula: ;

[0060] In the formula, is the prior noise variance estimate, is the diffusion weight, yes The number of is the voxel position, is In the position The signal obtained when is the average signal;

[0061] Step S26: define the characteristic value threshold criterion of the denoising model, the formula used is as follows: ;

[0062] In the formula, is the eigenvalue threshold of the denoising model;

[0063] Step S27: The eigenvalues ​​below the eigenvalue threshold are reset to zero, the denoised signal of each sliding window is reconstructed through the eigenvector, and the intermediate voxels of each sliding window are completely reconstructed to obtain denoised brain imaging data.

[0064] Through the above operations, in view of the technical problem that among the existing denoising methods, MPPCA is an effective denoising method, but MPPCA is seriously affected by spatially correlated noise and cannot denoise spatially correlated data, the present invention designs a threshold-based MPPCA method, which combines the prior information about the noise variance and the PCA denoising threshold, can better distinguish the signal dominated by noise, give full play to the role of the prior information such as the noise variance, and provide a better data basis for subsequent phenotypic prediction.

[0065] Embodiment 3: This embodiment is based on the above embodiment, step S3: constructing a graph convolution network, using a multimodal graph convolution method to predict the patient phenotype, the multimodal graph convolution method specifically includes the following steps:

[0066] Step S31: collecting existing brain rs-fMRI data and DTI data of autistic patients, and annotating the patient phenotypes to form a brain data set;

[0067] Step S32: creating and initializing a graph convolution framework, wherein the graph convolution framework is composed of a learning module and an artificial neural network, assuming that N is the number of patients and P is the number of brain regions;

[0068] Step S33: Extract the structural connectivity graph from the DTI data and use It indicates that, are nodes, which are segmented brain regions, It is an edge, indicating that there is at least one fiber bundle between the segmented brain regions. The brain rs-fMRI data and DTI data are input into the graph convolution framework. The graph convolution framework aggregates information at each node based on the topological structure of the structural connectivity graph and performs graph filtering on the input brain rs-fMRI data and DTI data. The formula used is as follows: ; ; ;

[0069] In the formula, is the structural connectivity graph of the input, is the structural connectivity graph after filtering, are the filter weights, is the adjacency matrix of the structural connectivity graph, is a dimensional identity matrix, is the reparameterized adjacency matrix and the depth matrix The Laplacian graph of and represents the coordinates of the node, represents a graph filtering function;

[0070] Step S34: define a graph filtering operation to act on the correlation matrix of the rs-fMRI data to generate a connectivity embedding matrix. The formula used is as follows: ;

[0071] In the formula, Compute Node The weighted sum of the functional connectivity of is the number of channels, each channel has learnable parameters, is a row filter, is a column filter, is a bias term, each parameter is parameterized by row and column filters, is the Laplacian matrix of the DTI data, is the correlation matrix of rs-fMRI data;

[0072] Step S35: Use two graph convolutional layers with pooling to calculate a node-level representation and full-graph embedding. The formula used is as follows: ;

[0073] In the formula, and They are node-level representation and full-graph embedding, and is the bias term, is the channel vector, is the graph embedding matrix;

[0074] Step S36: Use channel vector, graph embedding matrix, bias term and The weights of the filters are parameterized and the full image embedding is input into an artificial neural network to predict the patient phenotype;

[0075] Step S37: define the objective function of the artificial neural network, and use the L2 loss function and the L1 loss function to train the artificial neural network. The formula used is as follows: ;

[0076] In the formula, is the objective function of training, is the total number of categories of patient phenotypes, is the annotated patient phenotype, is the predicted patient phenotype, is the L2 loss function, is the L1 loss function;

[0077] Step S38: extract the patient's rs-fMRI data and DTI data from the denoised brain imaging data, input them into the trained graph convolution framework, identify the patient's phenotype and output the results.

[0078] Through the above operations, in order to solve the technical problem that when identifying autism spectrum disorders, it is necessary to observe the children in many aspects, including abnormal social interaction, communication disorders, repetitive stereotyped behaviors and limited interests, etc., which have certain limitations and are difficult to accurately judge, the present invention uses a multimodal graph convolutional network to predict the phenotype based on the patient's brain imaging data, behavioral data and genetic data, and utilizes the complementary advantages of different data types. By performing specific parameterized design on the filter weights, it can better adapt to the characteristics and relationships of these different types of data, make the model more in line with actual needs, and can more comprehensively understand the characteristics and patterns related to the disease, thereby improving the accuracy of autism spectrum disorder identification.

[0079] Embodiment 4: This embodiment is based on the above embodiment. In Embodiment 3, the existing brain rs-fMRI data and DTI data of autistic patients are obtained from the ADNI database (Alzheimer's Disease Neuroimaging Initiative), which complies with the data acquisition process and requirements, is obtained under legal circumstances, and is a case of legal use.

[0080] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0081] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

[0082] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.

Claims

1. A method for identifying autism spectrum disorders based on a multimodal graph convolutional network, characterized in that: The specific steps include: Step S1: patient data collection; Step S2: data preprocessing; Step S3: construct a graph convolutional network; The patient data collection collects brain imaging data, behavioral data, and genetic data of the patient; The data preprocessing uses a threshold-based MPPCA method to denoise the patient's brain imaging data, delete the part of the behavioral data containing blank values, manually correct and mark the obvious erroneous data, map the behavioral data to the interval [0, 1], remove the part of the genetic data with a high sequencing error rate, and delete duplicate data; The graph convolutional network is constructed, and the attention mechanism is used to determine the importance weights of brain imaging data, behavioral data, and genetic data and fuse them, and a graph convolution framework is designed to predict the patient phenotype.

2. The method for identifying autism spectrum disorders based on a multimodal graph convolutional network according to claim 1, characterized in that: Step S2, data preprocessing, specifically includes the following steps: Step S21: Create and initialize an MPPCA model as a denoising model, input the patient's brain imaging data into the denoising model, define a U×V matrix X as a sliding window, wherein U represents the number of rows in the sliding window, rows represent adjacent voxels, V represents the number of columns in the sliding window, columns represent redundant measurements, calculate the column mean of each column of the matrix X, subtract the corresponding column mean from each redundant measurement in the matrix X, and perform principal component analysis in the eigendecomposition of the covariance matrix of the matrix X; Step S22: fill the matrix X with pure Gaussian random noise, and propagate the pure Gaussian random noise to the principal component analysis eigenvalues ​​in the form of probability distribution; Step S23: Calculate the noise variance estimate based on the MP distribution bandwidth, using the following formula: ； In the formula, is the noise variance estimate, is the noise-related Quantity, for Principal component analysis eigenvalues ​​on components; Step S24: Classify the eigenvalues ​​related to noise into The maximum number of the smallest eigenvalues ​​that best matches the probability distribution of , and the largest eigenvalues ​​are iteratively removed until the average of the eigenvalues ​​is higher than the noise variance estimate calculated according to the MP distribution; Step S25: Calculate the prior noise variance estimate using the following formula: ； In the formula, is the prior noise variance estimate, is the diffusion weight, yes The number of is the voxel position, is In the position The signal obtained when is the average signal; Step S26: define the characteristic value threshold criterion of the denoising model, the formula used is as follows: ； In the formula, is the eigenvalue threshold of the denoising model; Step S27: The eigenvalues ​​below the eigenvalue threshold are reset to zero, the denoised signal of each sliding window is reconstructed through the eigenvector, and the intermediate voxels of each sliding window are completely reconstructed to obtain denoised brain imaging data.

3. The autism spectrum disorder identification method based on a multimodal graph convolutional network according to claim 2, characterized in that: Step S3, constructing a graph convolutional network, specifically includes the following steps: Step S31: collecting existing brain rs-fMRI data and DTI data of autistic patients, and annotating the patient phenotypes to form a brain data set; Step S32: creating and initializing a graph convolution framework, wherein the graph convolution framework is composed of a learning module and an artificial neural network, assuming that N is the number of patients and P is the number of brain regions; Step S33: Extract the structural connectivity graph from the DTI data and use It indicates that, are nodes, which are segmented brain regions, It is an edge, indicating that there is at least one fiber bundle between the segmented brain regions. The brain rs-fMRI data and DTI data are input into the graph convolution framework. The graph convolution framework aggregates information at each node based on the topological structure of the structural connectivity graph and performs graph filtering on the input brain rs-fMRI data and DTI data. The formula used is as follows: ； ； ； In the formula, is the structural connectivity graph of the input, is the structural connectivity graph after filtering, are the filter weights, is the adjacency matrix of the structural connectivity graph, is a dimensional identity matrix, is the reparameterized adjacency matrix and the depth matrix The Laplacian graph of and represents the coordinates of the node, represents a graph filtering function; Step S34: define a graph filtering operation to act on the correlation matrix of the rs-fMRI data to generate a connectivity embedding matrix. The formula used is as follows: ； In the formula, Compute Node The weighted sum of the functional connectivity of is the number of channels, each channel has learnable parameters, is a row filter, is a column filter, is a bias term, each parameter is parameterized by row and column filters, is the Laplacian matrix of the DTI data, is the correlation matrix of rs-fMRI data; Step S35: Use two graph convolutional layers with pooling to calculate a node-level representation and full-graph embedding. The formula used is as follows: ； In the formula, and They are node-level representation and full-graph embedding, and is the bias term, is the channel vector, is the graph embedding matrix; Step S36: Use channel vector, graph embedding matrix, bias term and The weights of the filters are parameterized and the full image embedding is input into an artificial neural network to predict the patient phenotype; Step S37: defining the objective function of the artificial neural network, and using a combination of an L2 loss function and an L1 loss function to train the artificial neural network; Step S38: extract the patient's rs-fMRI data and DTI data from the denoised brain imaging data, input them into the trained graph convolution framework, identify the patient's phenotype and output the results.

Images