GNN-based multi-modal brain imaging autism spectrum disorder detection method

Abstract

The invention discloses a GNN-based multi-mode brain imaging infantile autism spectrum disorder detection method, which is characterized in that a function connection network which is different from traditional characteristics and takes an edge as a center is introduced, higher-order information interaction in a brain interval is mined, and a brain function abnormity mode of an infantile autism patient is accurately captured. Meanwhile, integrating brain imaging data and clinical text information into a unified multi-modal graph by utilizing the graph structure learning ability of GNN, capturing a complex nonlinear relationship, and realizing deep combination of multi-modal features; according to the method, the accuracy and robustness of autism detection are effectively improved, technical support is provided for personalized diagnosis and treatment, key technical problems in multi-modal data fusion and brain network analysis are solved, and the accuracy and reliability of ASD early diagnosis are improved.

Description

Technical Field

[0001] The present invention belongs to the fields of computer technology and brain science, and in particular to a multimodal brain imaging autism spectrum disorder detection method based on GNN. Background Art

[0002] ASD is a common neurodevelopmental disorder characterized by social difficulties, language disorders, and repetitive behaviors. With the continuous development of medical imaging technology, brain imaging techniques such as functional magnetic resonance imaging (fMRI), structural magnetic resonance imaging (sMRI), and diffusion tensor imaging (DTI) have achieved remarkable results in the early diagnosis of ASD. These technologies can reveal abnormal changes in brain function and structure, providing valuable clues for the early identification of ASD. However, existing brain imaging methods mainly focus on analyzing the connection strength or functional association between nodes (i.e., brain regions), ignoring the edge-centered brain network characteristics, which limits its potential as a biomarker in ASD detection. With the rapid development of deep learning and GNN, more and more studies have begun to focus on the analysis of graph structured data, especially in processing conventional graph data such as social networks and traffic networks. Although GNN has achieved significant applications in many fields, its application in the joint analysis of brain imaging and clinical text data is still in the initial exploration stage. At present, graph neural network technology is mainly focused on processing conventional graph data, while the application of GNN technology has not been fully developed in the framework of fusing multimodal brain imaging data with clinical text information; the existing technology has the following shortcomings: 1. The potential of edge-centered brain networks as biomarkers has not been fully explored: Most current brain network analysis methods focus more on node-centered functional connectivity matrices, which are mainly used to explore abnormal changes in the strength of connections between pairs of brain regions of patients, ignoring the importance of edge-centered functional connectivity networks in the flow and interaction of information between brain regions. As a key carrier of information transmission in brain networks, edges contain interactive features. However, traditional methods are weak in extracting and analyzing edge information and fail to fully explore the value of edges as potential biomarkers. This limits the accuracy and reliability of autism detection based on brain networks.

[0003] 2. Limitations of graph structure analysis methods: Existing GNN technology applications are mainly limited to the analysis of conventional graph data in conventional fields. The application of GNN to the joint analysis of brain imaging and clinical text data is still a relatively new research direction, especially in the application of ASD detection, the use of GNN is still in the exploratory stage. Current research lacks a unified framework that can effectively combine multimodal brain imaging data and clinical text information, which limits the potential of brain imaging technology in the early detection of ASD.

[0004] Therefore, it is necessary to propose a GNN-based multimodal brain imaging autism spectrum disorder detection method to solve the above problems. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a GNN-based multimodal brain imaging autism spectrum disorder detection method, which constructs an edge-centered functional connection network, deeply explores high-order information interactions between brain regions, accurately captures abnormal brain function patterns of autistic patients, integrates brain imaging data and clinical text information into a unified multimodal graph, and utilizes the graph structure learning ability of GNN to capture complex nonlinear relationships and achieve deep feature fusion, thereby effectively improving the accuracy and robustness of autism detection; the present invention aims to improve the accuracy and reliability of early diagnosis of ASD, solve key technical problems in multimodal data fusion and brain network analysis, and thus promote the innovative development of ASD detection methods.

[0006] In order to achieve the above technical effects, the technical solution adopted by the present invention is: A multimodal brain imaging autism spectrum disorder detection method based on GNN, comprising the following steps: S1, obtain fMRI data of subjects; the data comes from public datasets or individual data collected in clinical trials; S2, preprocessing the acquired fMRI data, including denoising, spatial normalization and time alignment; through preprocessing, the data is converted into a time series that meets the analysis requirements; S3, based on the extracted time series, the edge-centered functional connectivity matrix is ​​calculated; S4, feature selection was performed on the edge-centered functional connectivity matrix to screen out key features that were closely related to ASD diagnosis; S5, constructing an adjacency matrix describing edge features in combination with the clinical text information of the subject; including performing a clinical diagnosis on the subject and obtaining text data of the patient's gender and age as a basis for constructing the adjacency matrix; S6, the preprocessed fMRI features and clinical text features are taken as input and sent to GNN for joint learning and classification.

[0007] Preferably, the data preprocessing method in step S2 is: The fMRI data obtained in step S1 were preprocessed using the open source software package C-PAC, including head motion correction, time slice correction, spatial registration, spatial smoothing, and bandpass filtering.

[0008] Furthermore, head motion correction: During the data collection process, although the subject's head was fixed in advance with auxiliary methods, the subject's head was likely to move due to the long scanning time, and slight head motion would affect the accuracy of subsequent experimental data analysis. Therefore, the subject's brain was assumed to be a rigid body, and the data was corrected using the six head motion parameters of translation and rotation to eliminate the influence of the subject's head motion.

[0009] Furthermore, head time slice correction is performed: the acquisition time of different slices in the fMRI data is adjusted, and the data of all slices are aligned to the same time point to eliminate the impact of scanning time differences on the timing of brain activity signals; through the interpolation method, the slice data is unified to the time point of the benchmark slice to ensure the consistency of the data timing.

[0010] Furthermore, head space registration is performed: the low-resolution functional image is registered to the high-resolution structural image of the same subject.

[0011] Furthermore, head space smoothing: During the acquisition process, the subject and the scanning machine will generate certain interference signals, and the acquired image will have certain image noise. Therefore, it is necessary to smooth the image to eliminate the interference signals generated by the subject and the scanning machine.

[0012] Furthermore, bandpass filtering: Uncertain factors such as noise will be generated during acquisition, which will cause the acquired images to contain high-frequency signals, which will have a certain impact on subsequent analysis. Therefore, it is necessary to filter the acquired data to reduce noise interference.

[0013] The functional image is registered to the standard anatomical space MNI152 to obtain the preprocessed fMRI data matrix, the dimension of the matrix is: ; in Is the size of The matrix of is the number of brain regions, is the number of time points; Indicates The time series of brain regions is , indicating that this brain region is A signal at a time point; In standard space, C-PAC divides the brain regions according to the AAL template and extracts the average signal of the time series of all voxels in the region for each brain region of interest: ; Indicates The time series of voxels, Indicates Regions of interest, It is The number of all voxels in a region; finally a time series matrix is ​​obtained, which represents the signal changes of each brain region.

[0014] Preferably, in step S3, the method for calculating the edge-centered functional connectivity matrix is: The z-scores of the average time series extracted from all brain regions are normalized to evaluate the deviation of the sample points from the overall mean, thereby achieving data normalization: ; in and is the average time series of the two brain regions, and yes and Two time series Fraction; For any two average time series extracted from brain regions, their element-wise product is calculated to obtain a new set of time series, called "marginal time series": ; ; In the formula, Represents brain area and brain regions The “marginal time series” between . Represents brain area and brain regions The “marginal time series” between . and Represents brain area and brain regions The z-score of the mean time series.

[0015] Compute the element-wise product between any two “edge time series” to obtain the final edge-centric functional connectivity matrix: ; Where EFCN represents the obtained edge-centric functional connectivity matrix.

[0016] Preferably, in step S4, feature selection is performed on the edge-centered functional connectivity matrix to screen out key features closely related to ASD diagnosis, including: For the high-dimensional edge functional connectivity matrix of the subject, a random sampling method without replacement is used to comprehensively and evenly cover the feature subset; assuming that each time from Select from the features Features, each sampling is expressed as: ; in is from A set of randomly selected indices from features, ; Use the random forest algorithm to perform classification analysis on the sampled feature subsets to obtain the importance ranking of each diagnostic feature: ; in, represents the importance score; Indicates Tree, features of Impurity reduction; The importance ranking of each feature is calculated : ; Among them, sort represents the sorting function; The top 10%, 20%, and 30% of features were selected and included in the final feature set to provide data support for subsequent classification and brain region localization.

[0017] Preferably, in step S6, the method of taking the preprocessed fMRI features and clinical text features as input and sending them to GNN for joint learning and classification is: The final feature set extracted is used as the patient's node features, and combined with the text features obtained through clinical diagnosis to construct an adjacency matrix, which is used as edge features and node features to form a multimodal graph; The multimodal graph is fed into GNN for processing, and the high-level features of the brain network are captured through GNN node embedding learning; GNN optimizes node representation by passing information and finally obtains the embedding vector of each node : ; in, Is a node In the The layer representation, Is a node The set of neighbor nodes of is the normalization coefficient, usually the degree matrix of the adjacency matrix or other forms of weights, It is The weight matrix of the layer, is the activation function; After completing the GNN learning, the shared feature award is input into the fully connected classifier for ASD classification prediction. The output of the fully connected layer It is expressed as: ; in, is the weight matrix of the output layer, is the bias term, Is a node prediction labels; finally, the prediction results are output and a personalized diagnosis report is generated to provide support for clinical decision-making.

[0018] The beneficial effects of the present invention are as follows: 1. This invention breaks through the limitation of existing brain network analysis methods that are overly dependent on brain region nodes, and proposes a functional connection network analysis method centered on edges; edges, as carriers of information flow and interaction between brain regions, carry the core characteristics of brain function, and existing methods fail to fully tap this feature; by focusing on the dynamic information flow of edges, this invention can more accurately capture the abnormal brain function patterns of autistic patients and provide more reliable biomarkers for the early diagnosis of ASD. This innovation improves the accuracy and reliability of autism detection.

[0019] 2. The present invention adopts a GNN-based multimodal data fusion framework, which can effectively integrate brain imaging data and clinical text information; through the graph structure learning ability of graph neural networks, the present invention can capture the complex nonlinear relationship between brain imaging data and clinical information, and make up for the limitation of traditional methods that only rely on a single data source; this framework can improve the utilization efficiency of multimodal data, thereby improving the comprehensiveness and accuracy of ASD detection.

[0020] 3. As a powerful graph structure data analysis tool, GNN has achieved remarkable results in social networks, traffic networks and other fields, but it is still in the initial exploration stage in the joint analysis of brain imaging and clinical text data; this scheme innovatively applies GNN technology to the field of ASD detection, and improves the accuracy, robustness and generalization ability of detection by adaptively learning the complex relationship between multimodal data; this innovative application breaks through the bottleneck of the application of graph neural networks in the medical field and provides more powerful technical support for the detection of ASD.

[0021] 4. The present invention emphasizes the role of edges in brain function transmission, explores high-order information interaction between brain regions, breaks through the reliance on node connection strength in traditional brain network analysis, and can more accurately identify abnormal brain functions in patients with autism. It solves the problem that traditional methods only rely on a single data source. The framework can automatically learn the complex relationship between multimodal data and improve the accuracy and robustness of ASD detection. By combining graph neural networks with multimodal data fusion frameworks, an accurate and robust ASD detection technology is provided. The technical framework can effectively process complex high-dimensional data and heterogeneous data, improve the generalization ability of ASD detection, and improve detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a schematic diagram of a flow chart of the present invention; Figure 2 is a schematic diagram of a modeling process in an embodiment of the present invention; Figure 3 It is a schematic diagram of abnormal brain areas located according to the edge-centered functional connection matrix in an embodiment of the present invention. DETAILED DESCRIPTION

[0023] Embodiment 1: like Figure 1 As shown, a GNN-based multimodal brain imaging autism spectrum disorder detection method includes the following steps: S1, obtain fMRI data of subjects; the data comes from public datasets or individual data collected in clinical trials; S2, preprocessing the acquired fMRI data, including denoising, spatial normalization and time alignment; through preprocessing, the data is converted into a time series that meets the analysis requirements; S3, based on the extracted time series, the edge-centered functional connectivity matrix is ​​calculated; S4, feature selection was performed on the edge-centered functional connectivity matrix to screen out key features that were closely related to ASD diagnosis; S5, constructing an adjacency matrix describing edge features in combination with the clinical text information of the subject; including performing a clinical diagnosis on the subject and obtaining text data of the patient's gender and age as a basis for constructing the adjacency matrix; S6, the preprocessed fMRI features and clinical text features are taken as input and sent to GNN for joint learning and classification.

[0024] Embodiment 2: like Figure 2 As shown, this embodiment provides a specific modeling process of a multimodal brain imaging autism spectrum disorder detection method based on GNN, and the specific process is: The data preprocessing method in step S2 is: The fMRI data obtained in step S1 were preprocessed using the open source software package C-PAC, including head motion correction, time slice correction, spatial registration, spatial smoothing, and bandpass filtering.

[0025] Furthermore, head motion correction: During the data collection process, although the subject's head was fixed in advance with auxiliary methods, the subject's head was likely to move due to the long scanning time, and slight head motion would affect the accuracy of subsequent experimental data analysis. Therefore, the subject's brain was assumed to be a rigid body, and the data was corrected using the six head motion parameters of translation and rotation to eliminate the influence of the subject's head motion.

[0026] Furthermore, head time slice correction is performed: the acquisition time of different slices in the fMRI data is adjusted, and the data of all slices are aligned to the same time point to eliminate the impact of scanning time differences on the timing of brain activity signals; through the interpolation method, the slice data is unified to the time point of the benchmark slice to ensure the consistency of the data timing.

[0027] Furthermore, head space registration is performed: the low-resolution functional image is registered to the high-resolution structural image of the same subject.

[0028] Furthermore, head space smoothing: During the acquisition process, the subject and the scanning machine will generate certain interference signals, and the acquired image will have certain image noise. Therefore, it is necessary to smooth the image to eliminate the interference signals generated by the subject and the scanning machine.

[0029] Furthermore, bandpass filtering: Uncertain factors such as noise will be generated during acquisition, which will cause the acquired images to contain high-frequency signals, which will have a certain impact on subsequent analysis. Therefore, it is necessary to filter the acquired data to reduce noise interference.

[0030] The functional image is registered to the standard anatomical space MNI152 to obtain the preprocessed fMRI data matrix, the dimension of the matrix is: ; in Is the size of The matrix of is the number of brain regions, is the number of time points; Indicates The time series of brain regions is , indicating that this brain region is A signal at a time point; In standard space, C-PAC divides the brain regions according to the AAL template and extracts the average signal of the time series of all voxels in the region for each brain region of interest: ; Indicates The time series of voxels, Indicates Regions of interest, It is The number of all voxels in a region; finally a time series matrix is ​​obtained, which represents the signal changes of each brain region.

[0031] Preferably, in step S3, the method for calculating the edge-centered functional connectivity matrix is: The z-scores of the average time series extracted from all brain regions are normalized to evaluate the deviation of the sample points from the overall mean, thereby achieving data normalization: ; in and is the average time series of the two brain regions, and yes and Two time series Fraction; For any two average time series extracted from brain regions, their element-wise product is calculated to obtain a new set of time series, called "marginal time series": ; ; In the formula, Represents brain area and brain regions The “marginal time series” between . Represents brain area and brain regions The “marginal time series” between . and Represents brain area and brain regions The z-score of the mean time series.

[0032] Compute the element-wise product between any two “edge time series” to obtain the final edge-centric functional connectivity matrix: ; Where EFCN represents the obtained edge-centric functional connectivity matrix.

[0033] like Figure 3 As shown in the figure, this study uses the AAL atlas, and the subjects will get an edge connection matrix of dimension k (k=6105). The edge connection matrix is ​​symmetric, and in order to improve computational efficiency, only the k elements in the lower triangle of the matrix are used in the analysis.

[0034] The edge connection matrix can be regarded as an extension of the node-centered functional connection matrix, and its dimension is much larger than the traditional functional connection matrix. This high-dimensional feature provides more complementary information for studying the relationship between brain activity and behavior. On the one hand, the edge connection matrix omits the final averaging step in the traditional calculation process, thereby reducing the loss of information. The time series of each pair of nodes is retained on the time scale of a single frame, which significantly improves the temporal resolution and can more accurately reflect the resonant activity between brain regions. On the other hand, the edge connection matrix can capture the correlation in the interaction of multiple brain regions and form an edge-based high-order brain network representation. As the dimension of the edge connection matrix increases, it can provide richer high-order brain connection features, thereby revealing the complex interaction patterns and cross-regional correlations between multiple brain regions.

[0035] Preferably, in step S4, feature selection is performed on the edge-centered functional connectivity matrix to screen out key features closely related to ASD diagnosis, including: For the high-dimensional edge functional connectivity matrix of the subjects, a random sampling method without replacement is used to cover the feature subset comprehensively and evenly. The specific application in the experiment is to perform 2000 samplings, and randomly select 10000 features each time. Assuming that Select from the features Features, each sampling is expressed as: ; in is from A set of randomly selected indices from features, ; Use the random forest algorithm to perform classification analysis on the sampled feature subsets to obtain the importance ranking of each diagnostic feature: ; in, represents the importance score; Indicates Tree, features of Impurity reduction; The importance ranking of each feature is calculated : ; Among them, sort represents the sorting function; The top 10%, 20%, and 30% of features were selected and included in the final feature set to provide data support for subsequent classification and brain region localization.

[0036] Preferably, in step S6, the method of taking the preprocessed fMRI features and clinical text features as input and sending them to GNN for joint learning and classification is: The final feature set extracted is used as the patient's node features, and combined with the text features obtained through clinical diagnosis to construct an adjacency matrix, which is used as edge features and node features to form a multimodal graph; The multimodal graph is fed into GNN for processing, and the high-level features of the brain network are captured through GNN node embedding learning; GNN optimizes node representation by passing information and finally obtains the embedding vector of each node : ; in, Is a node In the The layer representation, Is a node The set of neighbor nodes of is the normalization coefficient, usually the degree matrix of the adjacency matrix or other forms of weights, It is The weight matrix of the layer, is the activation function; After completing the GNN learning, the shared feature award is input into the fully connected classifier for ASD classification prediction. The output of the fully connected layer It is expressed as: ; in, is the weight matrix of the output layer, is the bias term, Is a node prediction labels; finally, the prediction results are output and a personalized diagnosis report is generated to provide support for clinical decision-making.

[0037] Embodiment three: In order to objectively evaluate the effect of the present invention, this study used the ABIDE I public data set for model validation, and selected data containing 863 samples that met the imaging quality standards; the data was divided into a training set and a validation set at a ratio of 9:1. Compared with the traditional node-based functional connectivity matrix method, the model of the present invention improved the accuracy and sensitivity by 6 percentage points respectively, as shown in Table 1 below: Table 1: Evaluation of the advantages of edge-centered brain network analysis methods;

[0038] Furthermore, by comparing the support vector machine (SVM) with the GNN, the results show that the model using GNN as the classifier has an accuracy improvement of 14 percentage points and a sensitivity improvement of 12 percentage points compared with the model using SVM, as shown in Table 2 below: Table 2: Classification comparison between GNN and traditional SVM;

[0039] This paper proposes an innovative ASD detection method, which combines edge-centered brain network analysis, multimodal data fusion framework and GNN technology to significantly improve the diagnostic accuracy and robustness of ASD. In order to objectively evaluate the effect of the present invention, this study used the ABIDE I public data set for model validation, and selected 863 samples that met the imaging quality standards. The data was divided into a training set and a validation set at a ratio of 9:1.

[0040] The experimental results show that compared with the traditional node-based functional connection matrix method, the method of the present invention has improved accuracy and sensitivity by 6 percentage points respectively. At the same time, further evaluation and comparison show that the model using GNN as the classifier has improved accuracy and sensitivity by 14 percentage points and 12 percentage points respectively compared with the model using SVM; as shown in Table 3 below: Table 3: Comparison of classification performance between this model and other models;

[0041] This method demonstrates significant technical advantages and provides stronger support for the accurate classification and early diagnosis of ASD.

Claims

1. A multimodal brain imaging autism spectrum disorder detection method based on GNN, characterized in that: The following steps are involved: S1, obtain fMRI data of the subjects, which comes from public datasets or individual data collected in clinical trials; S2, preprocessing of acquired fMRI data, including denoising, spatial normalization, and temporal alignment; Through preprocessing, the data is converted into a time series that meets the analysis requirements; S3, based on the extracted time series, the edge-centered functional connectivity matrix is ​​calculated; S4, feature selection was performed on the edge-centered functional connectivity matrix to screen out key features that were closely related to ASD diagnosis; S5, constructing an adjacency matrix describing edge features in combination with the clinical text information of the subject; including performing a clinical diagnosis on the subject and obtaining text data of the patient's gender and age as a basis for constructing the adjacency matrix; S6, the preprocessed fMRI features and clinical text features are taken as input and sent to GNN for joint learning and classification.

2. According to claim 1, a GNN-based multimodal brain imaging autism spectrum disorder detection method is characterized in that: The data preprocessing method in step S2 is: The fMRI data obtained in step S1 were preprocessed using the open source software package C-PAC, including head motion correction, time slice correction, spatial registration, spatial smoothing, and bandpass filtering; The functional image is registered to the standard anatomical space MNI152 to obtain the preprocessed fMRI data matrix, the dimension of the matrix is: ； in Is the size of The matrix of is the number of brain regions, is the number of time points; Indicates The time series of brain regions is , indicating that this brain region is A signal at a time point; In standard space, C-PAC divides the brain regions according to the AAL template and extracts the average signal of the time series of all voxels in the region for each brain region of interest: ； Indicates The time series of voxels, Indicates Regions of interest, It is The number of all voxels in a region; finally a time series matrix is ​​obtained, which represents the signal changes of each brain region.

3. The GNN-based multimodal brain imaging autism spectrum disorder detection method according to claim 2, characterized in that: In step S3, the method for calculating the edge-centered functional connectivity matrix is: The z-scores of the average time series extracted from all brain regions are normalized to evaluate the degree of deviation of the sample points from the overall mean, thereby achieving normalization of the data: ； in and is the average time series of the two brain regions, and yes and Two time series Fraction; For any two average time series extracted from brain regions, their element-wise product is calculated to obtain a new set of time series, called "marginal time series": ； ； In the formula, Represents brain area and brain regions The "marginal time series" between Represents brain area and brain regions The "marginal time series" between and Represents brain area and brain regions The z-score of the average time series; Compute the element-wise product between any two "edge time series" to obtain the final edge-centric functional connectivity matrix: ； Where EFCN represents the obtained edge-centric functional connectivity matrix.

4. The method for detecting autism spectrum disorder based on multimodal brain imaging using GNN according to claim 3, characterized in that: In step S4, feature selection is performed on the edge-centered functional connectivity matrix to screen out key features that are closely related to ASD diagnosis, including: For the high-dimensional edge functional connectivity matrix of the subject, a random sampling method without replacement is used to comprehensively and evenly cover the feature subset; assuming that each time from Select from the features Features, each sampling is expressed as: ； in is from A set of randomly selected indices from features, ; Use the random forest algorithm to perform classification analysis on the sampled feature subsets to obtain the importance ranking of each diagnostic feature: ； in, represents the importance score; Indicates Tree, features of Impurity reduction; The importance ranking of each feature is calculated : ； Among them, sort represents the sorting function; The top 10%, 20%, and 30% of features were selected and included in the final feature set to provide data support for subsequent classification and brain region localization.

5. The method for detecting autism spectrum disorder based on multimodal brain imaging using GNN according to claim 4, characterized in that: In step S6, the method of taking the preprocessed fMRI features and clinical text features as input and sending them to GNN for joint learning and classification is as follows: The final feature set extracted is used as the patient's node features, and combined with the text features obtained through clinical diagnosis to construct an adjacency matrix, which is used as edge features and node features to form a multimodal graph; The multimodal graph is fed into GNN for processing, and the high-level features of the brain network are captured through GNN node embedding learning; GNN optimizes node representation by passing information and finally obtains the embedding vector of each node : ； in, Is a node In the The layer representation, Is a node The set of neighbor nodes of is the normalization coefficient, usually the degree matrix of the adjacency matrix or other forms of weights, It is The weight matrix of the layer, is the activation function; After completing the GNN learning, the shared feature award is input into the fully connected classifier for ASD classification prediction. The output of the fully connected layer It is expressed as: ； in, is the weight matrix of the output layer, is the bias term, Is a node prediction labels; finally, the prediction results are output and a personalized diagnosis report is generated to provide support for clinical decision-making.

Images