Brain map recognition method, system, computer and storage medium
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- UNIV OF ELECTRONICS SCI & TECH OF CHINA
- Filing Date
- 2023-11-29
- Publication Date
- 2026-07-21
AI Technical Summary
Existing technologies lack objective methods for autism assessment, subjective judgments affect assessment results, and most autism identification methods rely on deep information learning from static brain maps, failing to fully utilize dynamic brain region relationships.
We employ a deep learning approach based on dynamic brain maps, extracting BOLD signal features through a time-frequency domain encoder, combining self-attention mechanisms and graph neural networks to construct a dynamic brain map structure, and using a multilayer perceptron for autism diagnosis.
It improves the accuracy and reliability of autism diagnosis by acquiring deeper node and edge features through dynamic mind map learning, and constructs an efficient autism identification classifier.
Smart Images
Figure CN117648624B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of medical intelligent diagnostic technology, and specifically relates to an image recognition technology. Background Technology
[0002] Autism spectrum disorder (ASD) is a highly heterogeneous neurodevelopmental syndrome. Its core symptoms include social communication impairments, sensory abnormalities, and repetitive sensorimotor behaviors. Although the global prevalence of autism is less than 1%, it places a tremendous economic and emotional burden on families, as most autistic individuals are unable to work full-time or live independently. Treatment for autism requires early diagnosis and intervention to alleviate symptoms and strive for a better prognosis.
[0003] Accurate assessment of autism is crucial for guiding targeted rehabilitation treatment of autistic patients. However, current methods for identifying (including diagnosis and assessment) autistic patients still lack objective physiological means, and clinical practice primarily relies on behavioral observation and scale assessments. The subjective biases of scale respondents, the instability of the patient's psychological state during the assessment, and the subjective judgment of clinicians can all significantly influence the results. Therefore, it is essential to develop a more accurate and objective mind mapping method.
[0004] Functional magnetic resonance imaging (fMRI) is a non-invasive technique for studying brain function. It is primarily used to measure blood-oxygen-level-dependent (BOLD) levels or blood flow in the brain, thereby helping to simulate the functional and structural mechanisms of the human brain. In recent years, the combination of fMRI images and machine learning methods has been widely used in brain map recognition research. Specifically, a statistical measure (Pearson correlation coefficient or mutual information) is used to describe the functional connectivity between BOLD signals in different brain regions, while machine learning methods are used to explore the relationship between brain region function and autism. Finally, a classifier for autism auxiliary diagnosis is constructed through large-scale data learning.
[0005] Graph Neural Networks (GNNs) are deep neural networks capable of learning representations of graph-structured data. Because GNNs can explicitly capture the topological information of brain networks and learn graph representations using message passing and information aggregation, they are well-suited for studying and analyzing relationships between different brain regions. Currently, most GNN recognition methods using fMRI data rely primarily on learning depth information from static brain maps (which do not change over time), and the representation of features for each brain region node is relatively simple. Summary of the Invention
[0006] To address the aforementioned technical problems, this invention proposes a brain map recognition method and medium based on dynamic brain map deep learning. It learns dynamic brain map information from fMRI data in a supervised manner, extracts BLOD signal waveform features based on a time-frequency domain encoder, learns and establishes embedded representations of brain region node features, calculates the correlation between BLOD signal waveforms, and establishes embedded representations of edge features between brain regions. This leads to the construction of a dynamic brain map structure. Furthermore, it proposes a self-attention-based dynamic brain map adjacency matrix update mechanism and a learnable graph pooling operation to further improve the accuracy of brain map recognition results.
[0007] One of the technical solutions adopted in this invention is: a mind map recognition method based on dynamic mind map deep learning, comprising:
[0008] S1. For the input static functional magnetic resonance imaging data, preprocessing is performed. The preprocessing specifically includes: skull removal, slice time correction, motion correction, global average intensity normalization, interference signal regression, and bandpass filtering.
[0009] S2. Register the static functional magnetic resonance imaging data preprocessed in step S1 to the cc200 space to obtain a time series for each brain region;
[0010] S3. Extract features from the time series of each brain region to obtain the node features of each brain region;
[0011] S4. Calculate the Pearson correlation coefficient for the time series between brain regions to obtain the edge features between each brain region;
[0012] S5. Using the node features of each brain region and the edge features between brain regions as input, the node features and edge features of brain regions at different scales are obtained through a set of graph neural network blocks that introduce an attention mechanism for in-depth learning.
[0013] S6. The node features and inter-brain boundary features of brain regions at different scales are spliced together to obtain the spliced features;
[0014] S7. Using the concatenated features as input, train the multilayer perceptron, continuously adjust the multilayer perceptron model parameters, and finally obtain the trained multilayer perceptron.
[0015] S8. Input the stitched features obtained after processing the static functional magnetic resonance imaging data to be identified in steps S1-S7 into the trained multilayer perceptron to obtain the identification result.
[0016] The second technical solution adopted in this invention is: a brain map recognition system based on dynamic brain map deep learning, including: a preprocessing module, a brain region registration module, a time-domain-frequency domain encoder, a correlation calculation module, a GNN module, a stitching module, and a multilayer perceptron;
[0017] The preprocessing module is used to process the input static functional magnetic resonance imaging data, including: skull removal, slice time correction, motion correction, global average intensity normalization, interference signal regression, and bandpass filtering.
[0018] The brain region registration module is used to register the preprocessed static functional magnetic resonance imaging data to the cc200 space, and each brain region obtains a time series.
[0019] The time-domain-frequency encoder is used to extract features from the time series of each brain region to obtain the node features of each brain region;
[0020] The correlation calculation module is used to calculate the Pearson correlation coefficient of the time series between brain regions to obtain the edge features between each brain region.
[0021] The GNN module is used to extract node features and edge features between brain regions at different scales based on the node features of each brain region and the edge features between brain regions input.
[0022] The stitching module is used to stitch together the node features and inter-brain boundary features of brain regions at different scales;
[0023] The multilayer perceptron takes the stitching features output by the stitching module as input for training. Finally, the trained multilayer perceptron outputs the recognition result based on the stitching features obtained from the input static functional magnetic resonance imaging data to be recognized.
[0024] The third technical solution adopted in this invention is: a computer device, including a processor and a memory for storing processor-executable programs, wherein when the processor of the computer executes the program stored in the memory, it realizes the above-mentioned brain map recognition task based on dynamic brain map deep learning.
[0025] The fourth technical solution adopted in this invention is: a storage medium storing a program, which, when executed by a processor, realizes the above-mentioned brain map recognition task based on dynamic brain map deep learning.
[0026] The beneficial effects of this invention are as follows: The method proposed in this invention designs and employs an encoder capable of extracting time-domain and frequency-domain information from BOLD signals, uses a dynamic brain map self-attention mechanism to update the brain region connectivity matrix, and designs and proposes a sparsification operation for the brain region adjacency matrix. Extensive experiments were conducted on 871 subjects at 17 sites based on the ABIDE dataset. The results show that the method proposed in this invention can encode resting magnetic resonance imaging (rs-fMRI) images in both the time and frequency domains to obtain an embedded representation of the brain map. Through deep learning of graph neural network blocks, deeper node and edge features of the dynamic brain map can be obtained. Through training and learning with a multilayer perceptron, an autism diagnosis and recognition classifier is constructed, achieving higher accuracy evaluation results. Attached Figure Description
[0027] Figure 1 The flowchart of the deep learning-based mind map recognition method for dynamic mind maps;
[0028] Figure 2 This is a brain map recognition framework based on dynamic brain map deep learning;
[0029] Figure 3 It is a graph neural network block that can learn dynamic brain map features through deep learning. Detailed Implementation
[0030] To facilitate understanding of the technical content of this invention by those skilled in the art, the following description, in conjunction with the accompanying drawings, further illustrates the invention.
[0031] like Figure 2 As shown, the brain map recognition method based on dynamic brain map deep learning proposed in this invention mainly consists of six parts: preprocessing, brain region registration, time-domain-frequency domain encoder, correlation calculation, a set of graph neural network blocks (GNN blocks), and a multilayer perceptron. Its workflow is as follows: Figure 1 As shown, it includes:
[0032] For the input static functional MRI (rs-fMRI) data, the preprocessing module first preprocesses the rs-fMRI data, including skull removal, slice time correction, motion correction, global average intensity normalization, interference signal regression, and bandpass filtering (0.01-0.1Hz). Then, the brain region registration module registers the rs-fMRI to the CC200 space. Each brain region yields a time series (i.e., a BOLD signal sequence), totaling 200 BOLD (Blood Oxygen Level Dependent) signal sequences. Next, the time-domain-frequency encoder extracts features from the BOLD signal sequence of each brain region to obtain the node features of each brain region; the correlation calculation module calculates the Pearson correlation coefficient between the BOLD signal sequences of brain regions to obtain the edge features between each brain region; then, the embedded representation of the brain map (node features and edge features of each brain region) is used as input and is deeply learned through a set of GNN modules to obtain node features and edge features of brain regions at different scales; the multi-scale node features and edge features of brain regions are used as input and are trained and learned through a multilayer perceptron, continuously adjusting the multilayer perceptron model parameters, and finally obtaining a trained multilayer perceptron (classifier); using the trained multilayer perceptron, given rs-fMRI diagnostic identification of new samples, the end-to-end identification and diagnostic results are output, namely: ASD or normal reference.
[0033] The implementation process of each module is described in detail below:
[0034] 1. Time-frequency encoder
[0035] The time-domain to frequency-domain encoder is mainly divided into two parts: the first part is the frequency domain feature Z. F,i Extract, define * as the convolution operation, B as the learnable parameter, a i for The amplitude, where DFT is the Discrete Fourier Transform (DFT). Frequency domain characteristic Z. F,i The extraction steps are as follows:
[0036] 1) Using the BOLD signal sequence {x i} is the input, for each time x i Perform a discrete Fourier transform: v i =DFT(x) i ), to obtain v i ;
[0037] 2) For each time v i Perform convolution operations: get
[0038] 3) Extraction Amplitude: Save as Z F,i =[a i ].
[0039] The second part is the time-domain feature X. T,i Extraction, namely: achieved through a one-dimensional convolutional neural network (1D-CNN);
[0040] The BOLD signal sequence of brain region i is processed by a time-domain to frequency-domain encoder, and the extracted frequency-domain and time-domain features are concatenated in the form of [Z]. F,i ;X T,i ].
[0041] The input to the time-frequency encoder is 200 BOLD sequences. After passing through the time-frequency encoder, the output is 200 time-frequency features of length 16, which correspond to 200 brain regions.
[0042] 2. Graph Neural Network Blocks Introducing Attention Mechanisms
[0043] like Figure 3 The graph neural network block (GNN block) with an attention mechanism shown is a deep learning model. The method used in this invention employs five GNN blocks to continuously extract deep features from the brain map features, extracting node features and edge features at different scales at each stage.
[0044] For the i-th GNN Block, its input is the node features Z from the previous layer. i-1 Sum of edge features X i-1 Its output is the node feature Z. i Sum of edge features X i The deep feature extraction steps are as follows:
[0045] 1) Input the Z-features of the brain regions in the previous layer. i-1 Sum of edge features X i-1 ;
[0046] Z0 consists of 200 node features of length 16 extracted by the time-frequency encoder, and X0 consists of edge features calculated from 200 BOLD sequences using Pearson correlation, i.e., a 200*200 functional connectivity matrix. Z0 and X0 can constitute an embedded representation of the brain map structure.
[0047] 2) Introduce an attention-based adjacency matrix to update the brain map adjacency matrix A. i The specific operation is as follows:
[0048] 2.1) For Z i-1 Performing a 1D convolution operation to obtain Q, and setting K = Q, can be represented as:
[0049]
[0050] 2.2) For X i-1 Performing a 1D convolution operation to obtain V can be represented as
[0051]
[0052] 2.3) Calculate the adjacency matrix A of brain regions. i , can be represented as
[0053] A i =Softmax(Q·K) T )⊙V (3)
[0054] 3) For the brain region adjacency matrix A i Perform sparsification to obtain A' i , can be represented as
[0055] A' i =Sparsify(A i (4)
[0056] 4) For the brain region adjacency matrix A' i Perform graph deep learning (graph convolution) to extract new node features Z. i Sum of edge features X i The specific operation is as follows:
[0057] 4.1) Perform graph learning on the node features to extract new node features Z. i , can be represented as
[0058] Z i =σ(A' i Z i-1 W i Z )+Z i-1 (5)
[0059] Among them, Z i-1 Z i ∈R N×N These are the brain region node features before and after graph convolution, W. i Z ∈R are the weight parameters of the graph neural network.
[0060] 4.2) Perform graph learning on the edge features to extract new edge features X. i , can be represented as
[0061]
[0062] Among them, X i-1 ,X i ∈R N×N These are the brain region edge features before and after graph convolution, W. i X ∈R are the weight parameters of the graph neural network.
[0063] After each GNN block is processed, the brain region adjacency matrix A i The brain map structure is dynamically updated. Compared to static graph neural networks (GNNs), the proposed method introduces attention-based graph neural network blocks to perform deep learning on brain region node and edge features. Based on the temporal and frequency domain information of the brain map's bound signal, dynamic graph convolutions can help to better learn the embedded representation of the brain map. In this method, each GNN block uses skip connections for graph convolutions on edges and nodes, resulting in deeper GCNs and stronger learning capabilities.
[0064] Figure 3 In this context, Norm represents the normalization operation, Add represents the addition operation, Graph Conv represents the graph convolution operation, Sparsify represents the sparsification operation, and Attention-based Adjacent Matrix represents the adjacency matrix of the self-attention mechanism.
[0065] 3. Sparse brain region adjacency matrix
[0066] Brain region adjacency matrix A i Represented as a fully connected matrix, it describes the functional connectivity between nodes in various brain regions. Using fully connected graph neural networks to learn downstream tasks is computationally expensive and susceptible to noise. To address these issues, this invention studies and proposes a method for representing the adjacency matrix A of brain regions. i The sparsification operation is performed, which can be represented by the Sparsify() function, whose input is the adjacency matrix A. i After sparsification, its output is the updated adjacency matrix A'. i The sparsity operation sets a threshold operator. Forced adjacency matrix edge connections are sparse. Threshold operator. It can be represented as
[0067]
[0068] Wherein, Sigmoid(θ) p )∈(0,1) is a learnable edge weight threshold, a i,j Represents the adjacency matrix A iThe value of the element in the i-th row and j-th column, abs(a i,j ) indicates that for a i,j Take the absolute value.
[0069] When abs(a i,j )≤Sigmoid(θ p )hour, The result is close to 0. This operation will result in more zero elements and fewer non-zero elements in the adjacency matrix, so that the adjacency matrix only retains important features and ignores non-important features. That is, the matrix is characterized by sparsity of non-zero elements.
[0070] At the same time, in order to ensure the threshold Sigmoid(θ) p ) Initially close to 0, θ p Initialize it to -10; this initialization can prevent A l Excessive sparsity in the early stages of training. Compared to other methods that rely on absolute thresholds, the sparsity operation of the brain region adjacency matrix proposed in this invention can learn the optimal threshold for a specific task more smoothly during training.
[0071] A highly readable result layer (Readout)
[0072] This layer will combine the node features and edge features from each layer to obtain the final feature h. G , can be represented as
[0073]
[0074] Among them, Z l X l Z represents the node features and edge features of the l-th layer, respectively. l , This represents the weights and biases of the l-th layer network.
[0075] 4-layer perceptron classifier
[0076] Training and testing phase: h G The data is fed into a multilayer perceptron (MLP), and the weight parameters of the MLP classifier are trained using labeled data to obtain a trained MLP classifier.
[0077] Application phase: Given actual test data, output the classification results of autism diagnosis and identification.
[0078] During the training and testing phases, to avoid model overfitting, label smoothing was designed and implemented, and its calculation formula is as follows:
[0079]
[0080] The expression for calculating the loss function is as follows:
[0081]
[0082] Where W is the weight of the Readout layer and the classification layer, K=2 is the number of categories, and y k ∈{0,1} is the label. These are the corresponding smoothed labels, where α and λ are hyperparameters. We set α to 0.1 and λ to 0.0001, p k It is the probability of predicting category k.
[0083] This embodiment uses the ABIDE dataset to evaluate the model.
[0084] The ABIDE dataset aggregates data from 17 different acquisition sites, containing static functional magnetic resonance imaging (fMRI) images and phenotypic data from 1112 subjects. After preprocessing, 871 high-quality MRI images with phenotypic information were obtained, including those from 403 ASD patients and 468 normal controls.
[0085] To mitigate overfitting and provide a generalizable assessment of classification performance, 5-fold cross-validation is used for classification evaluation.
[0086] The experimental results are shown in Table 1. Compared with the random forest method and the graph convolution-based method, the method proposed in this invention is significantly better than the other two methods in terms of accuracy and sensitivity.
[0087] Accuracy (ACC) represents the proportion of correctly predicted samples out of the total sample size.
[0088]
[0089] Specific SPE represents the proportion of individuals correctly predicted to have autism out of the total number of actual autistic individuals.
[0090]
[0091] Sensitivity (SEN) represents the proportion of the number of individuals correctly predicted to not have autism out of the total number of individuals who do not actually have autism.
[0092]
[0093] Table 1 Experimental Results
[0094]
[0095] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Various modifications and variations can be made to the invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the scope of the claims of the invention.
Claims
1. A mind map recognition method based on dynamic mind map deep learning, characterized in that, include: S1. For the input static functional magnetic resonance imaging data, preprocessing is performed. The preprocessing specifically includes: skull removal, slice time correction, motion correction, global average intensity normalization, interference signal regression, and bandpass filtering. S2. Register the static functional magnetic resonance imaging data preprocessed in step S1 to the cc200 space to obtain a time series for each brain region; S3. Extract features from the time series of each brain region to obtain the node features of each brain region; S4. Calculate the Pearson correlation coefficient for the time series between brain regions to obtain the edge features between each brain region; S5. Using the node features of each brain region and the edge features between brain regions as input, a set of graph neural network blocks with an attention mechanism are used for deep learning to obtain the node features and edge features of brain regions at different scales; the process of deep feature extraction for each graph neural network block with an attention mechanism is as follows: A1. Introduce an attention mechanism to update the mind map adjacency matrix. ; i is the index of the graph neural network block that introduces the attention mechanism; A11. Brain region node features of the graph neural network block that currently incorporates an attention mechanism for the input. Perform a 1D convolution operation to obtain and order , is represented as: ； in, This represents a 1D convolution operation; A12. Brain region edge features of graph neural network blocks that currently incorporate attention mechanisms for input. Perform a 1D convolution operation to obtain , is represented as: ； A13. Calculate the adjacency matrix of brain regions. , is represented as: ； in, Represents the normalized exponential function, The symbol represents the transpose of K, and · represents matrix multiplication. This represents the element-wise multiplication operation of a matrix. A2. Adjacency matrix of brain regions Perform sparsification to obtain , represented as ； in, This indicates sparsification. A3. Adjacency matrix of brain regions Deep learning based on graph neural networks is used to extract new node features. Sum of edge features The specific operation is as follows: A31. Based on graph neural networks, deep learning is used to extract new node features. , represented as ； in, These are the brain region node features before and after graph convolution. These are the weight parameters of a graph neural network. Represents the nonlinear activation function ReLU; A32. For edge features, new edge features are extracted using deep learning based on graph neural networks. , represented as ； in, These are the brain region edge features before and after graph convolution. These are the weight parameters of a graph neural network; S6. The node features and inter-brain boundary features of brain regions at different scales are spliced together to obtain the spliced features; S7. Using the concatenated features as input, train the multilayer perceptron, continuously adjust the multilayer perceptron model parameters, and finally obtain the trained multilayer perceptron. S8. Input the stitched features obtained after processing the static functional magnetic resonance imaging data to be identified in steps S1-S7 into the trained multilayer perceptron to obtain the identification result.
2. The mind map recognition method based on dynamic mind map deep learning according to claim 1, characterized in that, Step S3 specifically includes: S31. Extract frequency domain features from the time series data of each brain region; S32. Extract temporal features from the time series of each brain region; S33. The frequency domain features extracted in step S31 are concatenated with the time domain features extracted in step S32 to obtain the node features of each brain region.
3. The mind map recognition method based on dynamic mind map deep learning according to claim 1, characterized in that, Step S5 specifically involves: using multiple graph neural network blocks with an attention mechanism to continuously extract deep features, extracting node features and edge features at different scales at each stage; the input of the first graph neural network block with an attention mechanism is the node features of each brain region obtained in step S3 and the edge features between brain regions obtained in step S4; the input of subsequent graph neural network blocks with an attention mechanism is the node features of each brain region and the edge features between brain regions extracted by the previous graph neural network block with an attention mechanism.
4. The mind map recognition method based on dynamic mind map deep learning according to claim 2, characterized in that, Step A2 The input is the adjacency matrix. After sparsification, its output is the updated adjacency matrix. ; The sparsity operation sets a threshold operator. The forced adjacency matrix edge connections are sparse; threshold operator Represented as ； in, It is a learnable edge weight threshold. , Representing the adjacency matrix The Middle OK Column element values, Indicates to Take the absolute value.
5. The mind map recognition method based on dynamic mind map deep learning according to claim 1, characterized in that, The splicing feature is represented as : ； in, , They represent the first The node and edge features output by a graph neural network block that incorporates an attention mechanism. , , , Indicates the first The network weights and biases of a graph neural network block that incorporates an attention mechanism.
6. A mind map recognition system based on dynamic mind map deep learning, characterized in that, include: Preprocessing module, brain region registration module, time-domain-frequency encoder, correlation calculation module, GNN module, splicing module, multilayer perceptron; The preprocessing module is used to process the input static functional magnetic resonance imaging data, including: skull removal, slice time correction, motion correction, global average intensity normalization, interference signal regression, and bandpass filtering. The brain region registration module is used to register the preprocessed static functional magnetic resonance imaging data to the cc200 space, and each brain region obtains a time series. The time-domain-frequency encoder is used to extract features from the time series of each brain region to obtain the node features of each brain region; The correlation calculation module is used to calculate the Pearson correlation coefficient of the time series between brain regions to obtain the edge features between each brain region. The GNN module is used to extract node features and edge features between brain regions at different scales based on the node features of each brain region and the edge features between brain regions input. The stitching module is used to stitch together the node features and inter-brain boundary features of brain regions at different scales; The multilayer perceptron takes the stitching features output by the stitching module as input for training. Finally, the trained multilayer perceptron outputs the recognition result based on the stitching features obtained from the input static functional magnetic resonance imaging data to be recognized.
7. A computer device, characterized in that... The computer includes a processor and a memory for storing processor-executable programs. When the processor executes the program stored in the memory, it implements the mind map recognition task based on dynamic mind map deep learning as described in any one of claims 1-5.
8. A storage medium storing a program that, when executed by a processor, performs the mind map recognition task based on dynamic mind map deep learning as described in any one of claims 1-5.