A mild cognitive impairment detection method based on combination of local and global spatiotemporal features

Abstract

The application discloses a mild cognitive impairment detection method based on combination of local and global spatiotemporal features. In recent years, the use of brain functional connectivity state for mild cognitive impairment detection has achieved certain results, however, the existing detection method extracts and fuses spatiotemporal features in a relatively simple way. The application proposes an adaptive brain region signal segmentation method, which regards brain region signal segmentation as a maximum likelihood estimation problem of Gaussian distribution, and adaptively divides different segments according to different brain connection states, overcoming the problem that the sliding window segmentation method cannot accurately segment. The application proposes a graph Transformer framework that fuses local and global spatiotemporal features, and the designed framework can effectively model local and global spatiotemporal features, thereby realizing accurate mild cognitive impairment detection. Experimental results show that the method can fully utilize local and global spatiotemporal information to accurately detect mild cognitive impairment.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing and relates to a method for detecting mild cognitive impairment based on a combination of local and global spatiotemporal features. Technical Background

[0002] Alzheimer's disease (AD) is a neurodegenerative disease that severely affects a patient's memory and other cognitive abilities, ultimately leading to a loss of independence. Although research in the field of AD has made some progress, there are still no effective treatments or drugs that can stop or reverse the progression of AD. Mild cognitive impairment (MCI) is a precursor stage of AD. Patients in this stage experience some cognitive impairment, but it does not affect their daily lives. Studies have shown that early intervention in patients at the MCI stage can effectively slow the progression of the disease and even restore normal cognitive levels. Therefore, current AD prevention and control mainly relies on early detection and intervention in patients at the MCI stage. Accurate detection of mild cognitive impairment is a prerequisite for early intervention and has significant clinical and social implications.

[0003] Currently, imaging examinations are the primary method used in clinical practice to detect Mild Cognitive Impairment (MCI). Commonly used imaging techniques include structural magnetic resonance imaging (SMRI) and functional magnetic resonance imaging (fMRI). SMRI displays the shape and degree of atrophy in different brain regions, while fMRI shows the activity and connectivity between these regions. Studies have shown that the level of brain atrophy in the MCI stage may be small and difficult to observe, while the connectivity between brain regions at this stage differs significantly from that in normal individuals. Therefore, fMRI is effective and widely used in the detection of MCI.

[0004] Functional magnetic resonance imaging (fMRI) measures blood oxygen level-dependent (BOLD) sequences in brain regions to reflect the activity of each region. Current research typically calculates the correlation coefficients of BOLD sequences across different brain regions to obtain the connectivity structure of brain functional networks. Existing methods for mechanical brain imaging (MCI) using fMRI data can be mainly divided into two categories: static brain network methods and dynamic brain network methods. The former assumes that the functional connectivity structure of the brain remains unchanged throughout the scan, primarily utilizing spatial information for detection; while the latter assumes that the functional connectivity structure of the brain changes over time, utilizing spatiotemporal information for detection. Current research shows that static brain network methods cannot accurately model the functional connectivity of the brain, as the functional connectivity structure of the brain is actually constantly changing. Therefore, recent MCI detection methods have mainly shifted towards dynamic brain network methods. However, current dynamic brain network methods have two main problems. First, existing dynamic brain network methods typically use sliding windows to segment the entire BOLD sequence, with different segments yielding different brain functional networks. The sliding window method is limited by its parameter selection; too small a window parameter makes it difficult to estimate a robust brain network, while too large a window parameter fails to accurately capture the temporal changes within it. Secondly, existing dynamic brain network methods handle spatiotemporal features in a relatively simple way. Most methods first extract the spatial information of each segment, and then obtain the time-varying patterns by analyzing the differences in spatial features between different segments. This relatively simple approach does not adequately model spatiotemporal features. The temporal features in dynamic brain networks are reflected not only in the transformation of global temporal patterns between segments, but also in the local temporal patterns within each segment.

[0005] Therefore, there is an urgent need for an accurate segmentation of BOLD sequences and a method that integrates local and global spatiotemporal features to achieve accurate detection of MCI by utilizing the rich spatiotemporal features in functional images. Summary of the Invention

[0006] The technical problem to be solved by this invention is to propose a mild cognitive impairment detection method based on the combination of local and global spatiotemporal features, which can solve the problems of difficulty in selecting sliding window parameters and insufficient utilization of spatiotemporal features, thereby improving the performance of MCI detection.

[0007] The present invention provides a method for detecting mild cognitive impairment based on a combination of local and global spatiotemporal features, comprising the following steps:

[0008] Step 1: Preprocessing of the functional magnetic resonance imaging (fMRI) data in the dataset mainly includes magnetization balancing, temporal correction, head motion correction, registration, spatial normalization, and template segmentation. Magnetization balancing is achieved by removing the first ten volumes of scans. Temporal correction and head motion correction are standard preprocessing procedures for improving data quality. Due to differences in brain morphology among different subjects, the functional images of all subjects need to be aligned and mapped to a standard space to ensure comparability. Since the most basic unit in functional imaging data is the voxel-level BOLD signal, to obtain the brain region-level BOLD signal, voxels need to be divided into different brain regions according to a predefined brain atlas. The BOLD signal of a brain region is the average of the BOLD signals of its constituent voxels. After completing the preprocessing of the fMRI data, each subject has brain region-level BOLD signals and demographic data. These processed data are further divided into training and testing sets.

[0009] Step 2: The BOLD signal of each subject's brain region obtained in Step 1 is input into the adaptive brain region signal segmentation module. This module is used to divide the entire BOLD signal into multiple segments corresponding to brain functional states. The role of this module in this invention is to adaptively determine the length of each segment to accurately obtain the evolutionary information of the dynamic brain network.

[0010] The adaptive brain region signal segmentation module consists of a signal segmentation module and a segment selection module;

[0011] The signal segmentation module is based on multivariate Gaussian distribution and greedy Gaussian segmentation algorithm, and is used to divide the signal into different segments according to the brain connectivity state;

[0012] The segment selection module is based on JS divergence and TopK algorithm and is used to select each segment after segmentation to remove redundant segments.

[0013] Step 3: Input the selected segments obtained in Step 2 into our local spatiotemporal feature aggregation module. This module accepts both spatial and temporal feature matrices as inputs, where the spatial input consists of the segments obtained in Step 2, and the temporal input is the transpose of these segments. This module is used to extract the spatial and temporal features within each segment and aggregates the spatial features using the temporal features.

[0014] The local spatiotemporal feature aggregation module consists of a spatial encoding module, a temporal encoding module, and a spatiotemporal fusion module.

[0015] The spatial encoding module is based on the Graphormer encoder. It constructs a topological structure based on the spatial features of each segment and integrates the topological structure into the calculation of the attention mechanism to obtain a higher-level spatial embedding for each segment.

[0016] The time encoding module is based on a Transformer encoder and performs attention calculations based on the time features of each segment to obtain the time embedding within the segment.

[0017] The spatiotemporal fusion module is also based on the Transformer encoder, which fuses the temporal and spatial features corresponding to each segment to obtain the local spatiotemporal features of each segment.

[0018] Step 4: Input the segmentation results for each subject obtained in Step 2 and the local spatiotemporal features of each segment obtained in Step 3 into our global temporal encoding module. This module is used to encode the temporal variation information between each segment into the local spatiotemporal features of the segment to obtain more comprehensive spatiotemporal features;

[0019] The global time encoding module consists of a global time information extraction module and a multi-head attention module;

[0020] The global time information extraction module obtains the global time features between each segment based on the difference of the multivariate Gaussian distribution of the segments and a feedforward neural network. The feedforward neural network is used to embed the difference vector of the multivariate Gaussian distribution with the local spatiotemporal features of the segments to unify the dimensions.

[0021] The multi-head attention module treats global temporal features as positional codes and fuses them with the local spatiotemporal features of their corresponding segments. Then, multi-head attention fusion is used to fuse the spatiotemporal features of multiple segments, and the resulting concatenation is used as the embedding output for each subject.

[0022] Step 5: Input the demographic data obtained in Step 1 and the embeddings of each subject obtained in Step 4 into a population graph network. Each subject is a node in the graph, and the node features are the subject's embeddings. The weights of the edges between nodes are obtained from the correlation between the demographic data and the correlation between the embeddings. Convolution on the population graph can effectively aggregate the embeddings based on the correlation between subjects. The stronger the correlation, the closer the embeddings of subjects will be in the feature space. This convolution method effectively integrates the embeddings obtained from demographic data and imaging into a unified graph learning framework to extract more accurate subject embeddings. After passing the embedding of each subject through a multilayer perceptron, the logits values ​​of each subject for each category can be obtained.

[0023] Step 6: Convert the logits values ​​for each subject and category obtained in Step 5 into probability values ​​using the softmax function, and input them along with the label information into the loss function. Optimize the parameters of the modules in Steps 3, 4, and 5 based on the gradient of the loss value calculated by the loss function. The loss function is the cross-entropy loss function, which is defined as follows:

[0024]

[0025] Where N is the number of samples, y i p is the true label of sample i. i This represents the model's prediction results.

[0026] Step 7: Input the test set divided in Step 1 into the trained model to obtain the detection performance of the model on the test set.

[0027] This invention overcomes the parameter selection problem of the sliding window segmentation method by using an adaptive brain region signal segmentation module, resulting in segments that more accurately represent dynamic brain networks. Accurate detection of Mid-Cyclical Interference (MCI) is achieved by extracting local and global spatiotemporal features within and between segments. Experimental results show that our model can effectively utilize the spatiotemporal features of dynamic brain networks to improve the performance of MCI detection. Attached Figure Description

[0028] Figure 1 This is a flowchart illustrating the method of the present invention.

[0029] Figure 2 This is a schematic diagram of the method of the present invention.

[0030] Figure 3 This is a schematic diagram of the local spatiotemporal feature fusion module of the method of the present invention.

[0031] Beneficial effects

[0032] The mild cognitive impairment detection model constructed in this invention, based on a combination of local and global spatiotemporal features, is an end-to-end deep learning network. It first segments the entire BOLD signal into multiple adaptively determined segments using the multivariate Gaussian distribution of brain region signals at each time point. Redundant segments are then filtered out using the JS divergence between segments. The advantage of this adaptive brain region signal segmentation is that it overcomes the problem of manually determining window parameters required by traditional sliding window methods, and the brain region signals segmented by this adaptive method are more robust. The local spatiotemporal feature fusion module models the temporal and spatial features within each segment and fuses them, enabling a better acquisition of the spatiotemporal pattern corresponding to each brain connectivity state. The global temporal encoding module can acquire the time-varying relationships between segments and integrate these global changes into the local spatiotemporal features, improving the comprehensiveness and robustness of the spatiotemporal features. Detailed Implementation

[0033] like Figure 1 As shown, the mild cognitive impairment detection method proposed in this invention, based on a combination of local and global spatiotemporal features, mainly includes the following steps:

[0034] Step 1: Preprocess the images in the dataset and divide them into training and test sets;

[0035] Routine preprocessing of functional magnetic resonance imaging (fMRI) data mainly includes magnetization balancing, time correction, head movement correction, registration, spatial normalization, and template segmentation. Since the MRI equipment may not be stable shortly after startup, data from the first 10 time points of the scan need to be removed to ensure magnetization balancing. FMRI scans are typically performed in a layered manner, and the scan times for different slices vary, requiring time correction. Head movement correction aims to eliminate the influence of head movements during scanning to improve data quality. Due to differences in brain morphology among subjects, brain imaging data from all subjects need to be aligned and mapped to a standard space for comparability. Since fMRI images reflect voxel-level BOLD signals, to obtain brain region-level BOLD signals, voxels need to be divided into different brain regions according to a predefined brain atlas; the BOLD signal of a brain region is the average of the BOLD signals of its constituent voxels. After completing the preprocessing of the fMRI data, each subject has brain region-level BOLD signals and demographic data, and all subjects are randomly divided into training and test sets.

[0036] Step 2: Use the adaptive brain region signal segmentation module to segment the BOLD signal of each subject's brain region obtained in Step 1;

[0037] like Figure 2As shown, the adaptive brain region signal segmentation module consists of a signal segmentation module and a segment selection module. To obtain dynamic information about the brain network, the entire brain region BOLD signal needs to be divided into multiple segments, each segment independently constructing a graph structure. However, traditional sliding window segmentation methods are limited by the selection of window parameters, making it difficult to effectively segment the entire brain region signal and insufficiently capture temporal information. Therefore, the signal segmentation module is used to adaptively segment the brain region signal based on brain connectivity states. Simultaneously, during the entire scan process, each subject may have redundant brain connectivity states. We designed a segment selection module to select the multiple segments with the highest diversity to eliminate the influence of redundant segments.

[0038] The signal segmentation module is implemented based on a multivariate Gaussian distribution and a greedy Gaussian segmentation algorithm. This module adaptively divides the entire brain region signal into different segments according to the brain connectivity state at each time point. First, the signal vector for each brain region at each time point is generated by a specific brain connectivity state, which can be described using a multivariate Gaussian distribution, defined as follows:

[0039] x i ~N(μ) i , Σ i ),

[0040] Where x i μ represents the brain region signal vector at the i-th time point. i and Σ i This represents the mean vector and covariance matrix of the multivariate Gaussian distribution to which the signal vector follows. The segmentation problem can then be viewed as adding multiple segment points to a complete sequence, with the time points between two adjacent segment points approximately following the same multivariate Gaussian distribution. The optimization objective is to maximize the probability that each time point within each segment belongs to the same distribution. We used a greedy Gaussian distribution algorithm to solve this maximum likelihood problem. Since the algorithm's operation involves attempting to add new segment points to the previous segmentation result until the probability cannot be further increased, we set the number of segment points to a relatively large value to ensure that the segmentation result for each subject cannot be further subdivided.

[0041] The segment selection module, based on JS divergence and TopK selection algorithms, is used to filter the segmentation results for each subject to remove redundant segments. Each segment obtained by the signal segmentation module is also described using a multivariate Gaussian distribution.

[0042] seg i ~N(μ) i , Σ i ),

[0043] Among them, seg i This represents a segment obtained by the signal segmentation module. JS divergence can be used to describe the difference between two distributions. To obtain the most unique segments, the segment selection module calculates the pairwise JS divergence between all segments, forming a divergence matrix M. The calculation of each element in matrix M is as follows:

[0044] M(i,j)=D JS (seg i ||seg j ),

[0045] Where D JS () indicates calculating the JS divergence between two distributions. Then we calculate the sum of the JS divergences of each segment and all other segments:

[0046]

[0047] Where C i C represents the sum of the JS divergences of the i-th segment and the remaining segments. The larger the value, the more unique the segment is. Therefore, we selected the K segments with the largest C values ​​as subsequent inputs.

[0048] Step 3: Use the local spatiotemporal feature aggregation module to extract the spatial and temporal features of the segment selected in Step 2, and use the temporal features to guide the aggregation of spatial features;

[0049] The local spatiotemporal feature aggregation module consists of a spatial encoding module, a temporal encoding module, and a spatiotemporal fusion module. Each segment contains topological connectivity information between brain regions in the corresponding brain connectivity state, as well as time-varying patterns at the whole-brain connectivity level. The spatial and temporal features significantly contribute to MCI detection. The local spatiotemporal feature aggregation module models the spatial and temporal features of each segment separately and fuses these two sets of features to obtain the corresponding local spatiotemporal features for each segment.

[0050] The spatial encoding module is used to extract spatial features from each segment. This module treats brain regions as nodes, and the initial feature of each brain region is its corresponding BOLD signal sequence. Spatial features are extracted using the connectivity relationships between brain regions. This module is based on the Graphormer architecture, and the connectivity relationships between brain regions consist of two parts: one part depends on predefined connections, and the other part is calculated by the model's self-attention mechanism during training. For the predefined connectivity relationships, we first calculate the distance between each pair of brain regions. The distance between brain regions i and is calculated as follows:

[0051]

[0052] Where L is the number of time points in the segment. The graph is a two-dimensional spatial feature matrix with N brain regions and L time points. After obtaining the distances between each brain region and other brain regions, we use the K-nearest neighbor method for graph construction. Specifically, each brain region is connected only to its K-1 nearest neighbors. Finally, we calculate the pairwise shortest path matrix between nodes in the graph obtained by the K-nearest neighbor method, and then use a threshold to set all elements in the shortest path matrix greater than a set threshold to negative infinity. This thresholded shortest path matrix represents the predefined connectivity, which will participate in the subsequent calculation of the self-attention mechanism. In addition, we add a special node similar to the [CLS] token in a language model to each segment. This node aggregates information from all brain regions in the self-attention mechanism to represent the whole-brain spatial features of the segment.

[0053] The formula for calculating the self-attention mechanism in the module is as follows:

[0054]

[0055] in

[0056] Q = W q S,K=W k S,V=W v S,

[0057] P is the thresholded shortest path matrix.

[0058] The temporal encoding module is used to extract local temporal features for each segment. This module uses time points as nodes, and the initial feature of each time point is a sequence of signal values ​​from all brain regions at that time point. Since the time points are sequentially related, this module extracts temporal features based on a traditional Transformer encoder.

[0059] The spatiotemporal fusion module mentioned above is as follows Figure 3 As shown, this module is used to fuse the spatial features and local temporal features of each segment. First, it concatenates the special nodes extracted by the spatial coding module with the temporal features extracted by the temporal coding module and performs self-attention calculation. Then, it performs residual connection between the special nodes fused with temporal features and the original special nodes. The computational flow of the designed spatiotemporal fusion module is as follows:

[0060]

[0061]

[0062] Where τ represents the Transformer encoder, concat is the concatenation operation, and [:,:] represents the matrix slicing operation.

[0063] Step 4: Use the global temporal coding module to model the transition relationships between segments and fuse them with the local spatiotemporal features of each segment in the form of positional coding.

[0064] The global temporal encoding module consists of a global temporal information extraction module and a multi-head attention module. The transition relationships between each segment reveal the patterns of change between different brain connectivity states, providing a time-varying pattern from a global perspective, which can provide useful information for MCI detection. The global temporal encoding module first extracts the time-varying information between adjacent segments from the multivariate Gaussian distribution of the segments, and this time-varying information is regarded as the positional encoding of each segment.

[0065] The global time information extraction module is used to extract the global time features of each segment. The global time features of a segment consist of two parts: the change pattern from the previous state to the current state, and the change pattern from the current state to the next state. The calculation formulas for these two parts of the pattern features are as follows:

[0066] TE post,i =concat(μ i -μ i-1 ,vec u (∑ i )-vec u (∑ i-1 )),

[0067] TE next,i =concat(μ i+1 -μ i ,vec u (∑ i+1 )-vec u (∑ i )),

[0068] Among them TE post,i and TE next,i These represent the two components of the global temporal feature, μ i and Σ i Let vec and covariance matrix represent the mean vector and covariance matrix of the multivariate Gaussian distribution to which the segment follows, respectively. Since the covariance matrix of each segment is a symmetric matrix, we use its upper triangular matrix to simplify the amount of input data, where vec u () indicates the upper triangular operation on the matrix. The final global temporal feature calculation formula for each segment is as follows:

[0069] TE i =FFN(concat(TE) post,i ,TE next,i )),

[0070] Here, FFN stands for Feedforward Neural Network, which has two main functions: first, to further extract the calculated original global temporal features; and second, to align the dimensions of the global temporal features of a segment with the dimensions of the segment's local spatiotemporal features.

[0071] The multi-head attention mechanism takes the global temporal and local spatiotemporal features of each segment as input and aggregates the features of each segment through the correlation between segments. To reduce overfitting caused by excessive data dimensionality, we use special nodes for each segment to represent the local spatiotemporal features of that segment. The calculation process of this module is as follows:

[0072]

[0073]

[0074] Where ST represents the feature matrix composed of special nodes of each segment of a single subject, MHA is a multi-head attention layer, flatten is the matrix flattening operation, and X represents the embedding of each subject.

[0075] Step 5: Input the embeddings of each subject obtained in Step 4 and the demographic data of each subject obtained in Step 1 into the population graph network for aggregation, and output the prediction result for each subject. The population graph network treats each subject as a node, and the subject correlation coefficient matrix is ​​composed of the embeddings and demographic data, calculated using the following formula:

[0076]

[0077]

[0078] C(i,j)=Sim1(i,j)*Sim2(i,j),

[0079] Where ρ represents the calculation of Euclidean distance, σ represents the average of the above distances, and d i This represents the demographic data of subject i. The convolutional layers of the population graph network are implemented using second-order Chebyshev convolutional kernels. A multilayer perceptron is used at the end of each convolutional layer to obtain the logits value for each subject's corresponding category. Then, a softmax function is used to process the logits value, transforming it into the probability value for the subject's corresponding category. Finally, the above probability values ​​and the true label are input into a loss function to calculate the loss value. The loss value is used to fine-tune the network containing the parameters to be optimized in steps 3, 4, and 5. The loss function is the cross-entropy loss function, defined as follows:

[0080]

[0081] We use the Adam optimizer to update the gradients with an initial learning rate of 0.01 and a maximum epoch of 300.

[0082] Step 6: The mild cognitive impairment detection method based on the combination of local and global spatiotemporal features is similar to the node classification task in graph neural networks. The samples in the training set and the test set are in the same population graph, but only the loss value of the training set is used for optimization.

[0083] After training is complete, the trained network can be used to calculate the classification results of the test set.

[0084] Experimental verification

[0085] To verify the effectiveness of the detection model proposed in this invention, the following methods were used to compare it with the proposed model:

[0086] EVGCN: It is a graph neural network framework that uses subjects as graph nodes. It proposes an encoder to obtain the association score of subjects using their demographic data. The framework aggregates the features of subjects using their association to obtain the detection results.

[0087] BrainGNN: It is a graph neural network framework that uses brain regions as graph nodes. This framework uses an adaptive method to calculate the importance coefficient of each brain region at each graph convolution. At the end of each graph convolution layer, the brain regions are pooled, and the brain regions with higher importance are used for subsequent graph convolutions.

[0088] LGGNN: It proposes to build brain region map convolution and group map convolution into a unified graph learning framework. The method first uses brain region map convolution to obtain the embeddings at the subject level, and then aggregates the subject embeddings according to the group map convolution to obtain the detection results.

[0089] ContrastPool proposes a dual-graph contrastive learning framework. This method inputs the two class distributions in the training set into the network to obtain the attention of brain regions under different classes, and uses contrastive learning to obtain the difference information between different classes to obtain the detection results.

[0090] STGCN: It constructs a spatiotemporal graph with brain regions as nodes. Each time point corresponds to a graph topology. Nodes in the graph structure at adjacent time points can be connected to each other. This method designs a time-point-wise spatiotemporal graph convolution module to extract and detect the spatiotemporal features of each subject.

[0091] GATE proposes a self-supervised learning framework that uses adjacent sliding windows and multi-scale sliding windows for data augmentation to address the problem of pseudo-components obtained by the sliding window segmentation method.

[0092] MDGL: It uses a sliding window to segment the BOLD signal of multiple spectra and inputs the segmentation results under different spectra into two graph neural networks with independent parameters to extract information from multiple spectra. This method uses Transformer to fuse multiple spectra features for detection.

[0093] HFBN: It proposes an adaptive method for constructing brain maps to overcome the lack of flexibility of traditional predefined maps. This method constructs maps at multiple scales simultaneously and uses the spatiotemporal information within them for detection.

[0094] In the experiments, accuracy (ACC), area under the ROC curve (AUC), true positive rate (SEN), true negative rate (SPE), and F1 score (F1-score) were used to evaluate the detection performance of each method. ACC reflects the ability of the detection model to correctly predict the proportion of all samples, and its calculation formula is as follows:

[0095]

[0096] The AUC value measures the overall performance of a model at different thresholds and is an important metric in detection tasks. SEN reflects the model's ability to correctly identify all actual positive instances as positive; a higher SEN indicates a lower false negative rate. Its calculation formula is as follows:

[0097]

[0098] SPE reflects the model's ability to correctly identify all actual negative instances as negative. The higher the SPE, the lower the false positive rate. The calculation formula is as follows:

[0099]

[0100] The F1 score is a combined metric of precision and recall, and its calculation formula is as follows:

[0101]

[0102] in The calculation method for Recall is the same as that for SEN.

[0103] This experiment uses the publicly available datasets ADNI3 and ADNI2. Both the proposed method and the comparison method employ the same test set partitioning, and ten-fold cross-validation is used for evaluation on the test set. The experimental results are shown in Table 1, which reports the mean and standard deviation of all indicators under ten-fold cross-validation, where the numbers in parentheses represent the standard deviation.

[0104] Table 1: Detection performance of the proposed method and the comparison methods on the ADNI3 and ADNI2 datasets.

[0105]

[0106]

[0107] As can be seen, our proposed method achieves ACC, AUC, and F1 scores of 92.80 (11.40)%, 92.35 (10.50)%, and 93.62 (7.63)% respectively on the MCI detection task in the ADNI3 dataset, outperforming all comparable methods. Compared to the best static brain functional connectivity method, LGGNN, our proposed method extracts temporal information from functional magnetic resonance imaging data and combines it with spatial features, improving ACC and AUC by 7.86% and 5.82% respectively. Our method outperforms the best dynamic brain functional connectivity method, HFBN, demonstrating the effectiveness of our proposed adaptive brain region signal segmentation method and global-local temporal feature processing module.

[0108] Our proposed method achieves ACC of 81.89 (7.32)% and 82.80 (13.95)% on the eMCI and lMCI detection tasks on the ADNI2 dataset, respectively, outperforming other comparable methods. Notably, on the ADNI2 dataset, the performance gap between our proposed method and other methods narrows. This is likely because both our proposed method and HFBN utilize the Transformer architecture, which makes training robust models on the relatively small sample size of the ADNI2 dataset more challenging.

[0109] In summary, this invention proposes a method for detecting mild cognitive impairment that combines local and global spatiotemporal features. Experimental results show that this method can effectively achieve adaptive segmentation of functional magnetic resonance imaging data and make full use of the local and global temporal and spatial features to achieve more accurate detection of mild cognitive impairment.

Claims

1. A method for detecting mild cognitive impairment based on a combination of local and global spatiotemporal features, characterized in that, Includes the following steps: Step 1: Preprocess the functional magnetic resonance imaging data in the dataset and divide it into training and test sets; Step 2: The adaptive brain region signal segmentation module is used to segment the BOLD signal of each subject's brain region obtained in Step 1. This module consists of a signal segmentation module and a segment selection module. The signal segmentation module adaptively divides the entire brain region signal into multiple segments based on the correspondence between each signal point and the brain connectivity state. The segment selection module removes segments corresponding to redundant brain connectivity states to avoid overfitting caused by redundant data. Furthermore, the signal segmentation module performs segmentation based on a multivariate Gaussian distribution and a greedy Gaussian segmentation algorithm. The module treats the whole-brain signal vector at each time point as following a multivariate Gaussian distribution and the segmentation algorithm as adding segmentation points to a continuous sequence, making each continuous segment a maximum likelihood problem where the segments belong to the same distribution. The segment selection module uses JS divergence to characterize the differences between segments; the sum of the JS divergences of any segment and all other segments represents the uniqueness of that segment. The Topk method is used to select the segments with the highest uniqueness. Step 3: Use the local spatiotemporal feature aggregation module to extract the spatial and temporal features of the segment selected in Step 2, and use the temporal features to guide the aggregation of spatial features; Step 4: Use the global temporal coding module to model the global time-varying relationship of the brain region signal segments obtained in Step 2, and fuse it with the local spatiotemporal features of each segment obtained in Step 3 in the form of location coding; Step 5: Input the embeddings of each subject obtained in Step 4 and the demographic data of each subject obtained in Step 1 into the population graph network for training; Step 6: Test the trained network on the test set divided in Step 1 to obtain the detection results on the test set.

2. The method for detecting mild cognitive impairment based on a combination of local and global spatiotemporal features according to claim 1, characterized in that, In step 1, the preprocessing of functional magnetic resonance imaging data includes: magnetization balancing, time correction, head motion correction, registration, spatial normalization, and template segmentation. The BOLD signal sequence of each brain region is obtained from the image data. The brain regions are defined by the AAL116 brain atlas, and the BOLD signal sequence of a brain region is the average value of the BOLD signal sequences of all voxels in that brain region.

3. The method for detecting mild cognitive impairment based on a combination of local and global spatiotemporal features according to claim 1, characterized in that, In step 3, the local spatiotemporal feature aggregation module consists of a spatial encoding module, a temporal encoding module, and a spatiotemporal fusion module. The spatial encoding module is used to extract the spatial features of each segment, the temporal encoding module is used to extract the local temporal features of each segment, and the spatiotemporal fusion module is used to fuse the spatial features and local temporal features of the segment.

4. The method for detecting mild cognitive impairment based on a combination of local and global spatiotemporal features according to claim 3, characterized in that, In step 3, the spatial encoding module acquires spatial features based on the Graphormer architecture. During forward propagation of this module, the connectivity between brain regions consists of two parts: one part depends on a predefined connection, and the other part is calculated by the model's self-attention mechanism during training. The formula for calculating the distance between brain regions in the predefined connectivity is as follows: Where L is the number of time points in the segment. It is a two-dimensional spatial feature matrix with N brain regions and L time points. Then, the module uses the K-nearest neighbor method to obtain the adjacency matrix of the graph based on the distance matrix, obtains the shortest path matrix based on the adjacency matrix and performs thresholding, and thus obtains the predefined self-attention mechanism calculation formula in the connectivity module as follows: in Q=W q S,K=W k S,V=W v S, P is the shortest path matrix after thresholding. The temporal encoding module, based on a Transformer encoder, is used to extract local temporal features for each segment. This module uses time points as nodes, and the initial feature for each time point is a sequence of signal values ​​from all brain regions at that time point. The spatiotemporal fusion module is used to fuse the spatial features and local temporal features of each segment. This module first concatenates the special nodes extracted by the spatial encoding module with the temporal features extracted by the temporal encoding module and performs self-attention calculation. Then, it performs residual connection between the special nodes after fusing the temporal features and the original special nodes. The calculation flow of the designed spatiotemporal fusion module is as follows: Where τ represents the Transformer encoder, concat is the concatenation operation, and [:,:] represents the matrix slicing operation.

5. The method for detecting mild cognitive impairment based on a combination of local and global spatiotemporal features according to claim 1, characterized in that, In step 4, the global time encoding module consists of a global time information extraction module and a multi-head attention module. The global time information extraction module is used to extract global time-varying information between adjacent segments, and the multi-head attention module fuses the global time-varying information and the local spatiotemporal features of each segment to obtain the subject's embedding.

6. The method for detecting mild cognitive impairment based on a combination of local and global spatiotemporal features according to claim 5, characterized in that, In step 4, the global temporal features of the segment consist of two parts: the change pattern from the previous state to the current state, and the change pattern from the current state to the next state. The calculation formulas for these two parts of the pattern features are as follows: TE post,i =concat(μ i -μ i-1 thing u (∑ i )-thing u (∑ i-1 )), TE next,i =concat(μ i+1 -μ i thing u (∑ i+1 )-thing u (∑ i )), Among them TE post,i and TE next,i These represent the two components of the global temporal feature, μ i and Σ i Let vec and mean vector represent the mean vector and covariance matrix of the multivariate Gaussian distribution to which the segment follows, respectively. Since the covariance matrix of each segment is a symmetric matrix, its upper triangular matrix is ​​used to simplify the amount of input data. u () indicates an upper triangular operation on the matrix. The final global time feature calculation formula for each segment is as follows: yourself i =FFN(concat(TE post,i ,TE next,i )), The multi-head attention module takes the global temporal features and local spatiotemporal features of each segment as input, and aggregates the features of each segment through the correlation between segments. In order to reduce overfitting caused by excessive data dimensionality, special nodes of each segment are used to represent the local spatiotemporal features of that segment. The calculation process of this module is as follows: Where ST represents the feature matrix composed of special nodes of each segment of a single subject, MHA is a multi-head attention layer, flatten is the matrix flattening operation, and X represents the embedding of each subject.

7. The method for detecting mild cognitive impairment based on a combination of local and global spatiotemporal features according to claim 1, characterized in that, In step 5, the group graph network uses subjects as nodes, the node features are the subject embeddings, and the edge weights between nodes are obtained from the subject's age, gender, and the correlation of the embeddings. The graph convolution is implemented based on Chebyshev convolution.

8. The method for detecting mild cognitive impairment based on a combination of local and global spatiotemporal features according to claim 1, characterized in that, In step 6, the training set and the test set are constructed on the same population graph. The network trained on the training set is then calculated again on the population graph to obtain the detection results on the test set.

Images