Dynamic brain function connection classification method based on space-time attention mechanism

Abstract

The invention discloses a dynamic brain function connection classification method based on a space-time attention mechanism, and belongs to the field of brain disease classification. The method comprises the following steps: acquiring fMRI data; preprocessing the fMRI data to obtain an fMRI time sequence; constructing a dynamic function connection mode for the fMRI time sequence through a sliding window method to obtain dynamic function connection data; constructing a space-time attention network model; inputting the dynamic function connection data into a space-time attention network model for training to obtain a trained space-time attention network model; and performing classification prediction on the dynamic function connection data to be tested by using the trained model. According to the method, the problem of high dimension of function connection is solved, extraction of instantaneous information of function connection is realized, complex interaction between space information and time information in a DFC mode is explored, and ASD classification accuracy is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of brain disease classification, and particularly relates to a dynamic brain functional connection classification method based on a spatio-temporal attention mechanism. Background Art

[0002] Autism spectrum disorder (ASD) is considered to be one of the most complex neurodevelopmental disorders. Patients generally suffer from symptoms such as repetitive behaviors, social communication deficits, and restricted interests. Its etiology and neurobiological mechanisms remain an unsolved problem in world medicine to this day. It is estimated that one in every 68 children worldwide suffers from some form of ASD. Faced with such a severe incidence situation, ASD still relies on scale assessments that are somewhat subjective and time-consuming in the diagnosis process for clinical diagnosis. In order to diagnose ASD more effectively, the development of non-invasive neuroimaging techniques has provided important support for studying the abnormal brain structure and functional changes in ASD. Functional magnetic resonance imaging (fMRI) is the most commonly used technique in neuroimaging research, which allows non-invasively examining spontaneous brain activities by detecting changes in blood oxygenation level-dependent signals. In recent years, computer-aided diagnosis methods based on fMRI have developed rapidly.

[0003] Traditional fMRI analysis methods usually focus on the activity levels of individual brain regions. However, more and more evidence has shown recently that different cortical regions in the brain are essentially interconnected during cognitive processing. Therefore, functional connectivity analysis has been introduced to better understand the interaction relationships between brain regions. Compared with the analysis of individual brain regions, functional connectivity analysis can reveal the overall functional state of the brain network by capturing the co-activities between brain regions. However, traditional functional connectivity is static. Given the dynamic nature of brain activities, the analysis of dynamic functional connectivity between different brain regions has attracted more and more attention. The dynamic changes contained in dynamic functional connectivity may provide richer biomarkers than static functional connectivity.

[0004] With the proposal of Transformer, most current studies use the attention mechanism to extract spatio-temporal information in dynamic functional connectivity patterns. However, in the spatial dimension, since functional connectivity requires calculating the connection strength between any two brain regions, this leads to the high-dimensional characteristic of functional connectivity data. Directly using the traditional attention mechanism to extract the spatial information of dynamic functional connectivity patterns will result in low computational efficiency. In the time dimension, some people use the hidden Markov model to estimate the dynamic functional connectivity of the functional network. They found that brain activity not only contains stable connection states, but may also exhibit sudden brain activity fluctuations. These instantaneous and rapidly changing functional connectivity patterns may pose challenges to the process of extracting time information by the model. In addition, due to the characteristics of high-dimensionality and small sample size in brain disease classification tasks, existing models usually face serious overfitting. Simply reducing the dimension of dynamic functional connectivity patterns during the feature extraction process can reduce the risk of overfitting, but it may also lead to the loss of key information. Therefore, how to more effectively extract the spatio-temporal information in dynamic functional connectivity patterns while reducing the risk of model overfitting is the main problem faced by the current task.

[0005] Functional connectivity (FC) reflects the phenomenon of spontaneous synchronous activation of activities between different brain regions and is an effective biomarker for identifying patients with autism spectrum disorder (ASD). However, due to the high-dimensionality and instantaneousness of functional connectivity, existing classification methods are inefficient in extracting spatial information and fail to consider the temporal variation characteristics of functional connectivity when extracting time information, thus hindering the accurate detection of its dynamic changes. Summary of the Invention

[0006] The purpose of the present invention is to provide a dynamic brain functional connectivity classification method based on spatio-temporal attention mechanism, which improves the attention mechanism to solve the limitation that the traditional attention mechanism cannot effectively extract spatio-temporal information in dynamic functional connectivity patterns, and improves the accuracy of deep learning models in brain disease classification tasks.

[0007] To solve the above technical problems, the present invention provides a dynamic brain functional connectivity classification method based on spatio-temporal attention mechanism, including the following steps:

[0008] Obtain fMRI data;

[0009] Preprocess the fMRI data to obtain fMRI time series;

[0010] Construct dynamic functional connectivity patterns for the fMRI time series by means of a sliding window method to obtain dynamic functional connectivity data;

[0011] Construct a spatio-temporal attention network model;

[0012] Input the dynamic functional connectivity data into the spatio-temporal attention network model for training to obtain the trained spatio-temporal attention network model;

[0013] Use the trained model to classify and predict the dynamic functional connectivity data to be measured.

[0014] Preferably, preprocess the fMRI data to obtain the fMRI time series, which specifically includes the following steps:

[0015] Divide the brain of the fMRI data into several brain regions through the CC200 template and the AAL template to obtain the data after brain region division;

[0016] Preprocess the data after brain region division through CPAC and DPARSF to obtain the fMRI time series.

[0017] Preferably, construct the dynamic functional connectivity pattern for the fMRI time series by the sliding window method to obtain the dynamic functional connectivity data, which specifically includes the following steps:

[0018] Calculate the Pearson correlation coefficient of the fMRI time series of pairwise brain regions to obtain the functional connectivity matrix; Let x i (t) and x j (t) are respectively used to represent the values of the brain region time series at time t between the i-th ROI and the j-th ROI. For the brain region time series with a length of T, the Pearson correlation coefficient between two brain regions is:

[0019]

[0020] In the formula: and are respectively used to represent the average values of x i (t) and x j (t);

[0021] Retain the upper triangular features of the functional connectivity matrix, flatten them into a one-dimensional vector, and perform Fisher z-transform on it to obtain the brain functional connectivity vector;

[0022] For the fMRI time series with a total length of T containing R brain regions, let the size of the sliding window be W and the sliding step be s. By calculating the time series correlation of the brain regions within each sliding window, N brain functional connectivity features with dynamic characteristics are obtained, where

[0023] Arrange the brain functional connectivity features of each time window in chronological order to obtain the dynamic functional connectivity pattern containing time features and spatial features.

[0024] Preferably, the spatio-temporal attention network includes a focused linear spatial attention module, a local causal temporal attention module, a channel attention module, and a multi-scale feature training process.

[0025] Preferably, the focused linear spatial attention module is as follows:

[0026] Based on the focused linear attention mechanism, calculate the attention in the spatial dimension in a linear manner, and add a depth convolution module to the calculation of the attention matrix.

[0027] Preferably, based on the focused linear attention mechanism, calculate the attention in the spatial dimension in a linear manner, and add a depth convolution module to the calculation of the attention matrix, which specifically includes the following steps:

[0028] Use a kernel function to decouple the Softmax function;

[0029]

[0030] In the formula: x ** p represents the p-th power of x;

[0031] Use the RuLU function to ensure the non-negativity of the input and the validity of the denominator in the equation; then use f p (x) to make similar query-key pairs closer and different query-key pairs farther away, so as to ensure the concentrated distribution of attention.

[0032] Preferably, the local causal temporal attention module is as follows:

[0033] Divide the time dimension of the dynamic functional connection data into multiple windows by the sliding window method; each window contains the local features of a time period;

[0034] Calculate the temporal attention in parallel on the local features of each window;

[0035] For each segment of the time series, the last functional connection vector is designated as the feature vector of the current time point; model the dependence relationship between the feature vector of the previous time point and the feature vector of the current time point in an attention-based manner;

[0036] Calculate the correlation score between the feature vector of the current time point and the feature vectors of the previous time points within the window;

[0037] Aggregate the correlation scores and their respective feature vectors through weighted summation to obtain an aggregated feature vector;

[0038] Concatenate the aggregated feature vectors obtained from each window to obtain the output feature of the local temporal causal attention module.

[0039] Preferably, the channel attention module is as follows:

[0040] Use global average pooling to compress the temporal information into the spatial dimension, thereby obtaining the statistical information of the spatial dimension;

[0041] Adopt a two-layer fully connected network with activation functions to identify the complex interdependencies between functional connections; perform dimensionality reduction operation in the first fully connected layer with a reduction rate of r, and in the second fully connected layer, restore the features to the original dimension;

[0042] Generate a weight vector by the Sigmoid function;

[0043] Fuse the spatio-temporal information through the weight vector and the output features of the local temporal causal attention module to obtain the fused features.

[0044] Preferably, the multi-scale feature training process is as follows:

[0045] Input the features of different dimensions between multiple spatial attentions and the classification features of the model into the classification network for classification and calculate the classification loss; introduce a balancing mechanism, and different proportions are dynamically assigned to the loss functions of different scale features through learnable parameters, thereby calculating the final classification loss.

[0046] Preferably, input the dynamic functional connection data into the spatio-temporal attention network model for training to obtain the trained spatio-temporal attention network model, which specifically includes the following steps:

[0047] Divide all the dynamic functional connection data into a training set and a test set according to a ratio of 4:1;

[0048] Input the training set data into the spatio-temporal attention network model for multiple rounds of training, and construct a multi-scale classification loss through the classification results of different scale features and the true labels. Optimize the network parameters through the classification loss and the Adam optimizer to obtain the trained weight information;

[0049] Input the test set data into the spatio-temporal attention network for testing with the trained weights.

[0050] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0051] The present invention uses a multi-layer focused linear spatial attention (FLSA) module to efficiently extract spatial information from the DFC pattern and focus the attention on the most important functional connections, thereby solving the high-dimensional problem of functional connections.

[0052] The present invention captures the instantaneous changes in functional connectivity in the DFC pattern from a causal perspective through a local causal temporal attention (LCTA) module based on a sliding window, and realizes the extraction of instantaneous information of functional connectivity.

[0053] The present invention passes through a channel attention (CA) module, which adaptively learns the weights of spatial features and multiplies them with the output features of the LTA module through element-wise multiplication to achieve effective fusion of spatio-temporal information and explore the complex interaction between spatial and temporal information in the DFC pattern.

[0054] Experimental results on large-scale fMRI datasets show that MSTA-LTA-Net outperforms existing models in different preprocessing procedures, brain templates, and cross-validation schemes, significantly improving the ASD classification accuracy and verifying the effectiveness and robustness of this method. Description of the Drawings

[0055] The following further elaborates the specific implementation manners of the present invention in conjunction with the drawings.

[0056] Figure 1 is the construction process of the dynamic brain functional connectivity pattern;

[0057] Figure 2 is the overall framework of the dynamic brain functional connectivity classification model based on the spatio-temporal attention mechanism;

[0058] Figure 3 is the implementation details of each attention module in the dynamic brain functional connectivity classification model based on spatio-temporal attention. Specific Embodiments

[0059] Many specific details are set forth in the following description in order to provide a thorough understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the connotation of the present invention. Therefore, the present invention is not limited by the specific implementations disclosed below.

[0060] The terms used in one or more embodiments of this specification are merely for the purpose of describing specific embodiments and are not intended to limit one or more embodiments of this specification. The singular forms "a", "the", and "said" used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly dictates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more of the associated listed items.

[0061] It should be understood that although the terms first, second, etc. may be used in one or more embodiments of this specification to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining".

[0062] The following further describes the present invention in detail with reference to the accompanying drawings:

[0063] The present invention provides a dynamic brain functional connectivity classification method based on a spatio-temporal attention mechanism, including the following steps:

[0064] Obtain fMRI data;

[0065] Preprocess the fMRI data to obtain an fMRI time series;

[0066] Construct a dynamic functional connectivity pattern for the fMRI time series by means of a sliding window method to obtain dynamic functional connectivity data;

[0067] Construct a spatio-temporal attention network model;

[0068] Input the dynamic functional connectivity data into the spatio-temporal attention network model for training to obtain a trained spatio-temporal attention network model;

[0069] Use the trained model to classify and predict the dynamic functional connectivity data to be measured.

[0070] Preferably, preprocessing the fMRI data to obtain an fMRI time series specifically includes the following steps:

[0071] Divide the brain of the fMRI data into several brain regions through the CC200 template and the AAL template to obtain the data after brain region division;

[0072] Preprocess the data after brain region division through CPAC and DPARSF to obtain an fMRI time series.

[0073] Preferably, constructing a dynamic functional connectivity pattern for the fMRI time series by means of a sliding window method to obtain dynamic functional connectivity data specifically includes the following steps:

[0074] Calculate the Pearson correlation coefficient of the fMRI time series of pairwise brain regions to obtain a functional connectivity matrix; Let x i (t) and x jLet \(x_{ij}(t)\) represent the value of the time series of the brain regions at time \(t\) between the \(i\)-th ROI and the \(j\)-th ROI. For a brain region time series of length \(T\), the Pearson correlation coefficient between two brain regions is:

[0075]

[0076] where: and are used to represent the average values of \(x_{i}(t)\) and \(x_{j}(t)\) respectively; i \(x_{i}(t)\) and \(x_{j}(t)\) j (t);

[0077] Retain the upper triangular features of the functional connectivity matrix and flatten them into a one-dimensional vector, and perform Fisher z-transform on it to obtain the brain functional connectivity vector;

[0078] For an fMRI time series of total length \(T\) containing \(R\) brain regions, let the size of the sliding window be \(W\) and the sliding step be \(s\). By calculating the time series correlation of the brain regions within each sliding window, \(N\) brain functional connectivity features with dynamic characteristics are obtained, where

[0079] Arrange the brain functional connectivity features of each time window in chronological order to obtain a dynamic functional connectivity pattern containing temporal features and spatial features.

[0080] Preferably, the spatio-temporal attention network includes a focused linear spatial attention module, a local causal temporal attention module, a channel attention module, and a multi-scale feature training process.

[0081] Preferably, the focused linear spatial attention module is:

[0082] Based on the focused linear attention mechanism, calculate the attention in the spatial dimension in a linear manner, and add a depth convolution module to the calculation of the attention matrix.

[0083] Preferably, based on the focused linear attention mechanism, calculate the attention in the spatial dimension in a linear manner, and add a depth convolution module to the calculation of the attention matrix, specifically including the following steps:

[0084] Use a kernel function to decouple the Softmax function;

[0085]

[0086] where: \(x^{p}\) **p represents the \(p\)-th power of \(x\);

[0087] Use the RuLU function to ensure the non-negativity of the input and the validity of the denominator in the equation; then use \(f\) p(x) to make similar query-key pairs closer and different query-key pairs farther apart, thus ensuring a concentrated distribution of attention.

[0088] Preferably, the local causal temporal attention module is:

[0089] Divide the temporal dimension of the dynamic functional connectivity data into multiple windows by a sliding window method; each window contains local features of a time period;

[0090] Compute temporal attention in parallel on the local features of each window;

[0091] For each segment of the time series, the last functional connectivity vector is designated as the feature vector of the current time point; model the dependence relationship between the feature vectors of the previous time point and the current time point in an attention-based manner;

[0092] Compute the correlation score between the feature vector of the current time point and the feature vectors of the previous time points within the window;

[0093] Aggregate the correlation scores and their respective feature vectors through weighted summation to obtain an aggregated feature vector;

[0094] Concatenate the aggregated feature vectors obtained from each window to get the output feature of the local temporal causal attention module.

[0095] Preferably, the channel attention module is:

[0096] Use global average pooling to compress the temporal information into the spatial dimension, thereby obtaining statistical information in the spatial dimension;

[0097] Adopt a two-layer fully connected network with activation functions to identify complex interdependencies between functional connectivities; perform a dimensionality reduction operation in the first fully connected layer with a reduction rate of r, and in the second fully connected layer, restore the features to the original dimension;

[0098] Generate a weight vector by the Sigmoid function;

[0099] Fuse the spatio-temporal information through the weight vector and the output feature of the local temporal causal attention module to obtain a fused feature.

[0100] Preferably, the multi-scale feature training process is:

[0101] Input the features of different dimensions between multiple spatial attentions and the classification features of the model into the classification network for classification and calculate the classification loss; introduce a balancing mechanism, and different ratios are dynamically assigned to the loss functions of different scale features through learnable parameters, thereby calculating the final classification loss.

[0102] Preferably, the dynamic functional connectivity data is input into the spatio-temporal attention network model for training to obtain the trained spatio-temporal attention network model, which specifically includes the following steps:

[0103] All the dynamic functional connectivity data is divided into a training set and a test set according to a ratio of 4:1;

[0104] The training set data is input into the spatio-temporal attention network model for multiple rounds of training, and a multi-scale classification loss is constructed through the classification results of different scale features and the true labels. The network parameters are optimized through the classification loss and the Adam optimizer to obtain the trained weight information;

[0105] The test set data is input into the spatio-temporal attention network for testing with the trained weights.

[0106] In the present invention, a dynamic functional connectivity (DFC) pattern is constructed, and a new ASD classification network (MSTA-LTA-Net) combining multi-scale spatio-temporal attention and local time perception is proposed. Specifically, to solve the high-dimensional problem of functional connectivity, a multi-layer focused linear spatial attention (FLSA) module is designed to efficiently extract spatial information from the DFC pattern and focus the attention on the most important functional connectivities. Considering the instantaneousness of functional connectivity, a local causal time attention (LCTA) module based on a sliding window is designed to capture the instantaneous changes of functional connectivities in the DFC pattern from a causal perspective. In addition, to explore the complex interaction between spatial and temporal information in the DFC pattern, a channel attention (CA) module is designed, which adaptively learns the weights of spatial features and multiplies them with the output features of the LTA module through element-wise multiplication to achieve the effective fusion of spatio-temporal information. The experimental results on a large-scale fMRI dataset show that MSTA-LTA-Net is superior to existing models in different preprocessing processes, brain templates, and cross-validation schemes, significantly improving the ASD classification accuracy and confirming the effectiveness and robustness of the method.

[0107] To better illustrate the technical effects of the present invention, the following specific embodiments are provided to illustrate the above technical processes:

[0108] Embodiment 1. A dynamic brain functional connectivity classification method based on a spatio-temporal attention mechanism, including the following steps:

[0109] S1 Download the publicly available large-scale rs-fMRI dataset Autism Brain Imaging Data Exchange (ABIDE) and preprocess it through a standard preprocessing process to obtain the fMRI time series;

[0110] Download the sample data of 823 subjects from different institutions in the ABIDE dataset, and divide the brains of the subject samples into 200 and 116 brain regions using the CC200 template and the AAL template respectively. Subsequently, use two preprocessing pipelines, ConfigurablePipeline for the Analysis of Connectomes (CPAC) and Data Processing Assistant for Resting-State fMRI (DPARSF), to preprocess the data after brain region division to obtain the corresponding fMRI time series.

[0111] S2 uses the sliding window correlation method to construct dynamic functional connectivity data;

[0112] The present invention uses the sliding window correlation technique to quantify the dynamic changes in functional connectivity between brain regions. During the construction of brain functional connectivity, the Pearson correlation coefficient between the time series of brain regions is often used to represent the functional connectivity strength between two brain regions. Assume x i (t) and x j (t) are respectively used to represent the values of the brain region time series at time t between the i-th ROI and the j-th ROI. Then, for the brain region time series with a length of T, the Pearson correlation coefficient between two brain regions can be defined as:

[0113]

[0114] where and are respectively used to represent the average values of x i (t) and x j (t).

[0115] The functional connectivity matrix calculated from the Pearson correlation coefficient is a symmetric matrix. Only the upper triangular features of this symmetric matrix are retained and flattened into a one-dimensional vector, and a Fisher z-transform is performed on it to obtain the final functional connectivity vector. In this vector, each element at a position represents the correlation strength between paired brain regions. During the analysis of dynamic brain functional connectivity features, a common sliding window-based method will be used to construct dynamic functional connectivity patterns. For a time series with a total length of T containing R brain regions, assume the size of the sliding window is W and the sliding step is s. By calculating the correlation for the time series within each sliding window, N dynamic brain functional connectivity features can be obtained, where Arranging the brain functional connectivity vector features of each time window in chronological order can obtain a dynamic functional connectivity pattern containing time features and spatial features.

[0116] S3 constructs a spatio-temporal attention network model for extracting spatio-temporal information in dynamic functional connectivity patterns;

[0117] The spatio-temporal attention network mainly includes a focused linear spatial attention module, a local causal temporal attention module, a channel attention module, and a multi-scale feature training process.

[0118] Traditional attention mechanisms capture global dependencies in input data by computing queries, keys, and values. In the calculation of spatial attention, first, the input dynamic functional connectivity pattern θ ∈ R T×N is linearly transformed to obtain query vectors, key vectors, and value vectors:

[0119] q t = θW Q ∈ R T×N

[0120] k t = θW k ∈ R T×N

[0121] v t = θW V ∈ R T×N

[0122] where W Q ∈ R N×N 、W K ∈ R N×N and W V ∈ R N×N are learnable parameter matrices. Then, the traditional self-attention mechanism is used to calculate spatial dependencies. However, since dynamic functional connectivity data usually has high-dimensional features, traditional attention calculations involve high-complexity matrix multiplications, with a computational complexity of O(N 2 ), where N is the number of functional connections. For high-dimensional functional connectivity patterns, this calculation method consumes a large amount of computing resources, resulting in long training times and low efficiency. To improve computational efficiency and reduce the computational burden of high-dimensional features, focused linear attention is introduced. Focused linear attention calculates attention in a linear manner, which can reduce the computational complexity from O(N 2 ) of the traditional attention mechanism to O(T 2 ), thus effectively improving computational efficiency. In addition, to overcome the shortcoming that the linear attention matrix is too smooth, focused linear attention designs a unique kernel function to ensure the concentrated distribution of attention, and its overall calculation process can be expressed as:

[0123]

[0124] where x**p Denote the p-th power of x. Use the RuLU function to ensure the non-negativity of the input and the validity of the denominator in the equation. Then use f p (x) to make similar query-key pairs closer and different query-key pairs farther away, thus ensuring the concentrated distribution of attention. It should be noted that the norm of the feature does not change after mapping, that is, ||x|| = ||f p (x)||, which means that only the direction of the feature changes. In this way, the focused linear attention can not only significantly reduce the computational complexity, but also ensure that the attention is concentrated on important spatial features, so that the model can more effectively identify the key functional connections related to brain diseases. Finally, to address the low-rank shortcoming of the linear attention matrix, a depth convolution module is added to the calculation of the attention matrix. By adding the local features of each key, the information of the linear attention matrix is enriched. The final attention calculation method can be expressed as follows:

[0125] O = φ p (Q)φ p (K) T V + DWC(V)

[0126] The depth convolution module enriches the information of the attention matrix by adding the local features of each key, and improves the modeling ability of the linear attention mechanism for spatial dependence relationships.

[0127]

[0128] The local causal attention module divides the time dimension of the dynamic functional connection pattern into multiple equal-sized windows by the sliding window method. Each window contains the local features of a time period, which enables the model to focus on capturing the changes in functional connections within that time period. Specifically, the feature after extracting spatial information through multiple spatial attention modules is The model samples into k time series segments by the sliding window method, denoted as Subsequently, the temporal attention is calculated in parallel on the local features of each window. Recognizing that the functional connections at the current time point usually have a more obvious causal relationship with the functional connections at the previous time point, the attention mechanism is improved. Specifically, for each segment of the time series, the last functional connection vector is designated as the feature vector at the current time point. The dependence relationship between the feature vector at the previous time point and the feature vector at the current time point will be modeled in an attention-based manner.

[0129] This process can be represented by the following mathematical formula:

[0130]

[0131] wherein represents the feature vector at the current time point, represents the feature vectors at previous time points within the window.

[0132] In this process, first, the correlation score between the feature vector at the current time point and the feature vectors at previous time points within the window is calculated to capture local temporal dependencies. Subsequently, the correlation scores and their respective feature vectors are aggregated through weighted summation, enabling the feature vector at the current time point to reflect the dynamic changes in functional connectivity within the current window. Finally, the aggregated feature vectors obtained for each window are concatenated to obtain the final output features. This process ensures that the module can capture the instantaneous changes in functional connectivity within each window, effectively enhancing the model's ability to learn complex temporal dynamics in dynamic functional connectivity patterns.

[0133] To more accurately represent the spatial information in the features, the channel attention module uses global average pooling to compress the temporal information into the spatial dimension, thereby obtaining statistical information in the spatial dimension. The process is as follows:

[0134]

[0135] where θ represents the input features of the module, and T represents the size of the temporal dimension in the input features.

[0136] In addition, to fully utilize the potential of the aggregated features and robustly evaluate the importance of each functional connection, a two-layer fully connected network with activation functions is adopted to identify the complex interdependencies between functional connections, as shown in the following formula:

[0137] Weight = F ex (θ', W) = σ(g(θ', W)) = σ(W 2 δ(W 1 θ′))

[0138] where δ represents the Relu function, σ represents the Sigmoid function, represents the parameter matrix of the two-layer fully connected network.

[0139] Compared with the traditional method that only uses the aggregated features as weights, this method adaptively learns more reliable and effective weight information by capturing the dependencies between functional connections. Finally, by element-wise multiplying this weight vector with the output features of the local causal temporal attention module, the model achieves effective fusion of spatio-temporal information. Compared with the traditional concatenation method, this method can more comprehensively reflect the complex interaction between temporal dynamics and spatial features, significantly enhancing the model's ability to capture the deep-level associations between spatio-temporal information, thereby generating more effective fused features.

[0140] During the multi-scale feature training process, features of different dimensions between multiple layers of spatial attention are retained, and they are input into the classification network together with the classification features of the model for classification and calculation of the classification loss. In practical applications, the present invention introduces an appropriate balance mechanism in the multi-scale feature training method to ensure effective cooperation between high-dimensional and low-dimensional features. This mechanism can be adjusted according to task requirements and the dataset to further optimize the model performance. Specifically, different ratios will be dynamically assigned to the loss functions of different scale features through learnable parameters, thereby calculating the final classification loss.

[0141] This process is described as follows:

[0142]

[0143] where y ij represents the true label, represents the predicted probability of the i-th scale feature, and λ 1 and λ 2 respectively represent the ratios of the loss functions assigned to the corresponding scale features.

[0144] Through this improvement, the present invention achieves a finer balance among model complexity, generalization performance, and information utilization ability. Therefore, when dealing with high-dimensional dynamic functional connection patterns, the proposed model achieves a significant performance improvement.

[0145] S4 divides the dynamic functional connection data into a training set and a test set. In the spatio-temporal attention network model constructed in S3, the dynamic functional connection training set data constructed in S2 is trained to obtain the trained weights, and the data of the test set is classified using the trained weights.

[0146] The training process of the spatio-temporal attention network model includes the following steps:

[0147] S4-1, divides the dynamic functional connection dataset into a training set and a test set according to a ratio of 4:1.

[0148] S4-2, inputs the training set data into the spatio-temporal attention network for multiple rounds of training. The specific number of rounds is set to 200 rounds, and a multi-scale classification loss is constructed through the classification results of different scale features and the true label. The network parameters are optimized using the classification loss and the Adam optimizer to obtain the trained weight information.

[0149] S4-3, after obtaining the trained weights, inputs the test set data into the spatio-temporal attention network for testing with the trained weights to verify the classification performance of the network.

[0150] S5 uses the trained model to perform classification prediction on the dynamic functional connectivity data to be measured;

[0151] The dynamic functional connectivity data to be measured is obtained by: acquiring the fMRI data to be measured, and then through the same preprocessing process as in step S1 and the same construction of the dynamic functional connectivity pattern as in step S2, the dynamic functional connectivity data to be measured is obtained.

[0152] For the classification performance of the proposed spatio-temporal attention network model, on the ASD datasets with different preprocessing processes (CPAC and DPARSF) and different brain templates (CC200 and AAL), the ASD classification network (MSTA-LTA-Net) of the present invention is compared with several brain disease classification methods using traditional attention mechanisms. The comparison methods include STA-BiGRU, STANet, and BoLT. STANet uses the traditional attention mechanism to serially extract the spatio-temporal information of the DFC pattern. STA-BiGRU uses the traditional spatial and temporal attention modules to parallelly extract the spatio-temporal information, and additionally adds a bidirectional BiGRU unit for further temporal feature analysis in the extraction of temporal information. BoLT, aiming at the disadvantage of large computational amount of the traditional attention mechanism, uses a sliding window method to extract the local temporal sequence information of the functional connectivity, but this method ignores the effective extraction of spatial information. To ensure fairness, different methods, preprocessing processes, and brain templates are all applied to the same experimental samples.

[0153] Table 1.

[0154]

[0155] As can be seen from the table, the MSTA-LTA Net proposed in the present invention has achieved the highest classification accuracy on all experimental datasets. For example, on the CPAC-preprocessed CC200 dataset, its classification accuracy reached 73.06%, which is 4.01%, 3.16%, and 2.81% higher than STA-BiGRU, STANet, and BoLT respectively. On the CPAC-preprocessed AAL dataset, the classification accuracy of MSTA-LTA Net was 71.73%, which is 4.56%, 5.47%, and 3.41% higher than STA-BiGRU, STANet, and BoLT respectively. It is worth noting that MSTA-LTA Net not only leads far ahead in terms of classification accuracy but also performs best in other evaluation metrics such as sensitivity, precision, F1-score, and AUC. These results indicate that MSTA-LTA Net has significant advantages in multi-dimensional performance evaluation when compared with other models. In addition, MSTA-LTA Net also performed excellently in terms of classification accuracy and precision in the CC200 and AAL datasets preprocessed by DPARSF, further demonstrating the consistency and robustness of the model on different datasets. Although the results are slightly inferior to those of the datasets under the CPAC preprocessing process on some datasets, MSTA-LTA Net still maintains the lead in overall performance, showing its wide applicability. Experiments show that MSTA-LTA Net can not only process data of multiple brain templates but also provide reliable classification performance for data under different preprocessing processes, which further verifies the effectiveness and reliability of the model in the field of brain disease classification.

[0156] In several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules, modules, or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units, modules, or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0157] The unit may or may not be physically separated. The component shown as a unit may be a physical unit or multiple physical units, that is, it may be located in one place or distributed to multiple different places. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0158] In addition, in each embodiment of the present invention, each functional unit may be integrated into a processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional unit.

[0159] Specifically, according to the embodiments disclosed in the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, an embodiment of the present disclosure includes a computer program product that includes a computer program carried on a computer-readable medium, and the computer program includes program codes for performing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), the above-mentioned functions defined in the method of the present invention are executed. It should be noted that the above-mentioned computer-readable medium of the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above.

[0160] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0161] As described above, the above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed by the present invention should be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A dynamic brain functional connection classification method based on spatiotemporal attention mechanism, characterized in that: include: Acquiring fMRI data; Preprocess the fMRI data to obtain fMRI time series; The dynamic functional connectivity model of fMRI time series was constructed by sliding window method to obtain dynamic functional connectivity data; Construct a spatiotemporal attention network model; The dynamic functional connection data is input into the spatiotemporal attention network model for training to obtain the trained spatiotemporal attention network model; The trained model is used to make classification predictions on the dynamic functional connectivity data to be tested.

2. The dynamic brain functional connection classification method based on spatiotemporal attention mechanism according to claim 1 is characterized in that: The fMRI data is preprocessed to obtain the fMRI time series, which specifically includes the following steps: The fMRI data of the brain is divided into several brain regions using the CC200 template and the AAL template to obtain the data after brain region division; After brain region division, the data were preprocessed using CPAC and DPARSF to obtain fMRI time series.

3. The dynamic brain functional connection classification method based on spatiotemporal attention mechanism according to claim 2 is characterized in that: The dynamic functional connectivity model of the fMRI time series is constructed by the sliding window method to obtain dynamic functional connectivity data, which specifically includes the following steps: Calculate the Pearson correlation coefficient of the fMRI time series of each brain region to obtain the functional connection matrix; let x i (t) and x j (t) are used to represent the value of the brain region time series between the i-th ROI and the j-th ROI at time t. For the brain region time series of length T, the Pearson correlation coefficient of the two brain regions is: Where: and Used to represent x i (t) and x j The average value of (t); The upper triangular features of the functional connectivity matrix are retained and flattened into a one-dimensional vector and subjected to Fisher z transformation to obtain the brain functional connectivity vector; For an fMRI time series with a total length of T and containing R brain regions, let the size of the sliding window be W and the sliding step be s. By calculating the time series correlation of the brain regions in each sliding window, N brain functional connection features with dynamic characteristics are obtained, where The brain functional connection features of each time window are arranged in chronological order to obtain a dynamic functional connection pattern containing temporal and spatial features.

4. The dynamic brain functional connection classification method based on spatiotemporal attention mechanism according to claim 3 is characterized by: The spatiotemporal attention network includes a focused linear spatial attention module, a local causal temporal attention module, a channel attention module and a multi-scale feature training process.

5. The dynamic brain functional connection classification method based on spatiotemporal attention mechanism according to claim 4 is characterized in that: The focused linear spatial attention module is: Based on the focused linear attention mechanism, the attention of the spatial dimension is calculated in a linear manner, and the deep convolution module is added to the calculation of the attention matrix.

6. The dynamic brain functional connection classification method based on spatiotemporal attention mechanism according to claim 5 is characterized in that: Based on the focused linear attention mechanism, the attention of the spatial dimension is calculated in a linear way, and the deep convolution module is added to the calculation of the attention matrix. The specific steps include: Use kernel function to decouple the Softmax function; Where: represents x raised to the power of p; Use the RuLU function to ensure the non-negativity of the input and the validity of the denominator in the equation; then use f p (x) is used to make similar query-key pairs closer and different query-key pairs farther away, thus ensuring a concentrated distribution of attention.

7. The dynamic brain functional connection classification method based on spatiotemporal attention mechanism according to claim 6 is characterized in that: The local causal temporal attention module is: The time dimension of dynamic functional connectivity data is divided into multiple windows through the sliding window method; each window contains the local features of a time period; Temporal attention is computed in parallel on the local features of each window; For each segment of the time series, the last functional connectivity vector is assigned as the feature vector of the current time point; the dependency between the feature vectors of the previous time points and the feature vector of the current time point is modeled in an attention-based manner; Calculate the correlation score between the feature vector at the current time point and the feature vector at the previous time point in the window; Aggregate the correlation scores and their respective feature vectors by weighted summation to obtain an aggregated feature vector; The aggregated feature vectors obtained for each window are concatenated to obtain the output features of the local temporal causal attention module.

8. The dynamic brain functional connection classification method based on spatiotemporal attention mechanism according to claim 7 is characterized in that: The channel attention module is: Use global average pooling to compress temporal information into the spatial dimension, thereby obtaining statistical information in the spatial dimension; A two-layer fully connected network with activation functions is used to identify the complex interdependencies between functional connections; a dimensionality reduction operation is performed in the first fully connected layer with a dimensionality reduction rate of r, and in the second fully connected layer, the features are restored to the original dimension; The weight vector is generated by the Sigmoid function; The spatiotemporal information is fused by the weight vector and the output features of the local temporal causal attention module to obtain the fused features.

9. The dynamic brain functional connection classification method based on spatiotemporal attention mechanism according to claim 8 is characterized in that: The multi-scale feature training process is: The features of different dimensions between multiple layers of spatial attention are input into the classification network together with the classification features of the model for classification and calculation of the classification loss. A balancing mechanism is introduced, and different proportions are dynamically assigned to the loss functions of features of different scales through learnable parameters to calculate the final classification loss.

10. The dynamic brain functional connection classification method based on spatiotemporal attention mechanism according to claim 9 is characterized in that: The dynamic functional connection data is input into the spatiotemporal attention network model for training to obtain the trained spatiotemporal attention network model, which specifically includes the following steps: All dynamic functional connectivity data are divided into training set and test set in a ratio of 4:1; The training set data is input into the spatiotemporal attention network model for multiple rounds of training, and the multi-scale classification loss is constructed through the classification results of different scale features and the true labels. The network parameters are optimized through the classification loss and the Adam optimizer to obtain the trained weight information; The test set data is input into the spatiotemporal attention network for testing with the trained weights.

Images