Dynamic brain network attention fusion working memory capacity evaluation method and device

Abstract

The application provides a dynamic brain network attention fusion working memory capacity evaluation method and device, and belongs to the technical field of intelligent medical treatment. The method comprises the following steps: segmenting a BOLD signal time sequence in functional magnetic resonance imaging data, extracting time sequence data of a region of interest, and constructing a functional connection matrix; extracting and fusing spatial dependence and time fluctuation characteristics in the functional connection matrix based on an attention mechanism to obtain a full connection graph; using multi-head attention to cluster the full connection graph to obtain functional magnetic resonance imaging embedding representation of each subject; training a prediction network by using the functional magnetic resonance imaging embedding representation of the subject; and completing working memory evaluation of a target subject by using the trained prediction network. The application combines dynamic multi-graph fusion and neuroscientific prior knowledge, eliminates individual differences and specific noise by using contrast learning, realizes accurate evaluation of working memory capacity, and provides an objective basis for evaluation of cognitive function.

Description

Technical Field

[0001] This invention relates to the field of intelligent medical technology, and in particular to a method and device for assessing working memory ability through dynamic brain network attention fusion. Background Technology

[0002] Working memory is a limited-capacity memory system that temporarily processes and stores information. Compared to short-term memory, working memory emphasizes the active processing and manipulation of information. Working memory plays a crucial role in cognitive function, significantly influencing reasoning, decision-making tendencies, and behavioral tendencies. It is an important concept in cognitive psychology, neuropsychology, and neuroscience. An individual's working memory capacity strongly predicts their performance on various higher cognitive indicators, such as fluid intelligence, abstract reasoning, mathematical and linguistic abilities, and overall academic achievement.

[0003] Traditional working memory tasks are primarily measured using scale-based experiments, such as delayed response paradigms, dual-task paradigms, and n-back tasks. These tests are typically conducted in a laboratory setting and are relatively simple in design. The subject's emotional state, fatigue level, attentional state, and testing motivation during the test can all influence the results. Traditional testing methods infer working memory capacity indirectly from behavioral data, lacking direct measurement of its neurophysiological mechanisms and interpretability. Furthermore, subjects may develop a learning effect after repeated testing, affecting the accuracy of the measurement. As mentioned earlier, working memory can be used to assess the severity of certain mental illnesses. In some extreme cases, patients with severe tissue damage in the prefrontal cortex may have difficulty completing working memory tasks due to attention-related problems.

[0004] Functional magnetic resonance imaging (fMRI) divides the brain space into multiple regions of interest (ROIs) through brain mapping and infers functional connectivity patterns between these regions based on dynamic changes in blood oxygen level-dependent (BOLD) signals. Functional connectivity is used to characterize the correlations between different brain regions and identify clusters of activated regions during cognitive tasks. However, existing fMRI-based models still have many limitations. Functional connectivity networks fluctuate over time, and the brain continues to generate activity spontaneously even at rest. However, this invention observes that existing methods often overlook these dynamic changes and the underlying neural mechanisms. Furthermore, the integration of prior medical knowledge with models remains insufficiently explored. In addition, individual differences and specific noise can also affect the results. Summary of the Invention

[0005] To overcome the shortcomings of the prior art, the purpose of this invention is to provide a method and device for assessing working memory ability through dynamic brain network attention fusion.

[0006] To achieve the above objectives, the present invention provides the following solution:

[0007] Methods for assessing working memory capacity through dynamic brain network attention fusion include:

[0008] S1, acquire functional magnetic resonance imaging data of several subjects, segment the time series of BOLD signal in the functional magnetic resonance imaging data, and extract the time series data of the region of interest;

[0009] S2, preprocess the time series data of the region of interest and divide the preprocessed time series data of the region of interest into multiple time window sequences, calculate the functional connectivity strength between time series within each time window, and construct a functional connectivity matrix using the Pearson correlation coefficient;

[0010] S3, Based on the attention mechanism, the spatial dependence and temporal fluctuation characteristics in the functional connection matrix are extracted and fused to obtain a fully connected graph;

[0011] S4. Based on prior knowledge of neuroscience, importance coefficients are assigned to nodes in the fully connected graph. Taking the target brain region in the fully connected graph as the core node, neighbors are selected to form clusters based on edge weights and neighbor importance. Multi-head attention is used to calculate the attention from unclustered nodes to each cluster, and the functional magnetic resonance imaging embedding representation of each subject is obtained.

[0012] S5, the functional magnetic resonance imaging (fMRI) embedding representation of each subject is input into the prediction network for training to obtain a trained prediction network;

[0013] S6, the trained prediction network is used to assess the working memory of the target subject. Preferably, in S2, the functional connectivity matrix is ​​represented using a dynamic brain functional connectivity map. Where M represents the number of graphs; each dynamic brain connectivity graph Contains the same set of nodes V represents the brain region of interest, but with different boundary sets. and node features Where N is the number of nodes and d is the feature dimension. Preferably, step S3, which extracts and fuses the spatial dependence and temporal fluctuation characteristics in the functional connectivity matrix based on the attention mechanism to obtain a fully connected graph, includes: S31, calculating the attention weights between nodes in the dynamic brain functional connectivity graph;

[0014] S32, extract the corresponding attention weights from each node to form an attention matrix, and obtain the cross-graph attention score of each node through the attention network;

[0015] S33, normalize the cross-graph attention score of each node to obtain the cross-graph attention weight, and fuse the cross-graph attention weight with the attention matrix to obtain the fused attention weight matrix;

[0016] S34, stack the node features of all the dynamic brain function connectivity graphs into a tensor and then calculate the feature fusion weights through a feature attention network;

[0017] S35. After calculating and normalizing the feature attention score, the feature fusion weights are fused with the normalized feature attention score to generate the fused node feature matrix.

[0018] S36. A fully connected graph is obtained based on the fused node feature matrix and the fused attention weight matrix.

[0019] Preferably, in S31, the expression for calculating the attention weights between nodes in the dynamic brain functional connectivity graph is: in, These represent the query matrix and the key matrix, respectively. and For learnable parameter matrix, d is the projection dimension, and d is the feature dimension. This represents the attention weights between nodes in the dynamic brain functional connectivity graph. Preferably, in S33, the expression for the fused attention weight matrix is: in, Indicates cross-graph attention weights. This represents the fused attention weight matrix. Indicates the first Nodes in Zhang's Dynamic Brain Function Connectivity Diagram The corresponding attention weight vector. Preferably, in S35, the fused node feature matrix is: ; ;in, This represents the fused node feature matrix. This represents the sum of the feature attention scores. In the feature attention, the first Slicing the key matrix of dimension In the feature attention, the first Dimensional query matrix slicing The hidden dimension representing feature attention. Indicates the first The original node feature matrix of a dynamic brain functional connectivity map This represents the normalized feature attention weight vector. Preferably, in step S4, importance coefficients are assigned to nodes in the fully connected graph based on prior neuroscience knowledge. Using the target brain region in the fully connected graph as the core node, neighbors are selected to form clusters based on edge weights and neighbor importance. Multi-head attention is used to calculate the attention from unclustered nodes to each cluster, resulting in a functional magnetic resonance imaging embedding representation for each subject, including:

[0020] S41, assigns initial importance coefficients to nodes in the fully connected graph based on prior knowledge of neuroscience, and uses a multilayer perceptron to learn and adjust the importance coefficients of each node.

[0021] S42, using key brain regions as core nodes, select neighborhoods based on edge weights and neighbor importance to form K clusters; S43, treat each cluster as a node and apply a multi-head attention mechanism to calculate the attention of unclustered nodes to each cluster;

[0022] S44. Local residuals are used to add the original unclustered node features to the attention aggregation output, and global residuals are used to add the global average pooling representation of all nodes in the first layer to the final output to obtain the functional magnetic resonance imaging embedding representation of each subject.

[0023] Preferably, in S42, a graph attention network is used to generate cluster representations: in, GAT represents the graph attention network aggregation representation of the k-th cluster. This represents the feature of the j-th node in the i-th dynamic brain functional connectivity graph. This indicates that node j belongs to the k-th cluster. Preferably, the formula for calculating the attention of each cluster is: in, Let K represent the query matrix containing all unclustered nodes, where K is the key matrix, V is the value matrix, S is the node importance coefficient, and d is the feature dimension. This represents the normalization function.

[0024] Preferably, step S5, which involves inputting the functional magnetic resonance imaging (fMRI) embedding representation of each subject into a prediction network for training to obtain a trained prediction network, includes:

[0025] S51, through feature engineering, the metadata in the functional magnetic resonance imaging data is converted into feature vectors, and the functional magnetic resonance imaging embedding representation and feature vector of each subject are projected into a shared embedding space to form training samples.

[0026] S52, the prediction network is trained using the training samples to obtain the trained prediction network; wherein, the loss function during the training process is: ; ; ;in, Indicates comparative loss, Let L represent the mean squared error loss, and let L represent the total loss when the two are added together. Let N represent the similarity between the functional magnetic resonance imaging data embedding representation and the metadata embedding representation of the i-th sample, and let N represent the number of samples in the contrastive learning. This represents the temperature hyperparameter, and B represents the training batch size. This represents the true working memory score of the i-th sample. Let represent the prediction working memory score of the i-th sample.

[0027] A working memory assessment device based on dynamic brain network attention fusion includes: a region of interest extraction module, used to acquire functional magnetic resonance imaging (fMRI) data from several subjects, segment the time series of BOLD signals in the fMRI data, and extract the time series data of the region of interest; and a preprocessing module, used to preprocess the time series data of the region of interest, segment the preprocessed time series data of the region of interest into multiple time window sequences, calculate the functional connectivity strength between time series within each time window, and construct a functional connectivity matrix using the Pearson correlation coefficient.

[0028] The fully connected graph construction module is used to extract and fuse the spatial dependence and temporal fluctuation characteristics in the functional connectivity matrix based on the attention mechanism to obtain a fully connected graph; the sample construction module is used to assign importance coefficients to nodes in the fully connected graph based on neuroscience prior knowledge, take the target brain region in the fully connected graph as the core node, select neighbors to form clusters based on edge weights and neighbor importance, use multi-head attention to calculate the attention from unclustered nodes to each cluster, and obtain the functional magnetic resonance imaging embedding representation of each subject;

[0029] The training module is used to input the functional magnetic resonance imaging (fMRI) embedding representation of each subject into the prediction network for training to obtain a trained prediction network.

[0030] The working memory assessment module utilizes a trained prediction network to assess the working memory of target subjects. The beneficial effects of the dynamic brain network attention fusion working memory ability assessment method provided by this invention are as follows: Compared with existing technologies, this invention has the following advantages:

[0031] (1) By dividing the BOLD signal time series into multiple time windows, the pattern of brain activity changes over time can be dynamically depicted, and the model’s ability to capture dynamic features of brain function can be improved. (2) Multi-head attention is used to capture important features in different subspaces, making fuller use of the complex relationship information between nodes, improving the ability to represent brain network structure, and helping to distinguish the differences in brain function among different individuals. (3) This invention combines dynamic multi-graph fusion with prior knowledge of neuroscience, and uses contrastive learning to eliminate individual differences and specific noise to achieve accurate assessment of working memory ability. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 This is a flowchart of the working memory ability assessment method based on dynamic brain network attention fusion provided by the present invention;

[0034] Figure 2 This is a schematic diagram of the working memory ability assessment method based on dynamic brain network attention fusion provided by the present invention. Detailed Implementation

[0035] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0036] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments.

[0037] The terms "first," "second," "third," and "fourth," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, including a series of steps, processes, methods, etc., is not limited to the steps listed, but may optionally include steps not listed, or may optionally include other steps inherent to these processes, methods, products, or devices.

[0038] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0039] Please see Figures 1-2 A dynamic brain network attention fusion-based method for assessing working memory capacity includes:

[0040] S1, acquire functional magnetic resonance imaging data of several subjects, segment the time series of BOLD signal in the functional magnetic resonance imaging data, and extract the time series data of the region of interest;

[0041] In S1, functional magnetic resonance imaging data of several subjects and corresponding patient personal information data are collected, and the BOLD signal time series is segmented based on a predefined brain region template. Brain region blood oxygen level dependent signal dataset is obtained from the HCPS1200 dataset, and time series data of each ROI are extracted.

[0042] S2, preprocess the time series data of the region of interest and divide the preprocessed time series data of the region of interest into multiple time window sequences, calculate the functional connectivity strength between time series within each time window, and construct a functional connectivity matrix using the Pearson correlation coefficient;

[0043] In S2, the present invention preprocesses functional magnetic resonance imaging (fMRI) including removing scanner drift and motion artifacts, calculating the correlation matrix, and normalizing signals at the subject level. The ROI time-series data is segmented into multiple time window sequences using a sliding window method. The functional connectivity strength between ROIs within each time window is calculated. A functional connectivity matrix is ​​constructed using Pearson correlation coefficients as matrix elements, with ROIs as the rows and columns of the matrix. This matrix represents the functional connectivity strength between ROIs within the current time window. Specifically, the functional connectivity matrix is ​​represented by a set of dynamic brain network diagrams. Each image Contains the same set of nodes , representing regions of interest (ROIs) in the brain, but with different boundary sets. and node features Where N is the number of nodes and d is the feature dimension. S3, Based on the attention mechanism, the spatial dependency and temporal fluctuation characteristics in the functional connectivity matrix are extracted and fused to obtain a fully connected graph;

[0044] In S3, multi-graph fusion analysis is constructed based on capturing the spatial dependence and temporal fluctuation characteristics of attention in dynamic brain functional connectivity network sequences: self-attention weights between nodes are calculated for each graph. A cross-graph attention fusion mechanism is designed to extract the attention weight vector corresponding to each node and calculate the attention score matrix between nodes. After normalization, the cross-graph attention weights are obtained, and the fused attention weight matrix is ​​constructed. By fusing the node features of multiple graphs, the constructed fully connected graph network is obtained.

[0045] Specifically, S3 includes:

[0046] S31, for each image This invention first calculates the self-attention weights between nodes: In the formula: These represent the query matrix and the key matrix, respectively. and For learnable parameter matrix, For the projection dimension. Matrix Representation diagram Attention weights between nodes;

[0047] S32, In order to fuse information across multiple dynamic graphs, this invention designs a cross-graph attention fusion mechanism. For each node... This invention extracts the corresponding attention weight vectors from m images to form an attention matrix. Each attention vector is expanded to dimension . Then, nodes are obtained through an attention network. Cross-graph attention score for: In the formula, ;in, and These are the query and key projection matrices for graph attention. This represents the attention weight between the k-th node and all other nodes in the i-th dynamic brain connectivity graph. S33 represents the hidden dimension of graph attention. After normalization, this invention obtains the cross-graph attention weights. Finally, the fused attention weight matrix is ​​constructed: S34, similarly, this invention also integrates node features from multiple graphs. First, the node features of all graphs are stacked into a tensor. Then, feature fusion weights are calculated using a feature attention network: In the formula: It is the query and key projection matrix used for feature attention. This represents the hidden dimension of feature attention.

[0048] S35, Calculate and normalize the feature attention score to obtain These weights are then used to fuse node features, generating a fused node feature matrix. ; ; In the formula: Let FS represent the fused node feature matrix, and let FS represent the sum of feature attention scores. In the feature attention, the first Slicing the key matrix of dimension In the feature attention, the first Dimensional query matrix slicing The hidden dimension representing feature attention. This represents the original node feature matrix of the i-th dynamic brain functional connectivity map. This represents the normalized feature attention weight vector.

[0049] Finally, this invention yields a fully connected graph: Any two nodes are connected by weighted edges;

[0050] S4, based on prior knowledge in neuroscience, assigns importance coefficients to nodes in the fully connected graph. Using the target brain region in the fully connected graph as the core node, neighbors are selected to form clusters based on edge weights and neighbor importance. Multi-head attention is used to calculate the attention from unclustered nodes to each cluster, resulting in a functional magnetic resonance imaging embedding representation for each subject. Further, S4 includes:

[0051] S41, define the set of key ROI nodes as First, this invention assigns an initial importance coefficient to each brain region: key ROI nodes are initialized to 1, while other nodes are assigned a smaller value. Subsequently, this invention uses a multilayer perceptron to learn the importance of each node. This process enables the model to dynamically adjust the importance weights of each node based on the input data, thereby better capturing the influence of different brain regions on working memory. S42, using key brain regions as core nodes, this invention selects neighborhoods based on edge weights and neighbor importance to form K clusters. The cluster assignment weights are defined as follows: Subsequently, a graph attention network (GAT) is used to generate cluster representations: S43, Next, treat each cluster as a node and apply a multi-head attention mechanism to compute the attention of unclustered nodes to each cluster: In the formula: This represents the query matrix containing all unclustered nodes, where K is the key matrix. The value matrix contains representations of all clusters. S44 introduces two residual connections during the graph pooling stage. Local residuals: add the original unclustered node features to the attention aggregation output. Global residuals: add the globally averaged pooled representations of all nodes from the first layer to the final output. In this process, the functional magnetic resonance imaging embedding representation of each subject was obtained.

[0052] S5, the functional magnetic resonance imaging (fMRI) embedding representation of each subject is input into the prediction network for training to obtain a trained prediction network;

[0053] Furthermore, S5 includes: S51, to reduce modality-specific noise such as fMRI scan artifacts, behavioral sensor differences, and inter-individual variability, thereby improving the robustness of the model. After processing by the preceding module, an fMRI data embedding representation for each subject is obtained. In addition, this invention also converts metadata in the dataset into feature vectors through feature engineering. First, this invention projects both the brain network representation and metadata into a shared embedding space: ; In the formula: Encode represents the feature engineering process used to transform heterogeneous metadata into a unified vector representation. S52, during training, the model simultaneously receives both fMRI-based and metadata-based representations within a batch. Each pair of fMRI-metadata is considered a positive sample pair, while all other combinations within the same batch are considered negative sample pairs. The model is trained using the InfoNCE loss function, which encourages positive sample pairs to be closer together in the embedding space, while negative sample pairs are kept further apart. The prediction model is trained using the mean squared error (MSE) loss. The final total loss is a weighted sum of the prediction loss and the contrastive loss: ; ; ;in, Indicates comparative loss, Let L represent the mean squared error loss, and let L represent the total loss when the two are added together. Let N represent the similarity between the functional magnetic resonance imaging data embedding representation and the metadata embedding representation of the i-th sample, and let N represent the number of samples in the contrastive learning. This represents the temperature hyperparameter, and B represents the training batch size. This represents the true working memory score of the i-th sample. Let represent the prediction working memory score of the i-th sample.

[0054] Repeat steps S4-S5 to obtain the trained attention connection network, the trained subgraph generation network, and the trained prediction network. In step S6, use the trained prediction network to perform a working memory assessment of the target subject.

[0055] In this embodiment of the invention, the method of the present invention (AFDB) is compared with existing methods. GCN, which performs neighborhood aggregation to learn node representations through stacked layers, where each layer computes a new representation by aggregating features of the node's neighbors; GAT, which enhances GCN by learning different importance weights for neighbors, allowing the model to focus on more relevant connections; GIN, which has improved expressive power, enabling it to distinguish different graph structures more effectively than previous models; GRAPHSAGE, an inductive framework that samples and aggregates features from node neighborhoods, particularly useful in terms of scalability and applicable to unseen nodes; and EvolveGCN, a dynamic graph neural network model that dynamically updates the weight parameters of GCN through an RNN structure. Capturing the characteristics of graph structure evolution over time; DySAT, which combines structural attention and temporal attention mechanisms, considers both the topological relationships of nodes and their temporal evolution patterns, making it suitable for dynamic graphs; T-GCN, which combines graph convolutional networks with gated recurrent units, simultaneously capturing spatial dependencies between nodes and temporal dependencies between different time steps; ASTGCN, an attention-based spatiotemporal graph convolutional network that integrates spatial and temporal attention layers, enabling it to effectively handle complex spatiotemporal dynamics in graph structure data; Neurograph, a comprehensive benchmark specifically designed for graph-based neuroimaging tasks, serving as a dedicated tool for evaluating models on neuroimaging datasets.

[0056] Table 1. Experimental Results of Different Methods

[0057]

[0058] Table 1 presents the evaluation results of the method of this invention and the key baseline model on the HCP S1200 dataset. Numerical values ​​represent the best prediction performance for each metric. The method of this invention (AFDB) achieved the best performance on all evaluation metrics. Compared to the best-performing baseline method, AFDB improved the RMSE and MAE metrics by 5.97% and 3.01%, respectively. Static graph neural network methods (such as GCN, GraphSAGE, and GAT) outperform pure deep learning methods by modeling spatial dependencies between brain regions, but fail to capture the dynamic characteristics of functional brain connectivity. In contrast, dynamic graph neural network methods (such as EvolveGCN and DySAT) further introduce temporal dimension information, thus outperforming static graph methods in overall performance. This invention also provides a working memory assessment device for dynamic brain network attention fusion, comprising: a region of interest extraction module, used to acquire functional magnetic resonance imaging (fMRI) data from several subjects, segment the time series of the BOLD signal in the fMRI data, and extract the time series data of the region of interest; and a preprocessing module, used to preprocess the time series data of the region of interest and segment the preprocessed time series data of the region of interest into multiple time window sequences, calculate the functional connectivity strength between time series within each time window, and construct a functional connectivity matrix using the Pearson correlation coefficient.

[0059] A fully connected graph construction module is used to extract and fuse the spatial dependencies and temporal fluctuation characteristics in the functional connectivity matrix based on an attention mechanism to obtain a fully connected graph.

[0060] The sample construction module is used to assign importance coefficients to nodes in the fully connected graph based on prior knowledge of neuroscience. Taking the target brain region in the fully connected graph as the core node, it selects neighbors to form clusters based on edge weights and neighbor importance. Multi-head attention is used to calculate the attention from unclustered nodes to each cluster, and the functional magnetic resonance imaging embedding representation of each subject is obtained.

[0061] The training module is used to input the functional magnetic resonance imaging (fMRI) embedding representation of each subject into the prediction network for training to obtain a trained prediction network.

[0062] The working memory assessment module is used to assess the working memory of target subjects using a trained prediction network.

[0063] Compared with the prior art, the beneficial effects of the working memory ability assessment device with dynamic brain network attention fusion provided by the present invention are the same as those of the working memory ability assessment method with dynamic brain network attention fusion described in the above technical solution, and will not be repeated here.

[0064] This invention also provides an electronic device, including a bus, a transceiver, a memory, a processor, and a computer program stored in the memory and executable on the processor. The transceiver, the memory, and the processor are connected via the bus. When the computer program is executed by the processor, it implements the steps in the aforementioned dynamic brain network attention fusion working memory ability assessment method. Compared with the prior art, the beneficial effects of the electronic device provided by this invention are the same as those of the dynamic brain network attention fusion working memory ability assessment method described above, and will not be repeated here.

[0065] The present invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps in the above-described dynamic brain network attention fusion working memory capacity assessment method. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present invention are the same as those of the dynamic brain network attention fusion working memory capacity assessment method described above, and will not be elaborated further here.

[0066] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. Regarding the methods disclosed in the embodiments, since they correspond to the apparatus disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the apparatus description.

[0067] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for evaluating working memory capacity of dynamic brain network attention fusion, characterized in that, The method comprises the following steps: S1, acquiring functional magnetic resonance imaging data of a plurality of subjects, segmenting BOLD signal time series in the functional magnetic resonance imaging data, and extracting time series data of a region of interest; S2, preprocessing the time series data of the region of interest and segmenting the preprocessed time series data of the region of interest into a plurality of time window sequences, calculating the functional connection strength between the time series in each time window, and constructing a functional connection matrix using a Pearson correlation coefficient; S3, extracting and fusing the spatial dependence and time fluctuation characteristics in the functional connection matrix based on an attention mechanism to obtain a full connection graph; S4, assigning an importance coefficient to each node in the full connection graph based on prior knowledge of neuroscience, taking a target brain area in the full connection graph as a core node, selecting neighbors to form clusters based on edge weights and neighbor importance, and calculating the attention of unclustered nodes to each cluster using multi-head attention to obtain a functional magnetic resonance imaging embedding representation of each subject; S5, inputting the functional magnetic resonance imaging embedding representation of each subject into a prediction network for training to obtain a trained prediction network; S6, using the trained prediction network to complete the working memory evaluation of a target subject.

2. The method of claim 1, wherein the method is used for evaluating the working memory capacity of the subject. In S2, the functional connectivity matrix is represented using dynamic brain functional connectivity graphs ; where M represents the number of graphs; each dynamic brain functional connectivity graph contains the same set of nodes , V represents the brain region of interest, but has different edge sets and node features ; where N is the number of nodes, and d is the feature dimension.

3. The method of claim 2, wherein the method is used for evaluating the working memory capacity of the dynamic brain network attention fusion. The S3 comprises the following steps: S31, calculating the attention weight between nodes in the dynamic brain functional connection graph; S32, extracting the corresponding attention weight from each node to form an attention matrix, and obtaining a cross-graph attention score of each node through an attention network; S33, normalizing the cross-graph attention score of each node to obtain a cross-graph attention weight, and fusing the cross-graph attention weight with the attention matrix to obtain a fused attention weight matrix; S34, stacking the node features in all dynamic brain functional connection graphs into a tensor and then calculating feature fusion weights through a feature attention network; S35, calculating and normalizing the feature attention score, and then fusing the feature fusion weights with the normalized feature attention score to generate a fused node feature matrix; S36, obtaining a full connection graph based on the fused node feature matrix and the fused attention weight matrix.

4. The method of claim 3, wherein the method is used for evaluating the working memory capacity of the dynamic brain network attention fusion. In S31, the expression of attention weight between nodes in the dynamic brain functional connectivity graph is calculated as: ; wherein, denote the query matrix and the key matrix, respectively, and is a learnable parameter matrix, is a projection dimension, d is a feature dimension, denotes the attention weight between nodes in the dynamic brain functional connectivity graph; in S33, the expression of the fused attention weight matrix is: ; wherein, denotes the cross-graph attention weight, denotes the fused attention weight matrix, denotes the attention weight vector corresponding to the node K in the i-th dynamic brain functional connectivity graph.

5. The method of claim 4, wherein the method is used for evaluating the working memory capacity of the dynamic brain network attention fusion. In S35, the fused node feature matrix is: ; ; wherein, denotes the fused node feature matrix, FS denotes the sum of feature attention scores, denotes the key matrix slice of the i-th dimension in the feature attention, denotes the query matrix slice of the i-th dimension in the feature attention, denotes the key matrix slice of the i-th dimension in the feature attention, denotes the query matrix slice of the i-th dimension in the feature attention, denotes the hidden dimension of the feature attention, denotes the original node feature matrix of the i-th dynamic brain functional connectivity graph, denotes the normalized feature attention weight vector.

6. The method of claim 5, wherein the method is used for evaluating the working memory capacity of the dynamic brain network attention fusion. The S4 comprises the following steps: S41, assigning an initial importance coefficient to each node in the full connection graph based on prior knowledge of neuroscience, and using a multi-layer perceptron to learn and adjust the importance coefficient of each node; S42, taking a key brain area as a core node, selecting a neighborhood based on edge weights and neighbor importance, and forming K clusters; S43, regarding each cluster as a node, and applying a multi-head attention mechanism to calculate the attention of unclustered nodes to each cluster. S44, using local residuals to add original unclustered node features to the attention aggregation output, using global residuals to add global average pooling representations of all nodes in the first layer to the final output to obtain a functional magnetic resonance imaging embedding representation of each subject.

7. The method of claim 6, wherein the method is used for evaluating the working memory capacity of the dynamic brain network attention fusion. In S42, a cluster representation is generated using a graph attention network: ; wherein, represents the graph attention network aggregation representation of the kth cluster, GAT represents the graph attention network, represents the feature of the jth node in the ith dynamic brain functional connectivity graph, represents that the node j belongs to the kth cluster.

8. The method of claim 7, wherein the method is used for evaluating the working memory capacity of the dynamic brain network attention fusion. The calculation formula of the attention of each cluster is: ; wherein, represents a query matrix containing all unclustered nodes, K represents a key matrix, V represents a value matrix, S represents a node importance coefficient, d represents a feature dimension, represents a normalization function.

9. The method of claim 8, wherein the method further comprises: The S5, input the functional magnetic resonance imaging embedding representation of each subject into the prediction network for training to obtain a trained prediction network, comprising: S51, convert the metadata in the functional magnetic resonance imaging data into a feature vector through feature engineering, project the functional magnetic resonance imaging embedding representation of each subject and the feature vector into a shared embedding space to form a training sample; S52, training a prediction network using the training samples to obtain a trained prediction network; wherein a loss function in the training process is: ; ; ; wherein, represents a contrast loss, represents a mean square error loss, L represents a total loss of the addition of the two, represents the similarity between the functional magnetic resonance imaging data embedding of the i th sample and the metadata embedding, represents a temperature hyperparameter, B represents a training batch size, represents the real working memory score of the i th sample, represents the predicted working memory score of the i th sample.

10. The working memory capacity evaluation device of dynamic brain network attention fusion, characterized in that, Comprising: The region of interest extraction module is used for acquiring functional magnetic resonance imaging data of a plurality of subjects, segmenting BOLD signal time series in the functional magnetic resonance imaging data, and extracting time series data of the region of interest; The preprocessing module is used for preprocessing the time series data of the region of interest and segmenting the preprocessed time series data of the region of interest into a plurality of time window sequences, calculating the functional connection strength between the time series in each time window, and constructing a functional connection matrix using a Pearson correlation coefficient; The fully connected graph construction module is used for extracting and fusing spatial dependence and time fluctuation characteristics in the functional connection matrix based on an attention mechanism to obtain a fully connected graph; The sample construction module is used for assigning an importance coefficient to a node in the fully connected graph based on neuroscience prior knowledge, taking a target brain area in the fully connected graph as a core node, selecting neighbors to form a cluster based on edge weights and neighbor importance, calculating attention of unclustered nodes to each cluster using multi-head attention to obtain a functional magnetic resonance imaging embedding representation of each subject; The training module is used for inputting the functional magnetic resonance imaging embedding representation of each subject into the prediction network for training to obtain a trained prediction network; The working memory evaluation module is used for completing working memory evaluation of a target subject using the trained prediction network.

Images