Brain decoding method based on multi-view self-supervised brain topology inference

Abstract

This application discloses a brain decoding method based on multi-view self-supervised brain topology inference, comprising: constructing an initial topological structure model based on preset brain region information and a graph learner; using the brain topology in the initial topological structure model as the first view brain topology; obtaining the connection relationships of different brain regions based on the brain adjacency matrix in the initial topological structure model, aggregating the original brain data according to the connection relationships to obtain a second view brain topology; updating the first view brain topology through the second view brain topology; stopping the update iteration when the mutual information between the two view brain topologies no longer changes, and using the target features in the obtained second view brain topology as the global brain features after decoding the original brain data. This method can adaptively update and learn the view brain topology based on the dynamic plasticity of the original brain data, avoiding the inheritance and superposition of erroneous information in the topology, and obtaining accurate and comprehensive decoding information.

Description

Technical Field

[0001] This disclosure relates to the field of electroencephalography (EEG) decoding technology, and in particular to a brain decoding method based on multi-view self-supervised brain topology inference. Background Technology

[0002] Brain decoding aims to translate brain activity or neural interactions into signals that external devices can understand and use in order to predict or understand specific cognitive states, external stimuli, or thought content. It is crucial for the development of technologies such as artificial vision, brain-computer interfaces (BCI), and virtual reality devices, and will help scientists gain a more comprehensive understanding of how the brain works.

[0003] Existing brain decoding methods based on multi-view self-supervised brain topology inference typically rely on prior knowledge to predefine the brain topology, i.e., predefine the brain topology based on a large amount of fully annotated brain data, expert knowledge, etc. They assume that the predetermined brain topology is reliable enough. Neuroscience research shows that the brain's connectivity patterns are dynamic, and brain decoding based on predefined topologies cannot reflect the actual dynamic changes, thus leading to inaccurate and unreliable decoding information. Summary of the Invention

[0004] In view of this, the present disclosure provides a brain decoding method based on multi-view self-supervised brain topology inference, which can solve the problems of inaccurate and unreliable decoding information in the prior art.

[0005] In a first aspect, embodiments of this disclosure provide a brain decoding method based on multi-view self-supervised brain topology inference, specifically including:

[0006] An initial topological model is constructed based on pre-defined brain region information and a graph learner.

[0007] The brain topology in the initial topology model is used as the first view brain topology.

[0008] Based on the brain adjacency matrix in the initial topological model, the connectivity relationships between different brain regions are obtained;

[0009] YY+242265P-YX

[0010] The original brain data is aggregated based on the connectivity relationships of different brain regions to obtain a second view of the brain topology.

[0011] The first view brain topology is updated using the second view brain topology;

[0012] When the mutual information between the second view brain topology and the first view brain topology no longer changes, the update iteration stops, and the target features in the obtained second view brain topology are used as the global brain features after decoding the original brain data.

[0013] Optionally, the step of constructing an initial topological structure model based on preset brain region information and a graph learner includes:

[0014] A fully parameterized matrix is ​​constructed based on a preset number of brain regions. Each element in the fully parameterized matrix corresponds to the connection relationship between different brain regions, and the different connections are independent.

[0015] Determine the nonlinear activation function;

[0016] Applying the nonlinear activation function to the fully parameterized matrix yields a positive correlation matrix;

[0017] An initial topological structure model is constructed based on the positive correlation matrix.

[0018] Optionally, the step of constructing an initial topological structure model based on preset brain region information and a graph learner includes:

[0019] Based on the preset number of brain regions and the number of feature types, a feature matrix for each brain region is predefined;

[0020] The feature matrix of each brain region is fused with the information in the preset database to obtain the fused feature matrix of each brain region;

[0021] Based on the fusion feature matrix of each brain region, the similarity between different brain regions is calculated;

[0022] An initial topological model is constructed based on similarity and pre-defined brain connectivity information.

[0023] Optionally, when the preset database information does not contain prior knowledge, the method for obtaining the fused feature matrix includes:

[0024] An attentional embedding network is used to obtain the attentional weights between each brain region and its neighboring brain regions in the feature matrix;

[0025] Based on the attention weights, different importance is assigned to each brain region feature to obtain a fusion feature matrix.

[0026] Optionally, when the preset database information does not contain prior knowledge, the method for obtaining the fusion feature matrix includes: using a multilayer perceptron to obtain the linear correlation relationship of the brain region representations in the feature matrix, and obtaining the fusion feature matrix based on the linear correlation relationship.

[0027] YY+242265P-YX

[0028] Optionally, when the preset database information contains prior knowledge, the method for obtaining the fused feature matrix includes:

[0029] The feature matrix is ​​fused with prior knowledge from a pre-defined database using a Generative Neural Network (GNN) to obtain a fused feature matrix.

[0030] The fused feature matrix is ​​optimized to obtain the fused feature matrix.

[0031] Optionally, updating the first view brain topology through the second view brain topology includes:

[0032] The brain data in the first view brain topology is enhanced to obtain a first enhanced brain topology and a first augmented feature matrix.

[0033] The brain data in the second view brain topology is enhanced to obtain a second enhanced brain topology and a second augmented feature matrix.

[0034] The first augmented feature matrix, the first augmented brain topology, the second augmented feature matrix, and the second augmented brain topology are input into the graph neural network encoder to obtain the first augmented feature map representation, the first augmented brain topology map representation, the second augmented feature map representation, and the second augmented brain topology map representation.

[0035] The first augmented feature map representation, the first enhanced brain topology map representation, the second augmented feature map representation, and the second enhanced brain topology map representation are mapped to obtain the first augmented feature mapping feature, the first enhanced brain topology mapping feature, the second augmented feature mapping feature, and the second enhanced brain topology mapping feature.

[0036] Based on the first augmented feature mapping feature and the second augmented feature mapping feature, obtain the mutual information from the first view to the second view;

[0037] Based on the first enhanced brain topology mapping feature and the second enhanced brain topology mapping feature, the mutual information from the second view to the first view is obtained;

[0038] Based on the mutual information from the first view to the second view and the mutual information from the second view to the first view, the contrast loss value is obtained;

[0039] The iterative update stops when the comparison loss value no longer changes.

[0040] Optionally, the step of enhancing the brain data in the first view brain topology to obtain a first enhanced brain topology and a first augmented feature matrix includes:

[0041] Based on the first adjacency matrix in the first view topology model, a first random masking probability matrix is ​​generated based on the Bernoulli distribution;

[0042] The first adjacency matrix is ​​randomly masked based on the first random masking probability matrix to obtain the first enhanced brain topology;

[0043] A second random masking probability matrix is ​​generated based on the brain time-series features in the first view topology model.

[0044] The feature vectors of each region of interest are masked based on the second random masking probability matrix to obtain the first augmented feature matrix.

[0045] Optionally, the step of enhancing the brain data in the second view brain topology to obtain a second enhanced brain topology and a second augmented feature matrix includes:

[0046] Based on the second adjacency matrix in the second view topology model, a third random masking probability matrix is ​​generated based on the Bernoulli distribution; the topological masking probability of the third random masking probability matrix is ​​the same as the topological masking probability of the first random masking probability matrix.

[0047] The second adjacency matrix is ​​randomly masked based on the third random masking probability matrix to obtain the second enhanced brain topology;

[0048] A fourth random masking probability matrix is ​​generated based on the brain time-series features in the second view topology model; the feature masking probability of the fourth random masking probability matrix is ​​different from the feature masking probability of the second random masking probability matrix.

[0049] The feature vectors of each region of interest are masked based on the fourth random masking probability matrix to obtain the second augmented feature matrix.

[0050] Optionally, the contrast loss value is L total ,

[0051] Among them, L I→L For mutual information from the first view to the second view, L L→I For mutual information from the second view to the first view, H ,i H is the first augmented feature mapping feature. I,j The first enhanced brain topological mapping feature, H L,i For the second augmented feature mapping feature, H L,j The second feature is the enhanced brain topological mapping feature, where τ is the temperature parameter, sim(·) is the cosine similarity function, and z is the number of different brain regions.

[0052] Secondly, embodiments of this disclosure also provide a brain decoding system based on multi-view self-supervised brain topology inference, comprising:

[0053] The initial topology model building module is used to build an initial topology model based on preset brain region information and a graph learner.

[0054] The first view brain topology acquisition module is used to obtain the first view brain topology based on the initial topology model.

[0055] The connectivity acquisition module is used to acquire connectivity relationships between different brain regions based on the brain adjacency matrix in the initial topological structure model.

[0056] The second-view brain topology acquisition module is used to aggregate raw brain data based on the connection relationships of different brain regions to obtain the second-view brain topology.

[0057] The update module is used to update the first view brain topology through the second view brain topology;

[0058] The analysis module is used to stop the update iteration when the mutual information between the second view brain topology and the first view brain topology no longer changes, and to use the target features in the obtained second view brain topology as the global brain features after decoding the original brain data.

[0059] This application discloses a brain decoding method based on multi-view self-supervised brain topology inference. By combining pre-set brain region information with a graph learner, it can quickly construct an initial topological structure of brain regions, providing a foundation for subsequent analysis and decoding. The graph learner can handle complex and irregular brain region connectivity relationships and is suitable for the nonlinear structure of the brain. Through a first-view brain topology structure based on the initial topological structure model and a second-view brain topology structure based on brain region connectivity relationships, this multi-view strategy provides more comprehensive and richer information for decoding brain activity. Iterative updates between the two views can gradually approximate the actual brain connectivity state, thereby further improving decoding accuracy. The second-view brain topology structure can be progressively optimized to more closely resemble the actual brain region connectivity pattern. The update process allows for dynamic adjustment of brain region connectivity relationships to adapt to changes in different tasks and data. Through the update process, the brain topology in the two views can be adjusted according to the dynamic plasticity of the original brain data.

[0060] Adaptive updating and learning can avoid the inheritance and superposition of erroneous information in the topology; when the mutual information no longer changes, it is considered to have reached a stable state, and the update iteration stops. The target features in the second-view brain topology are used as global brain features after decoding the original brain data, so as to obtain accurate and comprehensive decoding information.

[0061] The above description is merely an overview of the technical solution disclosed herein. In order to better understand the technical means of this disclosure and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0062] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0063] Figure 1 This is a flowchart illustrating the brain decoding method based on multi-view self-supervised brain topology inference provided in an embodiment of this disclosure.

[0064] Figure 2 This is a flowchart illustrating a first embodiment of the method for constructing an initial topology model provided in this disclosure.

[0065] Figure 3 A flowchart illustrating a second embodiment of the method for constructing an initial topology model provided in this disclosure.

[0066] Figure 4 This is a flowchart illustrating a first embodiment of the method for obtaining a fusion feature matrix provided in this disclosure.

[0067] Figure 5 A flowchart illustrating a second embodiment of the method for obtaining a fusion feature matrix provided in this disclosure.

[0068] Figure 6 This is a flowchart illustrating a method for updating the brain topology of a first view using a second view brain topology, as provided in an embodiment of this disclosure.

[0069] Figure 7 for Figure 6 The flowchart illustrates a method for enhancing brain data in a first-view brain topology to obtain a first enhanced brain topology and a first augmented feature matrix.

[0070] YY+242265P-YX

[0071] Figure 8 for Figure 6 The flowchart illustrates the method for enhancing brain data in the second-view brain topology to obtain a second enhanced brain topology and a second augmented feature matrix.

[0072] Figure 9 This is a schematic diagram of a brain decoding framework based on multi-view self-supervised brain topology reasoning provided in an embodiment of this disclosure. Detailed Implementation

[0073] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0074] It should be understood that the following specific examples illustrate the implementation of this disclosure, and those skilled in the art can easily understand other advantages and effects of this disclosure from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. This disclosure can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this disclosure. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0075] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this disclosure, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.

[0076] It should also be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this disclosure. The drawings only show the components related to this disclosure and are not drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0077] YY+242265P-YX

[0078] Furthermore, specific details are provided in the following description to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that the described aspects can be practiced without these specific details.

[0079] Reference Figure 1This application discloses a brain decoding method based on multi-view self-supervised brain topology inference, which specifically includes the following:

[0080] S100 constructs an initial topological structure model based on preset brain region information and a graph learner.

[0081] S200, the brain topology in the initial topology model is used as the first view brain topology.

[0082] The first view can be labeled as the "guide" view. The first view of the brain topology provides a preliminary understanding of the connectivity between brain regions, which facilitates subsequent improvements and optimizations.

[0083] S300 obtains the connectivity relationships between different brain regions based on the brain adjacency matrix in the initial topological model.

[0084] S400 aggregates raw brain data based on the connectivity relationships of different brain regions to obtain a second-view brain topology.

[0085] The raw brain data can be either fMRI data or EEG data.

[0086] The second view can be labeled as the "learner" view. The "learner" view directly uses the brain adjacency matrix and brain time series learned by BTI as the feature matrix, and the brain adjacency matrix is ​​gradually updated with the model parameters.

[0087] In this embodiment, the brain time series of the brain adjacency matrix is ​​used as the feature matrix of the brain topology in the second view (“learner” view).

[0088] By aggregating raw brain data, we can enhance the data's representational power and help to better capture the global features of brain regions. The aggregation process combines data from different brain regions, which helps to reveal the collaborative working patterns of brain regions.

[0089] S500 updates the first view brain topology through the second view brain topology;

[0090] When the mutual information between the second-view brain topology and the first-view brain topology no longer changes, the update iteration stops, and the target features in the obtained second-view brain topology are used as the global brain features after decoding the original brain data.

[0091] The brain decoding method disclosed in this application, based on multi-view self-supervised brain topology inference, uses pre-YY+242265P-YX

[0092] The combination of brain region information and graph learners enables the rapid construction of initial brain region topology, providing a foundation for subsequent analysis and decoding. Graph learners can handle complex and irregular brain region connectivity, making them suitable for the nonlinear structure of the brain. This multi-view strategy, using a first-view brain topology based on the initial topology model and a second-view brain topology based on brain region connectivity, provides more comprehensive and richer information for decoding brain activity. Iterative updates between the two views gradually approximate the actual brain connectivity state, further improving decoding accuracy. The second-view brain topology can be progressively optimized to more closely resemble the actual brain region connectivity pattern. The update process allows for dynamic adjustments to brain region connectivity to adapt to changes in different tasks and data. Through the update process, the brain topology in both views can be adaptively updated and learned based on the dynamic plasticity of the original brain data, avoiding the inheritance and superposition of erroneous information in the topology. When the mutual information no longer changes, a stable state is considered reached, and the update iteration stops. The target features in the second-view brain topology are then used as the global brain features after decoding the original brain data, obtaining accurate and comprehensive decoding information.

[0093] This scheme, through multi-step iterative optimization, can efficiently decode raw brain data and extract representative global brain features. Using a graph learner and brain adjacency matrix, it can accurately capture the connectivity relationships between brain regions, improving decoding accuracy. It possesses dynamic adjustment capabilities, adapting to different brain data and task requirements, and exhibits strong robustness. Through aggregation and updating processes, it can provide a global perspective on brain function and structure, revealing the collaborative working patterns of brain regions. The generated topological structures and features have good visualization and interpretability, facilitating understanding and analysis by researchers. Based on these steps and benefits, this brain decoding method based on multi-view self-supervised brain topology inference has broad application prospects in fields such as brain science research, medical diagnosis, and brain-computer interfaces.

[0094] Reference Figure 2 Regarding the method for constructing the initial topology model, in the first embodiment, parameter-driven BTI (brain topology inference) is adopted, specifically including:

[0095] S111, a fully parameterized matrix is ​​constructed based on a preset number of brain regions. Each element in the fully parameterized matrix corresponds to the connection relationship between different brain regions, and the different connections are independent.

[0096] Each element can be a positive number (indicating positive correlation) or a negative number (indicating negative correlation), meaning the fully parameterized matrix includes both positive and negative correlations.

[0097] Suppose the fully parameterized matrix is ​​W, W∈R N×N Each element w in the fully parameterized matrixij YY+242265P-YX

[0098] This represents the connection strength between node i (i.e., brain region i) and node j (i.e., brain region j). For brain data, the interaction between regions of interest (ROIs) can be viewed as the topological structure of a graph, and the information changes in ROIs can be abstracted as node features.

[0099] Specifically, the interconnections in the brain are simulated by initializing a fully parameterized matrix. No additional task-related model input is required. An independent parameter of size N×N is directly initialized, where each element is used to simulate each interconnection in the brain topology, that is, the connection relationship (or connection weight) between different brain regions. The preset number of brain regions is N+1.

[0100] This step allows for a comprehensive consideration of all possible connections between each brain region and other brain regions, ensuring the comprehensiveness and integrity of the model.

[0101] S112, determine the nonlinear activation function.

[0102] Specifically, to ensure the stability of model training, a non-linear activation function σ(.) is introduced. This activation function can be a common non-linear function such as ReLU, Sigmoid, or Tanh.

[0103] Furthermore, activation functions can be selected according to specific needs. For example, ReLU is used to enhance positive correlations, while Sigmoid is used to balance positive and negative correlations. Nonlinear activation functions can better capture complex nonlinear relationships between brain regions and enhance the expressive power of the model.

[0104] S113, apply a nonlinear activation function to the fully parameterized matrix to obtain the positive correlation matrix.

[0105] Wherein, the positive correlation matrix is ​​A, A p-BTI =σ(W).

[0106] Specifically, by applying a nonlinear activation function to the fully parameterized matrix, the activated matrix is ​​obtained. This activated matrix can be directly used as the positive correlation matrix, representing the brain region connectivity after nonlinear transformation. Because activation functions suppress negative values ​​to zero (such as ReLU) or map them to a small value (such as Sigmoid and Tanh), positive values ​​in A can be considered as positively correlated connections. That is, by processing the fully parameterized matrix with a nonlinear activation function, the negatively correlated parts can be filtered out, while the positively correlated parts are retained.

[0107] Positive correlation means that if two nodes are strongly connected in the brain, then this strong connection should also be reflected in the positive correlation matrix.

[0108] Furthermore, the activated matrix can be normalized, and the normalized matrix can be used as the positive correlation matrix.

[0109] S114, Construct an initial topological structure model based on the positive correlation matrix.

[0110] Specifically, the connection weights in the positive correlation matrix are transformed into a network representation of inter-brain connections to generate an initial brain region functional network model (i.e., an initial topological structure model).

[0111] Furthermore, some optimization algorithms (such as gradient descent, stochastic gradient descent, etc.) can be used to train this topological model so that it can better simulate the state and behavior of the brain.

[0112] This embodiment discloses a method for constructing an initial topological model using parameter-driven BTI. Through fully parameterized matrices and nonlinear activation functions, it comprehensively considers and analyzes the complex relationships between brain regions, improving the model's accuracy and reliability. The construction of the positive correlation matrix simplifies the model's complexity, making subsequent analysis and optimization more efficient. The construction of the topological model makes the functional network relationships between brain regions intuitively visible, facilitating researchers' understanding and interpretation of brain functional networks. The parameterized matrices and nonlinear activation functions in the entire scheme give the model high flexibility and adaptability, enabling it to cope with different data characteristics and research needs. Through these steps, the overall scheme provides a comprehensive, accurate, and flexible tool for analyzing brain functional networks, contributing to in-depth insights and achievements in neuroscience and brain function research.

[0113] Reference Figure 3 Regarding the method for constructing the initial topology model, in the second embodiment, a similarity-driven BTI (brain topology inference) is adopted, specifically including:

[0114] S211, based on the preset number of brain regions and the number of feature types, predefine the feature matrix for each brain region.

[0115] Specifically, based on the preset number of brain regions N and the number of feature types d X Predefine (i.e. construct) the feature matrix for each brain region, and the size of the feature matrix.

[0116] S212, fuse the feature matrix of each brain region with the preset database information to obtain the fused feature matrix of each brain region.

[0117] By integrating information from a pre-set database, more historical data can be used to enrich the feature matrix, thereby improving the model's generalization ability and prediction accuracy.

[0118] S213 calculates the similarity between different brain regions based on the fusion feature matrix of each brain region.

[0119] Specifically, based on the fusion feature matrix, cosine similarity or other distance metrics can be used to calculate the similarity between different brain regions.

[0120] Similarity calculation can quantify the feature similarity between different brain regions, providing a reference for constructing topological structures; similarity indicators can intuitively reflect the feature relationships between brain regions, making it easier for researchers to understand and interpret model results.

[0121] S214, based on similarity and pre-set brain connectivity information, constructs an initial topological structure model.

[0122] Specifically, an initial topological model is constructed by combining the calculated similarity matrix S with pre-defined brain connectivity information (such as anatomical or functional connectivity data). A graph structure can be used to represent the relationships between brain regions, where nodes represent brain regions and the weights of edges are determined by both similarity and pre-defined connectivity information. For example, the edge weight W... ij It can be defined as:

[0123] W ij =βS ij +(1-β)C ij ; where C ij β is the preset connection strength, and β is the weight parameter.

[0124] For S212, when the preset database information does not contain prior knowledge, the following two schemes can be used to obtain the fused feature matrix: Scheme 1 is BTI based on the attention mechanism, Scheme 2 is BTI based on MLP, and Scheme 3 is BTI based on ROI specific pattern learning (RSPL-BTI). Brain topology learning is performed on completely unlabeled brain data, and the decoding process is based on the learned brain topology. The brain encoders in these two steps share parameters and have a completely consistent structure.

[0125] Reference Figure 4 The method for obtaining the fusion feature matrix using Scheme 1 specifically includes:

[0126] A100 uses an attention-embedded network to obtain the attention weights between each brain region and its neighboring brain regions in the feature matrix.

[0127] Here, each brain region refers to each node, and adjacent brain regions refer to adjacent nodes.

[0128] Specifically, e i,j =LeakyReLU(a T [Whi ||Wh j ]), where a is the attention vector, W is the weight matrix, and h i and h j e is the feature vector of brain region i and brain region j. i,j It is the attention score (i.e., attention weight) between brain region i and brain region j.

[0129] A200 assigns different importance to each brain region feature based on attention weights, resulting in a fusion feature matrix.

[0130] The calculation of attention weights typically involves a weighted summation of each feature vector in the feature matrix E1. The weights are dynamically generated by the attention mechanism and reflect the contribution of different brain region features to the topological structure.

[0131] Furthermore, the fused feature matrix may include:

[0132] σ is a nonlinear activation function, N i It is the set of neighboring nodes of brain region i.

[0133] Attention-based BTI employs an embedding network similar to graph attention networks to refine the representation of ROIs. Each embedding network layer computes the Hadamard product of the input ROI embedding E and the parameter vector: in, e represents the output of the l-th layer of the attention-based embedding network. l-1 express The transpose of the i-th row vector is given, where ⊙ represents the Hadamard product, t represents the transpose, and σ(.) also represents a nonlinear operation. It is worth noting that the input of the first layer... Initialized as a feature matrix X, the output of the last layer is used as the final ROI embedding. (i.e., the corresponding features in the fused feature matrix). Attention-based embedding networks assume that each ROI feature contributes differently to the brain topology and that there is no significant correlation between features.

[0134] During the brain's performance of different functions, the activation levels and collaborative patterns of different brain regions often vary. To better describe these differential representations, based on a predefined feature matrix for each brain region, different importance is adaptively assigned to each region's features. The model automatically captures the complex interactive connections between brain regions to obtain a fused feature matrix. This approach assumes no correlation between features between brain regions while taking into account the contribution of different brain region features to the brain's topology.

[0135] The method of obtaining the fusion feature matrix using Scheme 2 includes: using a multilayer perceptron to obtain the linear correlation between the brain regions represented in the feature matrix, and obtaining the fusion feature matrix based on the linear correlation.

[0136] Specifically, MLP-based BTI uses an MLP as the embedding network for mining ROI representations. Each layer of the MLP captures the correlation between ROI features through linear combination of features and non-linear activation functions. Compared to attention-based BTI, MLP-based BTI further considers the linear combination of correlations between different ROI features.

[0137] For an MLP-based embedding network, the input is a feature matrix, and the output is a fused feature matrix. Each layer of an MLP-based embedding network can be represented as:

[0138] in, This refers to the output of layer l-1 (i.e., the input of layer l) in an MLP-based embedded network. This is the weight matrix for the (l-1)th layer.

[0139] Specifically, the weight matrix The features of different ROIs are linearly combined, and the representational power of the features is enhanced by a non-linear activation function.

[0140] Option 3 is ROI-specific pattern learning-based BTI (RSPL-BTI). Specifically, the specific pattern of each ROI is introduced into the iteration and update of the GNN so as to learn the brain topology while adaptively updating the ROI-specific pattern.

[0141] In GNN, the input is a feature matrix, and the output is a fused feature matrix, which is a feature matrix that includes ROI-specific patterns. The process includes: embedding ROI-specific patterns into the iteration and update process of GNN by introducing specific patterns for each ROI.

[0142] H GNN =(I+softmax(σ(E·E)) T )))XEW+Eb, where H GNN The learned brain spatial representation is represented by I, which is the identity matrix, and W and b are the parameters of the GNN. The softmax function is used for topological normalization, and σ(.) represents nonlinear operation.

[0143] The original GNN shares weights and biases among all ROIs, allowing the model to learn common patterns among ROIs (Regions of Interest). However, previous research has shown that different ROIs have their own unique patterns when they work together to perform brain activities, and considering only common patterns among ROIs may affect decoding performance.

[0144] Reference Figure 5 For S212, when the preset database information contains prior knowledge, a GNN-based BTI is used. The specific methods for obtaining the fused feature matrix include:

[0145] B100 retrieves prior knowledge from a preset database.

[0146] B200 uses GNN to fuse the feature matrix with prior knowledge to obtain a fused feature matrix.

[0147] BTI based on GNN integrates the feature matrix (i.e., ROI features) and prior knowledge from the pre-defined database into the embedding E through GNN.

[0148] The fusion feature matrix is Among them, A p To predefine the feature matrix for each brain region, D p For A p The degree matrix, θ l-1 For the parameters of the (l-1)th layer of the GNN, Let I be the weight matrix of the (l-1)th layer, I be the identity matrix, and σ() be the nonlinear operation.

[0149] The BTI hypothesis based on GNNs assumes that the topological connectivity between ROIs is related not only to ROI features, but also to a predefined brain topology.

[0150] In this embodiment, the "mentor" view provides correct and effective guidance to the "learner" view. For BTO (braintopology optimization), which is the case where the brain topology can be predefined given prior knowledge, the predefined brain topology can be obtained through prior knowledge, and the "mentor" view is initialized to the predefined brain topology. For BTE (brain topology extrapolation), which is the case where prior knowledge is unavailable, the "mentor" view is initialized to the identity matrix.

[0151] Furthermore, this application also includes: optimizing the fusion feature matrix to obtain the fusion feature matrix.

[0152] To obtain the brain adjacency matrix in the initial topological model, considering that the interactions between ROIs are usually undirected and that not all ROIs are highly activated when the brain is working, this application also includes: symmetric and sparse initialization of the brain topology, where the symmetric sparse matrix can greatly improve the computational efficiency and scalability of GNN.

[0153] Wherein, top-TSR(.) represents the top TSR% brain connectivity with the strongest association (topology sparsity ratio, TSR).

[0154] Further symmetric and normalized A is as follows:

[0155]

[0156] Among them, A sym and A norm Representing symmetric and normalized brain topologies respectively, the final A norm As the optimized brain adjacency matrix.

[0157] Reference Figure 6 Methods for updating the brain topology of the first view using the brain topology of the second view include:

[0158] C100 performs enhancement processing on the brain data in the first view brain topology to obtain the first enhanced brain topology and the first augmented feature matrix.

[0159] C200 performs augmentation processing on the brain data in the second-view brain topology to obtain the second augmented brain topology and the second augmented feature matrix.

[0160] In this embodiment, the enhancement processing performed on the brain data includes brain topological masking and brain feature masking processing.

[0161] After data augmentation, the augmented topology of the first view (i.e., the "guide" view) can be represented as: G I =(X1) augumentation A1 augumentation The enhanced topology of the second view (i.e., the "learner" view) can be represented as: G L =(X2) augumentation A2 augumentation ); where A1 augumentation Representing the first enhanced brain topology, X1 augumentation A2 is the first augmented characteristic matrix. augumentation Represents the second enhanced brain topology, X2 augumentation This is the second augmented characteristic moment.

[0162] C300 inputs the first augmented feature matrix, the first augmented brain topology, the second augmented feature matrix, and the second augmented brain topology to the graph neural network encoder to obtain the first augmented feature map representation, the first augmented brain topology map representation, the second augmented feature map representation, and the second augmented brain topology map representation.

[0163] Specifically, GNNs are used as brain encoders to extract spatial representations of different ROIs from the “instructor” and “learner” views.

[0164] The spatial representation extracted from the first view (i.e., the "guide" view) includes:

[0165] The spatial representation extracted from the second view (i.e., the "learner" view) includes:

[0166] and d GNN This serves as the feature dimension. By stacking l GNN layers, fine spatial information about the brain is gradually extracted.

[0167] Furthermore, by inputting the first augmented feature matrix, the first augmented brain topology, the second augmented feature matrix, and the second augmented brain topology into the graph neural network encoder, the obtained first augmented feature map is represented as follows: The first enhanced brain topology diagram is represented as follows: The second augmented feature map is represented as follows: The second enhanced brain topology map is represented as follows:

[0168] C400 maps the first augmented feature map representation, the first enhanced brain topology map representation, the second augmented feature map representation, and the second enhanced brain topology map representation to obtain the first augmented feature mapping feature, the first enhanced brain topology mapping feature, the second augmented feature mapping feature, and the second enhanced brain topology mapping feature.

[0169] Specifically, after the GNN encoder, the features of the two views are processed through a fully connected layer (multilayer perceptron layer, MLP). and Perform mapping, that is, represent the first augmented feature map. First Enhanced Brain Topology Map Second augmented feature map Second Enhanced Brain Topology Map Mapping is performed to facilitate multi-view loss calculation.

[0170] First augmented feature mapping feature:

[0171] First enhanced brain topological mapping feature:

[0172] Second augmented feature mapping features:

[0173] Second enhanced brain topological mapping feature:

[0174] C500 obtains mutual information from the first view to the second view based on the first augmented feature mapping feature and the second augmented feature mapping feature.

[0175] The mutual information between the first view and the second view is L. I→L :

[0176] Among them, H I,i H is the first augmented feature mapping feature. I,j The first enhanced brain topological mapping feature, H L,i For the second augmented feature mapping feature, H L,j The second feature is the enhanced brain topological mapping feature, where τ is the temperature parameter, sim(·) is the cosine similarity function, and z is the number of different brain regions.

[0177] C600 obtains mutual information from the second view to the first view based on the first enhanced brain topology mapping feature and the second enhanced brain topology mapping feature.

[0178] The mutual information between the second view and the first view is L. L→I :

[0179] C700 obtains the contrast loss value based on the mutual information from the first view to the second view and the mutual information from the second view to the first view;

[0180] The iterative update stops when the comparison loss value no longer changes.

[0181] In this embodiment, the contrast loss value is L total ,

[0182] Reference Figure 7 The brain data in the first view brain topology structure are enhanced to obtain a first enhanced brain topology structure and a first augmented feature matrix, including:

[0183] C110, based on the first adjacency matrix in the first view topology model, generates the first random masking probability matrix based on the Bernoulli distribution.

[0184] Specifically, brain connections can be randomly masked by proportionally discarding them (topological masking rate, rT). Further, for the first adjacency matrix in the first-view topological model, based on probability p(a1)... I The Bernoulli distribution generates the first random masking probability matrix m. a1 m a1 ∈R N×N , where a1 is the adjacency matrix.

[0185] C120, based on the first random masking probability matrix, performs random masking on the first adjacency matrix to obtain the first enhanced brain topology.

[0186] Specifically, based on m a1 Randomly mask a1: Among them, A1 augumentation Represents the first enhanced brain topology, It is the topology enhancement strategy function.

[0187] C130 generates a second random masking probability matrix based on the brain time-series features in the first-view topological model.

[0188] Specifically, a subset of features with rF (feature masking rate) is randomly selected and zero-masked. Specifically, for the brain time-series feature (X1) in the first-view topological model, it is first based on probability p(X1). I The Bernoulli distribution generates the random masking probability matrix m X1 ∈R d .

[0189] C140, based on the second random masking probability matrix, masks the feature vector of each region of interest to obtain the first augmented feature matrix.

[0190] Specifically, based on m X1 Mask the feature vectors of each ROI:

[0191] X1 augumentation =F fm (X)=[X1⊙m X ,...,X N ⊙m X ];

[0192] Among them, X1 augumentation Let F be the first augmented characteristic matrix. fm For the feature enhancement policy function, X1-X N The feature vector for each ROI (i.e., the feature vector of each ROI in the brain time series features in the first-view topological model).

[0193] Reference Figure 8 The brain data in the second-view brain topology is augmented to obtain a second augmented brain topology and a second augmented feature matrix, including:

[0194] C210, based on the second adjacency matrix in the second view topology model, a third random masking probability matrix is ​​generated based on the Bernoulli distribution; the topology masking probability of the third random masking probability matrix is ​​the same as the topology masking probability of the first random masking probability matrix.

[0195] Specifically, brain connections can be randomly masked by proportionally discarding them (topological masking rate, rT). Further, for the second adjacency matrix in the second-view topological model, based on probability p(a²), L The Bernoulli distribution generates the second random masking probability matrix m a2 m a2 ∈R N×N , where a2 is the second adjacency matrix.

[0196] C220, based on the third random masking probability matrix, performs random masking on the second adjacency matrix to obtain the second enhanced brain topology.

[0197] Specifically, based on m a2 Randomly mask a2: Among them, A2 augumentation Represents the second enhanced brain topology, It is the topology enhancement strategy function.

[0198] C230 generates a fourth random masking probability matrix based on the brain time-series features in the second-view topological model; the feature masking probability of the fourth random masking probability matrix is ​​different from that of the second random masking probability matrix.

[0199] YY+242265P-YX

[0200] Specifically, a subset of features with rF (feature masking rate) is randomly selected and zero-masked. Specifically, for the brain time-series features (X2) in the second-view topological model, firstly, based on probability p(X2)... L The Bernoulli distribution generates the random masking probability matrix m X2 ∈R d .

[0201] C240, based on the fourth random masking probability matrix, masks the feature vectors of each region of interest to obtain the second augmented feature matrix.

[0202] Specifically, based on m X2 Mask the feature vectors of each ROI:

[0203] X2 augumentation =F fm (X)=[X1⊙m X ,...,X N ⊙m X ];

[0204] Among them, X2 augumentation For the second augmented characteristic matrix, F fm For the feature enhancement policy function, X1-X N The feature vector for each ROI (i.e., the feature vector of each ROI in the brain time series features in the second-view topological model).

[0205] Since both views use the same brain features, different feature masking probabilities are employed to ensure feature variability. For brain topology, the brain region connections under the "instructor" view and the "learner" view are inherently very different, so this invention uses the same topological masking probability.

[0206] This invention evaluates the performance of the proposed framework on the internationally published Human Connectome Project (HCP) dataset, which mainly consists of two parts: task-related data (Task-HCP) and resting state data (Rest-HCP). The proposed model was tested on both datasets. Furthermore, considering the computational speed and high difficulty of information fitting for multiple ROIs in unsupervised scenarios, the whole brain voxels are divided into 22 ROIs (N=22) for subsequent self-supervised learning optimization, ultimately benefiting from brain decoding. This invention uses four metrics—classification accuracy, area under the curve (AUC), precision, and recall—to verify the effectiveness of the proposed method.

[0207] To evaluate the decoding performance of the proposed method, it was compared with several state-of-the-art (SOTA) supervised and unsupervised brain analysis methods with multiple metrics. Supervised methods included 1) a 2-layer MLP method based on 256 hidden nodes per layer; 2) a series of GNN-based brain spatial information mining methods, including Graph Convolutional Networks (GCN), Graph Attention Networks (GAT) (6 convolutional layers, each containing 32 neurons), GroupINN, BrainGNN, and MVS-GCN; and 3) supervised GTL methods, including TAGNN and DAST-GNN.

[0208] Five state-of-the-art unsupervised fMRI analysis methods were used: 1) BrainUSL; 2) CFCGL; 3) BrainGSL; 4) GATE; and 5) PTGB. Means and variances were reported for all comparison methods, and quantitative results are shown in Tables 1 and 2.

[0209] Table 1 illustrates the decoding performance of the proposed method and the state-of-the-art method on Rest-HCP (mean value).

[0210]

[0211] As shown in Table 1, on the Rest-HCP dataset, the MLP-based method significantly outperforms various supervised GNN methods based on spatial brain features in decoding performance. Compared to most supervised GNN methods based on predefined brain topologies and multi-view self-supervised brain topology inference, TAGCN and DAST-GCN incorporate the dynamic characteristics of brain topology and automatically update and optimize the interaction relationships between brain regions through parameterization. The results show that the dynamic topology-based method significantly outperforms the predefined topology-based GNN method. Further comparison of the proposed method with several state-of-the-art graph-supervised multi-view self-supervised brain topology inference decoding methods reveals that while existing unsupervised methods achieve performance comparable to supervised GNN methods, their limitations are still significant, and decoding accuracy needs improvement. The proposed method achieves good results on multiple classification metrics, exceeding existing unsupervised GNN methods based on spatial brain features by approximately 9%.

[0212] Table 2 illustrates the decoding performance of the proposed method and the state-of-the-art method on Task-HCP (mean value).

[0213]

[0214] In Table 2, '\' indicates that the model does not converge. Table 2 shows that the decoding performance on the Task-HCP dataset is similar to that on the Rest-HCP dataset. The MLP-based method significantly outperforms the spatial brain feature-based method, while the GNN-based dynamic topology decoding method has a significant advantage over predefined topology methods. This is highly consistent with previous research finding that brain topology dynamically updates over time, and simulating this dynamic change is crucial for brain decoding. Furthermore, for the GNN-based self-supervised multi-view self-supervised brain topology inference brain decoding method, the BrainUSL model does not converge on Task-HCP, and the decoding performance of other methods for different tasks still needs improvement. BTI-MVCSL has a significant advantage over these methods, achieving approximately 20% performance improvement compared to previous unsupervised decoding methods, demonstrating the effectiveness of the proposed framework.

[0215] Simultaneously refer to Figure 9This application also proposes a novel brain decoding framework based on multi-view self-supervised brain topology reasoning, including a brain topology inference (BTI) module, a multi-view contrastive self-supervised learning (MVCSL) module, and a decoding module. The brain topology inference (BTI) module includes a series of graph learners for constructing appropriate “learner” view topologies. The multi-view contrastive self-supervised learning (MVCSL) optimizes the brain topology structure through consistency constraints between the “instructor” and “learner” views and achieves persistent optimization through a self-updating learning mechanism (SLM). The brain decoding identifies the brain state under different downstream tasks.

[0216] YY+242265P-YX

[0217] The scheme disclosed in this embodiment can dynamically embed brain topology without data labels. It mainly includes two new modules to improve existing graph self-supervised methods for brain data analysis: 1) Brain Topology Inference (BTI) updates the interactive connectivity relationships between ROIs. This invention attempts various currently popular graph learners and further compares their decoding performance; 2) Multi-view Constrained Self-Supervised Learning (MVCSL) includes a "supervisor" view and a "learner" view, aiming to promote model optimization through multi-view comparison. Furthermore, to avoid the inheritance and superposition of erroneous information in the predefined topology, a novel self-learning mechanism—SLM—is proposed, enabling the learning of the optimal brain decoding topology. Extensive experiments on two datasets demonstrate the effectiveness of this framework. This invention plays an important role in deeply understanding brain structure and function, and in understanding brain information separation and integration. By mimicking the brain network's interactive information processing mechanism, more efficient and intelligent algorithms can be developed, further promoting the development of the field of general artificial intelligence.

[0218] Secondly, embodiments of this disclosure also provide a brain decoding system based on multi-view self-supervised brain topology inference, comprising:

[0219] The initial topology model building module is used to build an initial topology model based on preset brain region information and a graph learner.

[0220] The first-view brain topology acquisition module is used to obtain the first-view brain topology based on the initial topology model.

[0221] The connectivity acquisition module is used to acquire connectivity relationships between different brain regions based on the brain adjacency matrix in the initial topological model.

[0222] The second-view brain topology acquisition module is used to aggregate raw brain data based on the connection relationships of different brain regions to obtain the second-view brain topology.

[0223] The update module is used to update the first view brain topology using the second view brain topology.

[0224] The analysis module stops the update iteration when the mutual information between the second-view brain topology and the first-view brain topology no longer changes, and uses the target features in the obtained second-view brain topology as the global brain features after decoding the original brain data.

[0225] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details of YY+242265P-YX disclosed above are for illustrative and illustrative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing these specific details for implementation.

[0226] In this disclosure, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The block diagrams of devices, apparatuses, devices, and systems involved in this disclosure are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as "comprising," "including," "having," etc., are open-ended terms meaning "including but not limited to," and are used interchangeably with them. The terms "or" and "and" as used herein refer to the terms "and / or," and are used interchangeably with them unless the context clearly indicates otherwise. The term "such as" as used herein refers to the phrase "such as but not limited to," and is used interchangeably with it.

[0227] Additionally, as used herein, the “or” used in a list of items beginning with “at least one” indicates a separate list, such that a list of, for example, “at least one of A, B, or C” means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word “exemplary” does not imply that the described example is preferred or better than other examples.

[0228] It should also be noted that in the systems and methods of this disclosure, the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions to this disclosure.

[0229] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufactures, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Therefore, the appended claims include such processes, machines, manufactures, events, means, methods, or actions within their scope.

[0230] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this disclosure.

[0231] The scope of this disclosure is not intended to be limited to the aspects shown herein, but rather to the widest extent consistent with the principles and novel features disclosed herein.

[0232] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. A brain decoding method based on multi-view self-supervised brain topology inference, characterized in that, include: An initial topological model is constructed based on pre-defined brain region information and a graph learner. The brain topology in the initial topology model is used as the first view brain topology. Based on the brain adjacency matrix in the initial topological model, the connectivity relationships between different brain regions are obtained; The original brain data is aggregated based on the connectivity relationships of different brain regions to obtain a second view of the brain topology. The first view brain topology is updated using the second view brain topology; When the mutual information between the second view brain topology and the first view brain topology no longer changes, the update iteration stops, and the target features in the obtained second view brain topology are used as the global brain features after decoding the original brain data. The step of updating the first view brain topology using the second view brain topology includes: performing enhancement processing on the brain data in the first view brain topology to obtain a first enhanced brain topology and a first augmented feature matrix; performing enhancement processing on the brain data in the second view brain topology to obtain a second enhanced brain topology and a second augmented feature matrix; inputting the first augmented feature matrix, the first enhanced brain topology, the second augmented feature matrix, and the second enhanced brain topology into a graph neural network encoder to obtain a first augmented feature map representation, a first enhanced brain topology map representation, a second augmented feature map representation, and a second enhanced brain topology map representation; and processing the first augmented feature map representation... The first augmented brain topology map representation, the second augmented feature map representation, and the second augmented brain topology map representation are mapped to obtain the first augmented feature mapping feature, the first augmented brain topology mapping feature, the second augmented feature mapping feature, and the second augmented brain topology mapping feature. Based on the first augmented feature mapping feature and the second augmented feature mapping feature, the mutual information from the first view to the second view is obtained. Based on the first augmented brain topology mapping feature and the second augmented brain topology mapping feature, the mutual information from the second view to the first view is obtained. Based on the mutual information from the first view to the second view and the mutual information from the second view to the first view, the contrast loss value is obtained. The iterative update stops when the contrast loss value no longer changes.

2. The brain decoding method based on multi-view self-supervised brain topology inference according to claim 1, characterized in that, The construction of the initial topological structure model based on preset brain region information and a graph learner includes: A fully parameterized matrix is ​​constructed based on a preset number of brain regions. Each element in the fully parameterized matrix corresponds to the connection relationship between different brain regions, and the different connections are independent. Determine the nonlinear activation function; Applying the nonlinear activation function to the fully parameterized matrix yields a positive correlation matrix; An initial topological structure model is constructed based on the positive correlation matrix.

3. The brain decoding method based on multi-view self-supervised brain topology inference according to claim 1, characterized in that, The construction of the initial topological structure model based on preset brain region information and a graph learner includes: Based on the preset number of brain regions and the number of feature types, a feature matrix for each brain region is predefined; The feature matrix of each brain region is fused with the information in the preset database to obtain the fused feature matrix of each brain region; Based on the fusion feature matrix of each brain region, the similarity between different brain regions is calculated; Based on similarity and pre-defined brain connectivity information, an initial topological structure model is constructed.

4. The brain decoding method based on multi-view self-supervised brain topology inference according to claim 3, characterized in that, When the preset database information does not contain prior knowledge, the method for obtaining the fused feature matrix includes: An attention-embedded network is used to obtain the attention weights between each brain region and its neighboring brain regions in the feature matrix; Based on the attention weights, different importance is assigned to each brain region feature to obtain a fusion feature matrix.

5. The brain decoding method based on multi-view self-supervised brain topology inference according to claim 3, characterized in that, When the preset database information does not contain prior knowledge, the method for obtaining the fusion feature matrix includes: using a multilayer perceptron to obtain the linear correlation relationship of the brain regions represented in the feature matrix, and obtaining the fusion feature matrix based on the linear correlation relationship.

6. The brain decoding method based on multi-view self-supervised brain topology inference according to claim 3, characterized in that, When the preset database information contains prior knowledge, the method for obtaining the fused feature matrix includes: The feature matrix is ​​fused with prior knowledge from a pre-defined database using a Generative Neural Network (GNN) to obtain a fused feature matrix. The fused feature matrix is ​​optimized to obtain the fused feature matrix.

7. The brain decoding method based on multi-view self-supervised brain topology inference according to claim 1, characterized in that, The step of enhancing the brain data in the first view brain topology to obtain a first enhanced brain topology and a first augmented feature matrix includes: Based on the first adjacency matrix in the first view brain topology, a first random masking probability matrix is ​​generated based on the Bernoulli distribution; The first adjacency matrix is ​​randomly masked based on the first random masking probability matrix to obtain the first enhanced brain topology. A second random masking probability matrix is ​​generated based on the brain time-series features in the brain topology of the first view. The feature vectors of each region of interest are masked based on the second random masking probability matrix to obtain the first augmented feature matrix.

8. The brain decoding method based on multi-view self-supervised brain topology inference according to claim 7, characterized in that, The enhancement processing of brain data in the second-view brain topology to obtain a second enhanced brain topology and a second augmented feature matrix includes: Based on the second adjacency matrix in the second view brain topology, a third random masking probability matrix is ​​generated based on the Bernoulli distribution; the topological masking probability of the third random masking probability matrix is ​​the same as the topological masking probability of the first random masking probability matrix. The second adjacency matrix is ​​randomly masked based on the third random masking probability matrix to obtain the second enhanced brain topology. A fourth random masking probability matrix is ​​generated based on the brain time-series features in the second-view brain topology; the feature masking probability of the fourth random masking probability matrix is ​​different from the feature masking probability of the second random masking probability matrix. The feature vectors of each region of interest are masked based on the fourth random masking probability matrix to obtain the second augmented feature matrix.

9. The brain decoding method based on multi-view self-supervised brain topology inference according to claim 8, characterized in that, The comparison loss value is , ; ； ； in, For mutual information from the first view to the second view, For mutual information between the second view and the first view, This is the first augmented feature mapping feature. First, enhance brain topological mapping features. This is the second augmented feature mapping feature. For the second enhanced brain topological mapping feature, τ is a temperature parameter. sim (·) is a cosine similarity function, and z is the number of different brain regions.

Images