Brain decoding method based on multi-view self-supervised brain topology inference

Abstract

The invention discloses a brain decoding method based on multi-view self-supervised brain topology inference. The method comprises the following steps: constructing an initial topological structure model by a graph learning device based on preset brain region information; taking a brain topological structure in the initial topological structure model as a first view brain topological structure; based on a brain adjacency matrix in the initial topological structure model, obtaining a connection relation of different brain regions, and aggregating the original brain data according to the connection relation to obtain a second view brain topological structure; updating the first view brain topological structure through the second view brain topological structure; and when mutual information between the two view brain topological structures does not change any more, updating iteration is stopped, and the obtained target features in the second view brain topological structure are used as global brain features after the original brain data are decoded. According to the method, self-adaptive updating and learning can be carried out on the view brain topology according to the dynamic plasticity of the original brain data, inheritance and superposition of error information in the topology can be avoided, and accurate and comprehensive decoding information can be obtained.

Description

Technical Field

[0001] The present disclosure relates to the technical field of electroencephalogram decoding, and in particular to a brain decoding method based on multi-view self-supervised brain topology inference. Background Art

[0002] Brain decoding aims to convert brain activity or neural interactions into signals that can be understood and used by external devices to predict or understand specific brain cognitive states, external stimuli or thought content. It is crucial for the development of technologies such as artificial vision, brain-computer interfaces (BCI) and virtual reality devices, and will help scientists understand how the brain works more comprehensively.

[0003] The brain decoding methods based on multi-perspective self-supervised brain topology inference disclosed in the prior art usually predefine the brain topology structure based on prior knowledge, that is, predefine the brain topology structure based on a large amount of fully annotated brain data, expert knowledge, etc., and they assume that the predefined brain topology structure is sufficiently reliable; neuroscience research has shown that the brain's connection pattern changes dynamically, and brain decoding through predefined topology cannot reflect the actual dynamic changes, resulting in inaccurate and unreliable decoding information. Summary of the invention

[0004] In view of this, the embodiments of the present disclosure provide a brain decoding method based on multi-perspective self-supervised brain topology inference, which can solve the problems of inaccurate and unreliable decoding information in the prior art.

[0005] In a first aspect, the present disclosure provides a brain decoding method based on multi-view self-supervised brain topology inference, specifically comprising:

[0006] Construct an initial topological structure model based on preset brain area information and graph learner;

[0007] Using the brain topology in the initial topology model as the first-view brain topology;

[0008] Acquiring connection relationships between different brain regions according to the brain adjacency matrix in the initial topological structure model;

[0009] YY+242265P-YX

[0010] Aggregate the original brain data according to the connection relationship between different brain regions to obtain the second-view brain topology structure;

[0011] updating the first-view brain topology structure by using the second-view brain topology structure;

[0012] When the mutual information between the second view brain topology structure and the first view brain topology structure no longer changes, the update iteration stops, and the target features in the second view brain topology structure are obtained as the global brain features after decoding the original brain data.

[0013] Optionally, the constructing an initial topological structure model based on preset brain region information and a graph learner includes:

[0014] Constructing a fully parameterized matrix based on the preset number of brain regions, wherein each element in the fully parameterized matrix corresponds to a connection relationship between different brain regions, and different connections are independent;

[0015] Determine the nonlinear activation function;

[0016] Applying the nonlinear activation function to the fully parameterized matrix to obtain a positive correlation matrix;

[0017] An initial topological structure model is constructed based on the positive correlation matrix.

[0018] Optionally, the constructing an initial topological structure model based on preset brain region information and a graph learner includes:

[0019] According to the preset number of brain regions and the number of feature types, a feature matrix for each brain region is predefined;

[0020] The feature matrix of each brain region is fused with the preset database information to obtain a fused feature matrix of each brain region;

[0021] Based on the fusion feature matrix of each brain region, the similarity between different brain regions is calculated;

[0022] Based on the similarity and preset brain connection information, an initial topological structure model is constructed.

[0023] Optionally, when the preset database information does not contain prior knowledge, the method for obtaining the fusion feature matrix includes:

[0024] Using an attention embedding network to obtain the attention weights between each brain region and its adjacent brain regions in the feature matrix;

[0025] According to the attention weights, different importances are assigned to each brain region feature to obtain a fusion feature matrix.

[0026] Optionally, when the preset database information does not contain prior knowledge, the method for obtaining the fused feature matrix includes: using a multi-layer perceptron to obtain the linear correlation relationship represented by each brain region in the feature matrix, and obtaining the fused feature matrix based on the linear correlation relationship.

[0027] YY+242265P-YX

[0028] Optionally, when the preset database information includes prior knowledge, the method for obtaining the fusion feature matrix includes:

[0029] The feature matrix is ​​fused with the prior knowledge in the preset database information through GNN to obtain a fused feature matrix;

[0030] The fused feature matrix is ​​optimized to obtain the fused feature matrix.

[0031] Optionally, updating the first-view brain topology structure by using the second-view brain topology structure includes:

[0032] Performing enhancement processing on the brain data in the brain topology structure of the first view to obtain a first enhanced brain topology structure and a first augmented feature matrix;

[0033] Performing enhancement processing on the brain data in the second view brain topology structure to obtain a second enhanced brain topology structure and a second augmented feature matrix;

[0034] Inputting the first augmented feature matrix, the first augmented brain topology structure, the second augmented feature matrix, and the second augmented brain topology structure into a graph neural network encoder to obtain a first augmented feature graph representation, a first augmented brain topology graph representation, a second augmented feature graph representation, and a second augmented brain topology graph representation;

[0035] Mapping the first augmented feature map representation, the first enhanced brain topology map representation, the second augmented feature map representation, and the second enhanced brain topology map representation to obtain first augmented feature mapping features, first enhanced brain topology mapping features, second augmented feature mapping features, and second enhanced brain topology mapping features;

[0036] Obtaining mutual information from the first view to the second view according to the first augmented feature map features and the second augmented feature map features;

[0037] Obtaining mutual information from the second view to the first view according to the first enhanced brain topology mapping feature and the second enhanced brain topology mapping feature;

[0038] Obtaining a contrast loss value according to mutual information from the first view to the second view and mutual information from the second view to the first view;

[0039] When the contrast loss value no longer changes, the iterative update stops.

[0040] Optionally, the enhancing the brain data in the first view brain topology structure to obtain a first enhanced brain topology structure and a first augmented feature matrix includes:

[0041] According to the first adjacency matrix in the first view topology structure model, generating a first random masking probability matrix based on Bernoulli distribution;

[0042] Randomly masking the first adjacency matrix based on the first random masking probability matrix to obtain a first enhanced brain topology;

[0043] generating a second random masking probability matrix according to the brain time series features in the first view topological structure model;

[0044] The feature vector of each region of interest is masked based on the second random masking probability matrix to obtain a first augmented feature matrix.

[0045] Optionally, the enhancing the brain data in the second view brain topology structure to obtain a second enhanced brain topology structure and a second augmented feature matrix includes:

[0046] According to the second adjacency matrix in the second view topological structure model, a third random masking probability matrix is ​​generated based on Bernoulli distribution; the topological masking probability of the third random masking probability matrix is ​​the same as the topological masking probability of the first random masking probability matrix;

[0047] Randomly masking the second adjacency matrix based on the third random masking probability matrix to obtain a second enhanced brain topology;

[0048] generating a fourth random masking probability matrix according to the brain time series characteristics in the second view topological structure model; the characteristic masking probability of the fourth random masking probability matrix is ​​different from the characteristic masking probability of the second random masking probability matrix;

[0049] The feature vector of each region of interest is masked based on the fourth random masking probability matrix to obtain a second augmented feature matrix.

[0050] Optionally, the contrast loss value is L total ,

[0051] Among them, L I→L is the mutual information from the first view to the second view, L L→I is the mutual information from the second view to the first view, H ,i is the first augmented feature map feature, H I,j The first enhanced brain topology mapping feature, H L,i is the second augmented feature map feature, H L,j is the second enhanced brain topology mapping feature, τ is the temperature parameter, sim(·) is the cosine similarity function, and z is the number of different brain regions.

[0052] In a second aspect, the present disclosure also provides a brain decoding system based on multi-view self-supervised brain topology inference, including:

[0053] An initial topological structure model building module is used to build an initial topological structure model based on preset brain area information and a graph learner;

[0054] A first-view brain topology structure acquisition module, used to obtain a first-view brain topology structure based on the initial topology structure model;

[0055] A connection relationship acquisition module, used to acquire the connection relationship between different brain regions based on the brain adjacency matrix in the initial topological structure model;

[0056] A second-view brain topology structure acquisition module, used to aggregate original brain data according to the connection relationship between different brain regions to obtain a second-view brain topology structure;

[0057] An updating module, configured to update the first-view brain topology structure by using the second-view brain topology structure;

[0058] The analysis module is used to stop updating iteration when the mutual information between the brain topology structure of the second view and the brain topology structure of the first view no longer changes, and use the target features in the brain topology structure of the second view as the global brain features after decoding the original brain data.

[0059] The brain decoding method based on multi-view self-supervised brain topology inference disclosed in the present application can quickly construct the initial topological structure of the brain region through the combination of preset brain region information and graph learner, providing a basis for subsequent analysis and decoding. The graph learner can handle complex and irregular brain region connection relationships and is suitable for the nonlinear structure of the brain; through the first view brain topological structure based on the initial topological structure model and the second view brain topological structure based on the brain region connection relationship, this multi-view strategy can provide more comprehensive and richer information to decode brain activity. Through iterative updates between the two views, the real brain connection state can be gradually approached, thereby further improving the decoding accuracy; the second view brain topological structure can be gradually optimized to be closer to the real brain region connection mode, and the update process allows dynamic adjustment of the brain region connection relationship to adapt to changes in different tasks and data; through the update process, the brain topology in the two views can be YY+242265P-YX based on the dynamic plasticity of the original brain data.

[0060] Adaptive updating and learning can avoid the inheritance and superposition of erroneous information in the topology; when the mutual information no longer changes, it is considered that a stable state has been reached, and the updating and iteration are stopped. The target features in the brain topology structure of the second view are used as the global brain features after decoding the original brain data to obtain accurate and comprehensive decoding information.

[0061] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the following preferred embodiments are specifically cited and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0063] Figure 1 A flowchart of a brain decoding method based on multi-view self-supervised brain topology inference provided in an embodiment of the present disclosure.

[0064] Figure 2 A schematic flow chart of a first embodiment of a method for constructing an initial topological structure model provided in an embodiment of the present disclosure.

[0065] Figure 3 A schematic flow chart of a second embodiment of the method for constructing an initial topological structure model provided in an embodiment of the present disclosure.

[0066] Figure 4 A flowchart of a first embodiment of a method for obtaining a fusion feature matrix provided in an embodiment of the present disclosure.

[0067] Figure 5 A flowchart of a second embodiment of the method for obtaining a fusion feature matrix provided in an embodiment of the present disclosure.

[0068] Figure 6 A flowchart of a method for updating a first-view brain topology structure through a second-view brain topology provided in an embodiment of the present disclosure.

[0069] Figure 7 for Figure 6 A flowchart of a method for enhancing brain data in a first view brain topology structure to obtain a first enhanced brain topology structure and a first augmented feature matrix.

[0070] YY+242265P-YX

[0071] Figure 8 for Figure 6 A flowchart of a method for enhancing the brain data in the second view brain topology structure to obtain a second enhanced brain topology structure and a second augmented feature matrix.

[0072] Fig. 9 A schematic diagram of a brain decoding framework based on multi-view self-supervised brain topology reasoning provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0073] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0074] It should be clear that the following embodiments of the present disclosure are described by specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other in the absence of conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present disclosure.

[0075] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.

[0076] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The drawings only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0077] YY+242265P-YX

[0078] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects described may be practiced without these specific details.

[0079] Reference Figure 1The present application discloses a brain decoding method based on multi-view self-supervised brain topology inference, which specifically includes the following contents:

[0080] S100, constructing an initial topological structure model based on preset brain area information and a graph learner.

[0081] S200, using the brain topology structure in the initial topology structure model as the first view brain topology structure.

[0082] The first view can be marked as the “guide” view. The first view brain topology structure provides a preliminary understanding of the connection relationship between brain regions, which facilitates subsequent improvement and optimization.

[0083] S300, obtaining the connection relationship between different brain regions according to the brain adjacency matrix in the initial topological structure model.

[0084] S400, aggregating original brain data according to the connection relationship between different brain regions to obtain a second view brain topology structure.

[0085] The original brain data may be fMRI data or EEG data.

[0086] The second view can be marked as a “learner” view. The “learner” view directly uses the brain adjacency matrix and brain time series learned by BTI as the feature matrix, and the brain adjacency matrix is ​​gradually updated with the model parameters.

[0087] In this embodiment, the brain time series of the brain adjacency matrix is ​​used as the feature matrix of the brain topology structure of the second view (“learner” view).

[0088] By aggregating raw brain data, the data representation capability can be enhanced, which helps to better capture the global characteristics of brain regions. The aggregation process combines data from different brain regions, which helps to reveal the collaborative working mode of brain regions.

[0089] S500, updating the first-view brain topology structure by using the second-view brain topology structure;

[0090] When the mutual information between the second view brain topology structure and the first view brain topology structure no longer changes, the update iteration stops, and the target features in the obtained second view brain topology structure are used as the global brain features after decoding the original brain data.

[0091] The brain decoding method based on multi-view self-supervised brain topology inference disclosed in this application is predicted by YY+242265P-YX

[0092] The combination of the assumed brain area information and the graph learner can quickly construct the initial topological structure of the brain area, providing a basis for subsequent analysis and decoding. The graph learner can handle complex and irregular brain area connection relationships and is suitable for the nonlinear structure of the brain. Through the first view brain topology structure based on the initial topological structure model and the second view brain topology structure based on the brain area connection relationship, this multi-view strategy can provide more comprehensive and richer information to decode brain activity. Through the iterative update between the two views, the real brain connection state can be gradually approached, thereby further improving the decoding accuracy. The second view brain topology structure can be gradually optimized to be closer to the real brain area connection mode. The update process allows the brain area connection relationship to be dynamically adjusted to adapt to the changes of different tasks and data. Through the update process, the brain topology in the two views can be adaptively updated and learned according to the dynamic plasticity of the original brain data, which can avoid the inheritance and superposition of erroneous information in the topology. When the mutual information no longer changes, it is considered that a stable state has been reached, and the update iteration is stopped. The target features in the second view brain topology structure are used as the global brain features after the original brain data is decoded to obtain accurate and comprehensive decoding information.

[0093] Through multi-step iterative optimization, this scheme can efficiently decode raw brain data and extract representative global brain features. Through graph learners and brain adjacency matrices, it can accurately capture the connection relationship between brain regions and improve decoding accuracy. It has dynamic adjustment capabilities and can adapt to different brain data and task requirements. It has strong robustness. Through the aggregation and update process, it can understand the function and structure of the brain from a global perspective and reveal the collaborative working mode of brain regions. The generated topological structure and features have good visualization and interpretability, which is convenient for researchers to understand and analyze. Through the above steps and benefits, this brain decoding method based on multi-perspective self-supervised brain topology inference has broad application prospects in the fields of brain science research, medical diagnosis, and brain-computer interface.

[0094] Reference Figure 2 As for the method for constructing the initial topology structure model, in the first embodiment, parameter-driven BTI (brain topology inference) is adopted, which specifically includes:

[0095] S111, constructing a fully parameterized matrix based on the preset number of brain regions, wherein each element in the fully parameterized matrix corresponds to a connection relationship between different brain regions, and different connections are independent.

[0096] Among them, each element can be a positive number (indicating positive correlation) or a negative number (indicating negative correlation), that is, the full parameterization matrix includes positive correlation and negative correlation.

[0097] Assume that the fully parameterized matrix is ​​W, W∈R N×N , each element w in the fully parameterized matrixij YY+242265P-YX

[0098] Represents the connection strength between node i (i.e., brain region i) and node j (i.e., brain region j); for brain data, the interactions between brain regions of interest (ROIs) can be regarded as the topological structure of the graph, and the information changes in ROIs can be abstracted as node features.

[0099] Specifically, a fully parameterized matrix is ​​initialized to simulate the interconnections in the brain. No additional task-related model input is required, and an independent parameter of size N×N is directly initialized, where each element is used to simulate each interconnection in the brain topology, that is, the connection relationship (or connection weight) between different brain regions, and the preset number of brain regions is N+1.

[0100] Through this step, all possible connections between each brain region and other brain regions can be fully considered to ensure the comprehensiveness and completeness of the model.

[0101] S112, determining a nonlinear activation function.

[0102] Specifically, in order to ensure the stability of model training, a nonlinear activation function σ(.) is introduced. This activation function can be a common nonlinear function such as ReLU, Sigmoid, Tanh, etc.

[0103] Furthermore, the activation function can be selected according to specific needs, such as ReLU for enhancing positive correlation and Sigmoid for balancing positive and negative correlations; nonlinear activation functions can better capture the complex nonlinear relationships between brain regions and enhance the expressive power of the model.

[0104] S113, applying a nonlinear activation function to the fully parameterized matrix to obtain a positive correlation matrix.

[0105] Among them, the positive correlation matrix is ​​A, A p-BTI =σ(W).

[0106] Specifically, by applying a nonlinear activation function to the fully parameterized matrix, the activated matrix is ​​obtained, and the activated matrix can be directly used as a positive correlation matrix, which represents the connection relationship between brain regions after nonlinear transformation. Because the activation function suppresses negative values ​​to zero (such as ReLU) or maps them to a smaller value (such as Sigmoid and Tanh), the positive values ​​in A can be regarded as positively correlated connections, that is, by processing the fully parameterized matrix through a nonlinear activation function, the negatively correlated part can be filtered out and the positively correlated part can be retained.

[0107] A positive correlation means that if two nodes are strongly connected in the brain, this strong connection should also be reflected in the positive correlation matrix.

[0108] Furthermore, the activated matrix may be normalized and the normalized matrix may be used as a positive correlation matrix.

[0109] S114, constructing an initial topological structure model based on the positive correlation matrix.

[0110] Specifically, the connection weights in the positive correlation matrix are converted into a network representation of the connections between brain regions to generate an initial brain region functional network model (i.e., an initial topological structure model).

[0111] Furthermore, some optimization algorithms (such as gradient descent, stochastic gradient descent, etc.) can be used to train this topological structure model so that it can better simulate the state and behavior of the brain.

[0112] The method disclosed in this embodiment uses parameter-driven BTI to construct an initial topological structure model. Through fully parameterized matrices and nonlinear activation functions, the complex relationships between brain regions are fully considered and analyzed to improve the accuracy and reliability of the model. The construction of a positive correlation matrix simplifies the complexity of the model, making subsequent analysis and optimization more efficient. The construction of a topological structure model makes the functional network relationships between brain regions intuitive and visible, making it easier for researchers to understand and interpret brain functional networks. The parameterized matrix and nonlinear activation function in the entire scheme make the model highly flexible and adaptable, and can cope with different data characteristics and research needs. Through these steps, the overall scheme can provide a comprehensive, accurate and flexible brain functional network analysis tool, which helps to gain in-depth insights and results in neuroscience and brain function research.

[0113] Reference Figure 3 As for the method for constructing the initial topology structure model, in the second embodiment, similarity-driven BTI (brain topology inference) is adopted, which specifically includes:

[0114] S211, predefine a feature matrix for each brain region according to the preset number of brain regions and the number of feature types.

[0115] Specifically, according to the preset number of brain regions N and the number of feature types d X , predefine (i.e. construct) the feature matrix of each brain region, the size of the feature matrix

[0116] S212, fusing the feature matrix of each brain region with the preset database information to obtain a fused feature matrix of each brain region.

[0117] By integrating the preset database information, more historical data can be used to enrich the feature matrix and improve the generalization ability and prediction accuracy of the model.

[0118] S213, calculating the similarity between different brain regions based on the fusion feature matrix of each brain region.

[0119] Specifically, based on the fused feature matrix, cosine similarity or other distance metrics can be used to calculate the similarity between different brain regions.

[0120] Similarity calculation can quantify the feature similarities between different brain regions and provide a reference for constructing topological structures; similarity indicators can intuitively reflect the feature relationships between brain regions, making it easier for researchers to understand and interpret model results.

[0121] S214, constructing an initial topological structure model based on the similarity and preset brain connection information.

[0122] Specifically, the initial topological structure model is constructed by combining the calculated similarity matrix S and the preset brain connection information (such as anatomical connection or functional connection data). The relationship between brain regions can be represented by a graph structure, where the nodes represent brain regions and the weights of the edges are determined by the similarity and the preset connection information. For example, the weight of the edge W ij It can be defined as:

[0123] W ij =βS ij +(1-β)C ij ; where C ij is the preset connection strength, and β is the weight parameter.

[0124] For S212, when the preset database information does not contain prior knowledge, the following two schemes can be used for processing to obtain a fused feature matrix; among them, Scheme 1 is BTI based on attention mechanism, Scheme 2 is BTI based on MLP, and Scheme 3 is BTI based on ROI specific pattern learning (RSPL-BTI). Brain topology learning is performed on completely unlabeled brain data, and the decoding process is based on the learned brain topology. The brain encoders in these two steps share parameters and have a completely consistent structure.

[0125] Reference Figure 4 , the method of obtaining the fusion feature matrix using scheme 1 specifically includes:

[0126] A100 uses an attention embedding network to obtain the attention weights between each brain region and its adjacent brain regions in the feature matrix.

[0127] Here, each brain region refers to each node, and adjacent brain regions refer to adjacent nodes.

[0128] Specifically, e i,j =LeakyReLU(a T [Whi ||Wh j ]), where a is the attention vector, W is the weight matrix, and h i and h j is the eigenvector of brain region i and brain region j, e i,j is the attention score (i.e., attention weight) between brain region i and brain region j.

[0129] A200, according to the attention weight, assigns different importance to each brain region feature and obtains the fused feature matrix.

[0130] Among them, the calculation of attention weights usually involves weighted summation of each eigenvector in the feature matrix E1. The weights are dynamically generated by the attention mechanism, and these weights reflect the contribution of different brain region features to the topological structure.

[0131] Furthermore, the fusion feature matrix may include:

[0132] σ is a nonlinear activation function, N i is the set of neighbor nodes of brain region i.

[0133] Attention-based BTI uses an embedding network similar to the graph attention network to refine the representation of ROIs. Each layer of the embedding network calculates the Hadamard product of the input ROI embedding E and the parameter vector: in, represents the output of the lth layer of the attention-based embedding network, e l-1 express The transpose of the i-th row vector in , ⊙ represents the Hadamard product, t represents the transpose, and σ(.) also represents a nonlinear operation. It is worth noting that the input of the first layer Initialized as feature matrix X, the output of the last layer is used as the final ROI embedding (i.e., corresponding features in the fused feature matrix). The attention-based embedding network assumes that each ROI feature contributes differently to the brain topology and there is no significant correlation between features.

[0134] When the brain performs different functions, the activation degree and cooperation mode of different brain regions often differ. In order to better describe these differential representation information, different importance is adaptively assigned to each brain region feature according to the predefined feature matrix of each brain region, and the complex interactive connections between brain regions are automatically captured with the model to obtain a fusion feature matrix; this method assumes that the features between brain regions are unrelated, and takes into account the contribution of different brain region features to the brain topology.

[0135] The method of obtaining the fused feature matrix using scheme 2 includes: using a multi-layer perceptron to obtain the linear correlation relationship represented by each brain region in the feature matrix, and obtaining the fused feature matrix based on the linear correlation relationship.

[0136] Specifically, MLP-based BTI uses MLP as an embedding network to mine ROIs representations, and each layer of MLP captures the correlation between ROIs features by linearly combining features and processing them with nonlinear activation functions. Compared with attention-based BTI, MLP-based BTI further considers the linear combination of correlations between different ROIs features.

[0137] For the MLP-based embedding network, the input is the feature matrix and the output is the fused feature matrix. Each layer of the MLP-based embedding network can be expressed as:

[0138] in, is the output of the l-1th layer in the MLP-based embedding network (i.e., the input of the lth layer), is the weight matrix of the l-1th layer.

[0139] Specifically, the weight matrix The features of different ROIs are linearly combined, and the representation ability of the features is enhanced through nonlinear activation functions.

[0140] Scheme 3 is BTI based on ROI-specific pattern learning (RSPL-BTI). Specifically, the specific pattern of each ROI is introduced into the iteration and update of GNN, so as to learn the brain topology while adaptively updating the ROI-specific pattern.

[0141] In GNN, the input is a feature matrix and the output is a fused feature matrix, that is, a feature matrix including ROI-specific patterns. The process includes: by introducing the specific pattern of each ROI, the ROI-specific pattern is embedded into the iteration and update process of GNN.

[0142] H GNN =(I+softmax(σ(E·E T )))XEW+Eb, where H GNN Represents the learned brain spatial representation, I is the identity matrix, W and b are the parameters of GNN, the softmax function is used for topological normalization, and σ(.) represents a nonlinear operation.

[0143] The original GNN shares weights and biases between all ROIs, allowing the model to learn common patterns between ROIs (Region of Interest). However, previous studies have shown that when different ROIs work together to complete brain activity, they have their own unique patterns, and only considering common patterns between ROIs may affect decoding performance.

[0144] Reference Figure 5 , for S212, when the preset database information contains prior knowledge, the BTI based on GNN is adopted, and the specific method for obtaining the fusion feature matrix includes:

[0145] B100, obtains prior knowledge from preset database information.

[0146] B200, fuses the feature matrix with prior knowledge through GNN to obtain a fused feature matrix.

[0147] GNN-based BTI integrates the feature matrix (i.e., ROIs features) and prior knowledge in the preset database information into the embedding E through GNN.

[0148] The fusion feature matrix is Among them, A p To predefine the feature matrix of each brain region, D p A p The degree matrix, θ l-1 is the l-1th layer parameter of GNN, is the weight matrix of the l-1th layer, I is the unit matrix, and σ() is a nonlinear operation.

[0149] GNN-based BTI assumes that the topological connectivity between ROIs is not only related to ROI features but also to the predefined brain topology.

[0150] In this embodiment, the "mentor" view provides correct and effective guidance for the "learner" view. For BTO (braintopology optimization), that is, the situation where the brain topology can be predefined when prior knowledge is available, the predefined brain topology can be obtained through prior knowledge, and the "mentor" view is initialized to the predefined brain topology; for BTE (brain topology extrapolation), that is, the situation where prior knowledge is not available, the "mentor" view is initialized to the unit matrix.

[0151] Furthermore, the present application also includes: optimizing the fused feature matrix to obtain the fused feature matrix.

[0152] Regarding obtaining the brain adjacency matrix in the initial topological structure model, considering that the interactions between ROIs are usually non-directional and not all ROIs are highly activated when the brain is working, the application also includes: symmetric and sparse initialization of the brain topological structure, and the symmetric sparse matrix can greatly improve the computational efficiency and scalability of GNN.

[0153] Among them, top-TSR(.) represents the top-TSR% brain connections with the strongest associations (topology sparsity ratio, TSR).

[0154] A is further symmetric and normalized as follows:

[0155]

[0156] Among them, A sym and A norm Represent the symmetric and normalized brain topology respectively, and the final A norm as the optimized brain adjacency matrix.

[0157] Reference Figure 6 , a method for updating a first view brain topology structure by using a second view brain topology, comprising:

[0158] C100, enhancing the brain data in the brain topology structure of the first view to obtain a first enhanced brain topology structure and a first augmented feature matrix.

[0159] C200, enhancing the brain data in the second view brain topology structure to obtain a second enhanced brain topology structure and a second augmented feature matrix.

[0160] In this embodiment, the enhancement processing performed on the brain data includes brain topology masking and brain feature masking processing.

[0161] After data augmentation, the enhanced topology of the first view (i.e., the “instructor” view) can be expressed as: G I =(X1 augumentation ,A1 augumentation ); The enhanced topology of the second view (i.e., the “learner” view) can be expressed as: G L =(X2 augumentation ,A2 augumentation ), where A1 augumentation Stands for the first enhanced brain topology, X1 augumentation is the first augmented feature matrix, A2 augumentation Stands for the second enhanced brain topology, X2 augumentation is the second augmented characteristic moment.

[0162] C300, input the first augmented feature matrix, the first augmented brain topology structure, the second augmented feature matrix, and the second augmented brain topology structure into the graph neural network encoder to obtain the first augmented feature map representation, the first augmented brain topology map representation, the second augmented feature map representation, and the second augmented brain topology map representation.

[0163] Specifically, GNNs are used as brain encoders to extract spatial representations of different ROIs from the “instructor” and “learner” views.

[0164] The spatial representation extracted from the first view (i.e., the “instructor” view) includes:

[0165] The spatial representation extracted from the second view (i.e., the “learner” view) includes:

[0166] and d GNN is the feature dimension. Through the superposition of l GNN layers, the fine brain spatial information is gradually mined.

[0167] Furthermore, by inputting the first augmented feature matrix, the first augmented brain topology, the second augmented feature matrix, and the second augmented brain topology into the graph neural network encoder, the first augmented feature graph obtained is expressed as The first enhanced brain topology map is represented as The second augmented feature map is expressed as The second enhanced brain topology map is represented as

[0168] C400, mapping the first augmented feature map representation, the first enhanced brain topology map representation, the second augmented feature map representation, and the second enhanced brain topology map representation to obtain first augmented feature mapping features, first enhanced brain topology mapping features, second augmented feature mapping features, and second enhanced brain topology mapping features.

[0169] Specifically, after the GNN encoder, the features of the two views are integrated through a fully connected layer (multilayer perceptron layer, MLP) and Mapping is performed, that is, the first augmented feature map is represented The first enhanced brain topography The second augmented feature map Second enhanced brain topography Mapping is performed to facilitate multi-view loss calculation.

[0170] The first augmented feature map features:

[0171] First Enhanced Brain Topology Mapping Features:

[0172] The second augmented feature map features:

[0173] Second Enhanced Brain Topology Mapping Features:

[0174] C500: Obtain mutual information from the first view to the second view according to the first augmented feature map features and the second augmented feature map features.

[0175] The mutual information from the first view to the second view is L I→L :

[0176] Among them, H I,i is the first augmented feature map feature, H I,j The first enhanced brain topology mapping feature, H L,i is the second augmented feature map feature, H L,j is the second enhanced brain topology mapping feature, τ is the temperature parameter, sim(·) is the cosine similarity function, and z is the number of different brain regions.

[0177] C600: Obtain mutual information from the second view to the first view according to the first enhanced brain topology mapping feature and the second enhanced brain topology mapping feature.

[0178] The mutual information from the second view to the first view is L L→I :

[0179] C700, obtaining a contrast loss value according to the mutual information from the first view to the second view and the mutual information from the second view to the first view;

[0180] When the contrast loss value no longer changes, the iterative update stops.

[0181] In this embodiment, the contrast loss value is L total ,

[0182] Reference Figure 7 , performing enhancement processing on the brain data in the brain topology structure of the first view to obtain a first enhanced brain topology structure and a first augmented feature matrix, including;

[0183] C110 , generating a first random masking probability matrix based on a Bernoulli distribution according to a first adjacency matrix in the first view topology structure model.

[0184] Specifically, brain connections can be randomly masked by discarding them in proportion (topological masking ratio, rT). Further, for the first adjacency matrix in the first view topological structure model, based on the probability p(a1) I The first random masking probability matrix m is generated by Bernoulli distribution a1 , m a1 ∈R N×N , a1 is the adjacency matrix.

[0185] C120, randomly masking the first adjacency matrix based on the first random masking probability matrix to obtain a first enhanced brain topology.

[0186] Specifically, based on m a1 Randomly mask a1: Among them, A1 augumentation represents the first enhanced brain topology, is the topology enhancement strategy function.

[0187] C130, generating a second random masking probability matrix according to the brain time series characteristics in the first view topological structure model.

[0188] Specifically, a portion of features with rF (feature masking rate) is randomly selected and zero-masked. Specifically, for the brain time series feature (X1) in the first view topological structure model, firstly based on the probability p(X1) I The Bernoulli distribution generates a random masking probability matrix m X1 ∈R d .

[0189] C140 , masking the feature vector of each region of interest based on the second random masking probability matrix to obtain a first augmented feature matrix.

[0190] Specifically, based on m X1 Mask the feature vector of each ROI:

[0191] X1 augumentation =F fm (X) = [X1⊙m X ,...,X N ⊙m X ];

[0192] Among them, X1 augumentation is the first augmented feature matrix, F fm is the feature enhancement strategy function, X1-X N is the feature vector of each ROI (i.e., the feature vector of each ROI in the brain time series features in the first view topological structure model).

[0193] Reference Figure 8 , enhance the brain data in the second view brain topology structure to obtain a second enhanced brain topology structure and a second augmented feature matrix, including:

[0194] C210, generating a third random masking probability matrix based on Bernoulli distribution according to the second adjacency matrix in the second view topological structure model; the topological masking probability of the third random masking probability matrix is ​​the same as the topological masking probability of the first random masking probability matrix.

[0195] Specifically, brain connections can be randomly masked by discarding them in proportion (topological masking ratio, rT). Further, for the second adjacency matrix in the second view topological structure model, based on the probability p(a2) L The second random masking probability matrix m is generated by Bernoulli distribution a2 , m a2 ∈R N×N , a2 is the second adjacency matrix.

[0196] C220, randomly masking the second adjacency matrix based on the third random masking probability matrix to obtain a second enhanced brain topology.

[0197] Specifically, based on m a2 Randomly mask a2: Among them, A2 augumentation represents the second enhanced brain topology, is the topology enhancement strategy function.

[0198] C230, generating a fourth random masking probability matrix according to the brain time series characteristics in the second view topological structure model; the characteristic masking probability of the fourth random masking probability matrix is ​​different from the characteristic masking probability of the second random masking probability matrix.

[0199] YY+242265P-YX

[0200] Specifically, a portion of features with rF (feature masking rate) is randomly selected and zero-masked. Specifically, for the brain time series features (X2) in the second view topological structure model, firstly based on the probability p(X2) L The Bernoulli distribution generates a random masking probability matrix m X2 ∈R d .

[0201] C240 , masking the feature vector of each region of interest based on a fourth random masking probability matrix to obtain a second augmented feature matrix.

[0202] Specifically, based on m X2 Mask the feature vector of each ROI:

[0203] X2 augumentation =F fm (X) = [X1⊙m X ,...,X N ⊙m X ];

[0204] Among them, X2 augumentation is the second augmented feature matrix, F fm is the feature enhancement strategy function, X1-X N is the feature vector of each ROI (i.e., the feature vector of each ROI in the brain time series features in the second view topological structure model).

[0205] Since the two views use the same brain features, different feature masking probabilities are used to ensure feature variability. For brain topology, the brain area connections under the "instructor" view and the "learner" view are essentially very different, so the present invention uses the same topology masking probability.

[0206] The present invention evaluates the performance of the proposed framework on the internationally disclosed Human Connectome Project dataset (HCP), which mainly consists of two parts, task-related data (Task-HCP) and resting state data (Rest-HCP), and tests the proposed model on both parts of data. In addition, considering the calculation speed of multiple ROIs and the high difficulty of information fitting in unsupervised scenarios, the whole-brain voxels are divided into 22 ROIs (N=22) for subsequent optimization of self-supervised learning, which ultimately benefits from brain decoding. The present invention uses four indicators: classification accuracy, area under the curve (AUC), precision, and recall rate to verify the effectiveness of the method.

[0207] To evaluate the decoding performance of the proposed method, it was compared with several SOTA supervised and unsupervised brain analysis methods with multiple metrics. The supervised methods include 1) a 2-layer MLP method with 256 hidden nodes per layer; 2) a series of GNN-based brain spatial information mining methods, including graph convolutional networks (GCN), graph attention networks (GAT) (6 convolutional layers, each containing 32 neurons), GroupINN, BrainGNN, and MVS-GCN; 3) supervised GTL methods, including TAGNN and DAST-GNN.

[0208] Five state-of-the-art unsupervised fMRI analysis methods: 1) BrainUSL; 2) CFCGL; 3) BrainGSL; 4) GATE; 5) PTGB. All compared methods report means and variances, and the quantitative results are shown in Tables 1 and 2.

[0209] Table 1. Results of decoding performance of the proposed method and SOTA method on Rest-HCP (average value (method))

[0210]

[0211] As can be seen from Table 1, the decoding effect on the Rest-HCP dataset, the MLP-based method is significantly better than a variety of supervised GNN methods based on spatial brain features. Compared with most supervised GNN brain decoding methods based on predefined brain topology and multi-view self-supervised brain topology inference, TAGCN and DAST-GCN take the dynamic characteristics of brain topology into consideration, and automatically update and optimize the interaction between brain regions through parameterized methods. The results show that the dynamic topology-based method is significantly better than the GNN method with predefined topology. Further comparing the proposed method with several state-of-the-art graph self-supervised brain decoding methods based on multi-view self-supervised brain topology inference, it can be seen that the decoding effect of the existing unsupervised methods can achieve performance comparable to that of the supervised GNN method, but it is still very limited, and the decoding accuracy needs to be improved. The proposed method has achieved good results in multiple classification indicators, exceeding the existing unsupervised GNN method based on spatial brain features by about 9%.

[0212] Table 2 Results of decoding performance of the proposed method and SOTA method on Task-HCP (average (method))

[0213]

[0214] In Table 2, \ indicates that the model does not converge. It can be seen from Table 2 that the decoding effect on the Task-HCP dataset is similar to the results on the Rest-HCP dataset. The MLP-based method is significantly better than the spatial brain feature-based method, and the dynamic topology-based GNN decoding method has obvious advantages over the predefined topology method. This is highly consistent with the previous research findings that brain topology is dynamically updated and changes over time, and simulating the dynamic change process of brain topology is of great significance for brain decoding. In addition, for the GNN-based self-supervised brain decoding method based on multi-view self-supervised brain topology inference, the BrainUSL model does not converge on Task-HCP, while the decoding performance of the other methods for different tasks still needs to be improved. BTI-MVCSL has obvious advantages over these methods, and has a performance improvement of about 20% compared to previous unsupervised decoding methods, which shows the effectiveness of the proposed framework.

[0215] Also refer to Fig. 9This application also proposes a new brain decoding framework based on multi-view self-supervised brain topology reasoning, including a brain topology inference (BTI) module, a multi-view contrast self-supervised learning (MVCSL) module and a decoding module, wherein the brain topology inference (BTI) module includes a series of graph learners for constructing an appropriate "learner" view topology; multi-view contrast self-supervised learning (MVCSL) optimizes the brain topology structure through consistency constraints between the "instructor" perspective and the "learner" perspective, and achieves persistent optimization through a self-updating learning mechanism SLM; brain decoding identifies the state of the brain under different downstream tasks.

[0216] YY+242265P-YX

[0217] The scheme disclosed in this embodiment can dynamically embed brain topology without data labels, and mainly includes two new modules to improve the existing graph self-supervision methods for brain data analysis: 1) Brain topology inference BTI updates the interactive connection relationship between ROIs. The present invention attempts various popular graph learners and further compares their decoding effects; 2) Multi-view constrained self-supervised learning MVCSL includes "instructor" view and "learner" view, aiming to promote model optimization through multi-view comparison. In addition, in order to avoid the inheritance and superposition of erroneous information in the predefined topology, a new self-learning mechanism-SLM is proposed, which enables the optimal brain decoding topology to be learned. A large number of experiments on two data sets have proved the effectiveness of the framework. The present invention plays an important role in deeply understanding the structure and function of the brain, and understanding the separation and integration of brain information. By imitating the interactive information processing mechanism of the brain network, more efficient and intelligent algorithms can be developed to further promote the development of the field of general artificial intelligence.

[0218] In a second aspect, the present disclosure also provides a brain decoding system based on multi-view self-supervised brain topology inference, including:

[0219] An initial topological structure model building module is used to build an initial topological structure model based on preset brain area information and a graph learner;

[0220] A first-view brain topology structure acquisition module, used to obtain a first-view brain topology structure based on an initial topology structure model;

[0221] A connection relationship acquisition module, used to obtain the connection relationship between different brain regions based on the brain adjacency matrix in the initial topological structure model;

[0222] A second-view brain topology structure acquisition module, used to aggregate original brain data according to the connection relationship between different brain regions to obtain a second-view brain topology structure;

[0223] An updating module, configured to update the first-view brain topology structure by using the second-view brain topology structure;

[0224] The analysis module is used to stop updating iteration when the mutual information between the second view brain topology structure and the first view brain topology structure no longer changes, and use the target features in the obtained second view brain topology structure as the global brain features after decoding the original brain data.

[0225] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be pointed out that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details of the above-mentioned disclosure YY+242265P-YX are only for the purpose of illustration and ease of understanding, rather than limitation. The above details do not limit the present disclosure to the necessity of adopting the above-mentioned specific details to be implemented.

[0226] In the present disclosure, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. The block diagrams of the devices, devices, equipment, and systems involved in the present disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.

[0227] Additionally, as used herein, "or" used in a list of items beginning with "at least one" indicates a separate list, so that, for example, a list of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not mean that the example described is preferred or better than other examples.

[0228] It should also be noted that in the system and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0229] Various changes, substitutions, and modifications of the techniques described herein may be made without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of the present disclosure is not limited to the specific aspects of the processes, machines, manufactures, compositions of events, means, methods, and actions described above. Currently existing or later to be developed processes, machines, manufactures, compositions of events, means, methods, or actions that perform substantially the same functions or achieve substantially the same results as the corresponding aspects described herein may be utilized. Thus, the appended claims include such processes, machines, manufactures, compositions of events, means, methods, or actions within their scope.

[0230] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the present disclosure.

[0231] Thus, the present disclosure is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0232] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A brain decoding method based on multi-view self-supervised brain topology inference, characterized in that: include: Construct an initial topological structure model based on preset brain area information and graph learner; Using the brain topology in the initial topology model as the first-view brain topology; Acquiring connection relationships between different brain regions according to the brain adjacency matrix in the initial topological structure model; Aggregate the original brain data according to the connection relationship between different brain regions to obtain the second-view brain topology structure; updating the first-view brain topology structure by using the second-view brain topology structure; When the mutual information between the second view brain topology structure and the first view brain topology structure no longer changes, the update iteration stops, and the target features in the obtained second view brain topology structure are used as the global brain features after decoding the original brain data.

2. The brain decoding method based on multi-view self-supervised brain topology inference according to claim 1, characterized in that: The initial topological structure model is constructed based on the preset brain area information and the graph learner, including: Constructing a fully parameterized matrix based on the preset number of brain regions, wherein each element in the fully parameterized matrix corresponds to a connection relationship between different brain regions, and different connections are independent; Determine the nonlinear activation function; Applying the nonlinear activation function to the fully parameterized matrix to obtain a positive correlation matrix; An initial topological structure model is constructed based on the positive correlation matrix.

3. The brain decoding method based on multi-view self-supervised brain topology inference according to claim 1, characterized in that: The initial topological structure model is constructed based on the preset brain area information and the graph learner, including: According to the preset number of brain regions and the number of feature types, a feature matrix for each brain region is predefined; The feature matrix of each brain region is fused with the preset database information to obtain a fused feature matrix of each brain region; Based on the fusion feature matrix of each brain region, the similarity between different brain regions is calculated; Based on the similarity and preset brain connection information, an initial topological structure model is constructed.

4. The brain decoding method based on multi-view self-supervised brain topology inference according to claim 3, characterized in that: When the preset database information does not contain prior knowledge, the method for obtaining the fusion feature matrix includes: Using an attention embedding network to obtain the attention weights between each brain region and its adjacent brain regions in the feature matrix; According to the attention weights, different importances are assigned to each brain region feature to obtain a fusion feature matrix.

5. The brain decoding method based on multi-view self-supervised brain topology inference according to claim 3, characterized in that: When the preset database information does not contain prior knowledge, the method for obtaining the fused feature matrix includes: using a multi-layer perceptron to obtain the linear correlation relationship represented by each brain region in the feature matrix, and obtaining the fused feature matrix according to the linear correlation relationship.

6. The brain decoding method based on multi-view self-supervised brain topology inference according to claim 3, characterized in that: When the preset database information contains prior knowledge, the The method for obtaining the combined feature matrix includes: The feature matrix is ​​fused with the prior knowledge in the preset database information through GNN to obtain a fused feature matrix; The fused feature matrix is ​​optimized to obtain the fused feature matrix.

7. The brain decoding method based on multi-view self-supervised brain topology inference according to claim 1, characterized in that: The updating of the first-view brain topology structure by using the second-view brain topology structure comprises: Performing enhancement processing on the brain data in the brain topology structure of the first view to obtain a first enhanced brain topology structure and a first augmented feature matrix; Performing enhancement processing on the brain data in the second view brain topology structure to obtain a second enhanced brain topology structure and a second augmented feature matrix; Inputting the first augmented feature matrix, the first augmented brain topology structure, the second augmented feature matrix, and the second augmented brain topology structure into a graph neural network encoder to obtain a first augmented feature graph representation, a first augmented brain topology graph representation, a second augmented feature graph representation, and a second augmented brain topology graph representation; Mapping the first augmented feature map representation, the first enhanced brain topology map representation, the second augmented feature map representation, and the second enhanced brain topology map representation to obtain first augmented feature mapping features, first enhanced brain topology mapping features, second augmented feature mapping features, and second enhanced brain topology mapping features; Obtaining mutual information from the first view to the second view according to the first augmented feature map features and the second augmented feature map features; According to the first enhanced brain topology mapping feature and the second enhanced brain topology mapping feature, YY+242265P-YX is obtained. Mutual information from the second view to the first view; Obtaining a contrast loss value according to mutual information from the first view to the second view and mutual information from the second view to the first view; When the contrast loss value no longer changes, the iterative update stops.

8. The brain decoding method based on multi-view self-supervised brain topology inference according to claim 7, characterized in that: The step of performing enhancement processing on the brain data in the brain topology structure of the first view to obtain a first enhanced brain topology structure and a first augmented feature matrix includes: According to the first adjacency matrix in the first view topology structure model, generating a first random masking probability matrix based on Bernoulli distribution; Randomly masking the first adjacency matrix based on the first random masking probability matrix to obtain a first enhanced brain topology; generating a second random masking probability matrix according to the brain time series features in the first view topological structure model; The feature vector of each region of interest is masked based on the second random masking probability matrix to obtain a first augmented feature matrix.

9. The brain decoding method based on multi-view self-supervised brain topology inference according to claim 8, characterized in that: The step of performing enhancement processing on the brain data in the second view brain topology structure to obtain a second enhanced brain topology structure and a second augmented feature matrix includes: According to the second adjacency matrix in the second view topological structure model, a third random masking probability matrix is ​​generated based on Bernoulli distribution; the topological masking probability of the third random masking probability matrix is ​​the same as the topological masking probability of the first random masking probability matrix; Randomly masking the second adjacency matrix based on the third random masking probability matrix to obtain a second enhanced brain topology; generating a fourth random masking probability matrix according to the brain time series characteristics in the second view topological structure model; the characteristic masking probability of the fourth random masking probability matrix is ​​different from the characteristic masking probability of the second random masking probability matrix; The feature vector of each region of interest is masked based on the fourth random masking probability matrix to obtain a second augmented feature matrix.

10. The brain decoding method based on multi-view self-supervised brain topology inference according to claim 9, characterized in that: The contrast loss value is L total , Among them, L I→L is the mutual information from the first view to the second view, L L→I is the mutual information from the second view to the first view, H I,i is the first augmented feature map feature, H I,j The first enhanced brain topology mapping feature, H L,i is the second augmented feature map feature, H L,j is the second enhanced brain topology mapping feature, τ is the temperature parameter, sim(·) is the cosine similarity function, and z is the number of different brain regions.

Images