Brain age prediction method based on EEG-fMRI knowledge graph enhancement

Abstract

The invention discloses a brain age prediction method based on EEG-fMRI knowledge graph enhancement, and relates to the technical field of brain age prediction, and the method comprises the steps: for each target measurer, respectively extracting an activated brain region sequence according to EEG data and fMRI data of the target measurer, then capturing similar subsequences of the two, and carrying out brain age prediction; after alignment, an EEG-fMRI knowledge graph established through brain region function connection is used for prompting a brain region-brain network affiliation relation, a brain region and brain network function connection relation and an EEG-fMRI cross validation supplementary relation in the brain region-brain network affiliation relation, the brain region and brain network function connection relation and the EEG-fMRI cross validation supplementary relation are merged into a semantic sequence, finally the semantic sequence is input into a CNN-Transform model to output a corresponding brain age prediction result, and the brain age prediction result is obtained by reversely outputting a token. Therefore, the resting-state brain activity signal transduction path can be observed. According to the method, the superiority of EEG data in time and the superiority of fMRI data in space are combined, and the accuracy of brain age prediction is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of brain age prediction, and more specifically, to a brain age prediction method based on EEG-fMRI knowledge graph enhancement. Background Art

[0002] Brain age prediction is an important topic in neuroscience and medical research. Its main purpose is to estimate the biological age of an individual's brain by analyzing their brain physiological and functional data. Accurate brain age prediction not only helps in the early diagnosis and intervention of neuropsychiatric diseases, but also provides a scientific basis for personalized medical care and brain health management. Knowledge graphs provide a multi-dimensional and structured knowledge framework for brain age prediction by integrating and associating a large amount of medical information and data. On this basis, combined with machine learning and data mining techniques, knowledge graphs can enhance feature representation, improve model interpretability, and promote multimodal data fusion.

[0003] In the field of brain science, knowledge graphs can integrate information on brain function, structural connectivity, disease-related genes, and other aspects, providing rich prior knowledge support for the model. Existing brain age prediction methods have achieved results to a certain extent, but there are still the following shortcomings: limitations of a single data modality, lack of an integrated model of prior knowledge, and lack of interpretability. In recent years, the Transformer architecture has been widely used in various data analysis tasks due to its outstanding performance in fields such as natural language processing. In brain age prediction, the Transformer can effectively capture long-range dependencies and complex features in EEG and fMRI data through its self-attention mechanism. Therefore, how to construct a knowledge graph-enhanced Transformer architecture based on EEG and fMRI data for brain age prediction is a problem that needs to be solved. Summary of the invention

[0004] In order to solve the above technical problems, the present invention proposes a brain age prediction method based on EEG-fMRI knowledge graph enhancement, which improves the accuracy, robustness and interpretability of brain age prediction by integrating multimodal neuroimaging data and domain prior knowledge, thereby providing more reliable technical support for brain health assessment and early diagnosis of related diseases.

[0005] The present invention provides a brain age prediction method based on EEG-fMRI knowledge graph enhancement, comprising the following steps:

[0006] Obtain the EEG data and fMRI data of the target subject, and extract the activated brain area sequence from the EEG dataset and fMRI data respectively;

[0007] identifying and capturing similar subsequences of EEG data and fMRI data in the activated brain region sequence, and aligning the captured subsequences;

[0008] Establish an EEG-fMRI knowledge graph based on the functional connectivity of brain regions, and use the EEG-fMRI knowledge graph to fuse and mine the relationships of the aligned subsequence pairs to obtain semantic sequences;

[0009] The semantic sequence is imported into the CNN-Transformer model, and the brain age prediction result of the target subject is output. By reversely outputting tokens, the observation of the signal conduction pathway of resting brain activity is achieved.

[0010] In this scheme, the EEG data and fMRI data of the target subject are obtained, and the activated brain area sequences are extracted from the EEG dataset and fMRI data, respectively, as follows:

[0011] Obtain the EEG data and fMRI data of the target measurement subject. When judging whether the current brain region is activated at the current timestamp based on the fMRI data, obtain the degree to which the fMRI power value of the current timestamp deviates from the mean fMRI power value of the current brain region during the observation period of the target measurement subject as the fMRI data activation value. The fMRI data activation value of the current brain region i at the current time t0 is for:

[0012]

[0013] in represents the normalized fMRI power value time vector of the fMRI data sample v in the current brain region i, σ represents the hyperparameter, Indicates the degree of mean deviation after normalization;

[0014] When judging whether the current brain region is activated at the current timestamp according to the EEG data, the weighted sum of the degree of deviation of the EEG power value of the current timestamp from the mean value of the EEG power value of the current brain region during the observation period of the target measurer and the degree of jitter of the EEG power value in the preset time period before the current moment is obtained as the EEG data activation value. The EEG data activation value of the current brain region i at the current time t0 is for:

[0015]

[0016] in represents the normalized EEG power value time vector of the EEG data sample v in the current brain region i, σ represents the hyperparameter, Indicates the degree of mean deviation after normalization, θ0, θ j represents the weight parameter, represents the jitter degree of EEG power value in the preset time period j before the current moment, and k represents the length of the preset time period;

[0017] A judgment is made based on the activation values ​​of the fMRI data and the EEG data, and the activated brain area sequences corresponding to the fMRI data and the EEG data are extracted.

[0018] In this solution, similar subsequences of EEG data and fMRI data are identified and captured, and the captured subsequence pairs are aligned, specifically:

[0019] Cycle verification is used to identify and capture subsequence pairs in the activated brain area sequences corresponding to fMRI data and EEG data, and the matching function f of the subsequence pairs is defined:

[0020]

[0021] Where x, y represent the fMRI data vector and EEG data vector to be compared, respectively; k represents the scale factor; i, j represent the starting position and the ending position of the subsequence to be compared, respectively; σ represents the maximum difference threshold of the match; and m represents the number of subsequence items;

[0022] When the function value of the matching function is less than or equal to 0, the subsequences are matched, otherwise they are not matched. All similar subsequences of EEG data and fMRI data that meet the conditions are identified and captured to obtain subsequence pairs, and the captured subsequence pairs are aligned.

[0023] In this scheme, an EEG-fMRI knowledge graph is established based on the functional connectivity of brain regions, specifically:

[0024] Obtain EEG data, fMRI data and corresponding brain age detection data of healthy individuals of different ages, construct data samples, extract the activated brain region sequence in each data sample, establish the association between EEG data and fMRI data corresponding to the activated brain regions according to the functional connection of brain regions, and initialize the EEG-fMRI knowledge graph;

[0025] In the EEG-fMRI knowledge graph, the activated brain regions corresponding to EEG data and fMRI data are used as head nodes, the power values ​​of the brain regions are used as attributes of the nodes, and the brain age of the data samples is used as the tail node. The entity nodes and attributes are mapped in the form of triples.

[0026] Through data retrieval methods, we can obtain knowledge related to brain regions and brain networks, EEG data, and fMRI data. Through structured processing, we can obtain feature vectors of prior knowledge, capture deep features, and construct entities to supplement the EEG-fMRI knowledge graph.

[0027] Read the knowledge related to brain regions and brain networks, the semantic features of the knowledge related to EEG data and fMRI data, obtain the attribution relationship of brain region nodes to brain networks, the functional connection relationship between brain regions or between brain networks, and the cross-complementary verification relationship between EEG and fMRI;

[0028] A relationship table is constructed based on the extracted relationships, and the supplemented entities are mapped to the relationship table in the form of triples to complete the EEG-fMRI knowledge graph in a graphical way.

[0029] In this solution, the EEG-fMRI knowledge graph is used to fuse and mine the aligned subsequence pairs to obtain semantic sequences, specifically:

[0030] Importing the aligned subsequence pairs into the EEG-fMRI knowledge graph for pre-training, determining the core entities corresponding to the EEG data representation and the fMRI data representation in the aligned subsequence pairs, inputting the core entities into the EEG-fMRI knowledge graph, and locating them in the EEG-fMRI knowledge graph;

[0031] According to the positioning information of the core entity, search for entities whose distance from the core entity meets a preset distance threshold, and obtain a knowledge graph subgraph including the core entity and all entities and relationships that meet the preset distance threshold;

[0032] The knowledge graph subgraph prompts the brain region-brain network affiliation relationship, the brain region and brain network functional connection relationship, and the EEG-fMRI cross-validation supplementary relationship therein, and the entities in the knowledge graph subgraph corresponding to the EEG data representation and the fMRI data representation are fused, the fused entity representation is matched with the extracted relationship to generate semantic prompts, and the semantic sequence of the knowledge graph representation is output.

[0033] In this solution, the semantic sequence is imported into the CNN-Transformer model to output the brain age prediction result of the target measurement subject, specifically:

[0034] A CNN-Transformer model is constructed and trained, the semantic sequence is used as the model input, a convolution operation is used to extract features of the input semantic sequence, the number of channels of the extracted features is expanded, each channel is processed independently by using a depthwise separable convolution, unimportant features are suppressed, and finally the processed features are mapped back to the original number of output channels by a linear convolution, and a semantic local feature sequence is obtained by using a CNN;

[0035] Importing the semantic local feature sequence into the Transformer encoder, converting each local feature into an embedding vector, performing position encoding embedding according to the position of the local feature in the sequence, importing the position-encoded embedding vector into the self-attention layer, and using a multi-head self-attention mechanism to process different local features in parallel;

[0036] The output of the multi-head attention is added to the embedding vector of the original local feature, and then normalized and imported into a feed-forward neural network, where global features are obtained through linear transformation and activation;

[0037] The global feature representation is compressed into a vector of fixed dimension by using global average pooling, and regression is performed through a fully connected layer to obtain the brain age prediction result of the target person being measured.

[0038] In this scheme, the signal transmission pathway of resting brain activity is observed by reversely outputting tokens, specifically:

[0039] In the CNN-Transformer model, the attention weights of the semantic local feature sequence are obtained, and the attention weights output by all attention heads of all encoder layers are averaged to obtain the comprehensive attention weight matrix;

[0040] Calculating the total attention weight of each token at all positions according to the comprehensive attention weight matrix, and identifying the token that contributes most to the prediction result of the brain age of the target measured person according to the total attention weight;

[0041] According to the brain regions corresponding to the tokens with the largest contribution at different time steps, a sequence of brain region changes over time is constructed to generate the signal conduction pathway of resting brain activity and perform visualization.

[0042] The second aspect of the present invention provides a brain age prediction system based on EEG-fMRI knowledge graph enhancement, the system comprising a data acquisition module, a feature extraction module, a knowledge graph module, a semantic sequence generation module, a training module, a brain age prediction module and a path observation module;

[0043] The data acquisition module is responsible for acquiring the EEG data and fMRI data of the target subject and performing preliminary data processing;

[0044] The feature extraction module is responsible for extracting the activated brain region sequence from the EEG data and fMRI data, identifying and capturing similar subsequences in the EEG and fMRI data, and performing alignment processing;

[0045] The knowledge graph module is responsible for constructing and maintaining the EEG-fMRI knowledge graph;

[0046] The semantic sequence generation module is responsible for converting the aligned subsequence pairs into semantic sequences using the EEG-fMRI knowledge graph;

[0047] The training module is responsible for training the CNN-Transformer model and for regularly adding new data for retraining to achieve model optimization;

[0048] The brain age prediction module is responsible for predicting the brain age of the target subject using the trained CNN-Transformer model and outputting the result;

[0049] The path observation module is responsible for visualizing and analyzing the signal conduction path of the target subject's brain activity.

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] Aiming at the problem of multimodal fusion analysis and prediction of brain age, this scheme proposes a brain age prediction method and system based on EEG-fMRI knowledge graph. For each normal individual, the EEG data and fMRI data of the individual are used to extract the activated brain area sequence, and then the similar subsequences of the two are captured. After alignment, the EEG-fMRI knowledge graph established by the functional connection of the brain area is used to prompt the brain area-brain network affiliation relationship, the brain area and brain network functional connection relationship, and the EEG-fMRI cross-validation supplementary relationship, and finally merged into a semantic sequence similar to a "sentence" in natural language, and finally input into the CNN-Transformer model to output the corresponding brain age prediction result. Through the above method, the temporal superiority of EEG data and the spatial superiority of fMRI data can be combined to improve the accuracy of brain age prediction. It is also possible to observe the signal conduction pathway of resting brain activity by outputting tokens in reverse. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments or exemplary embodiments of the present invention, the drawings required for use in the embodiments or exemplary descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained according to the drawings without paying creative work.

[0053] Figure 1 A flowchart of a brain age prediction method based on EEG-fMRI knowledge graph enhancement is shown;

[0054] Figure 2 A flowchart for establishing an EEG-fMRI knowledge graph is shown;

[0055] Figure 3 A flowchart of using the CNN-Transformer model to obtain brain age prediction results is shown;

[0056] Figure 4 A flowchart of the brain age prediction system based on EEG-fMRI knowledge graph enhancement is shown. DETAILED DESCRIPTION

[0057] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0058] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0059] like Figure 1 As shown, in the first embodiment of the present invention, a method for predicting brain age based on EEG-fMRI knowledge graph enhancement is provided, comprising:

[0060] S102, obtaining EEG data and fMRI data of the target subject, and extracting activated brain region sequences from the EEG data set and fMRI data respectively;

[0061] S104, identifying and capturing similar subsequences of EEG data and fMRI data in the activated brain region sequence, and aligning the captured subsequences;

[0062] S106, establishing an EEG-fMRI knowledge graph based on the functional connections of brain regions, and using the EEG-fMRI knowledge graph to fuse and mine the relationships of the aligned subsequence pairs to obtain a semantic sequence;

[0063] S108, importing the semantic sequence into the CNN-Transformer model, outputting the brain age prediction result of the target person being measured, and realizing the observation of the signal conduction pathway of the resting brain activity by reversely outputting the token.

[0064] It should be noted that, when obtaining the target subject's EEG data and functional magnetic resonance imaging data (fMRI data), and judging whether the current brain region is activated at the current timestamp based on the fMRI data, the degree to which the fMRI power value at the current timestamp deviates from the mean fMRI power value of the current brain region during the target subject's observation period is obtained as the fMRI data activation value, and the fMRI data activation value of the current brain region i at the current time t0 is for:

[0065]

[0066] in represents the normalized fMRI power value time vector of the fMRI data sample v in the current brain region i, σ represents the hyperparameter, Indicates the degree of mean deviation after normalization;

[0067] When judging whether the current brain region is activated at the current timestamp according to the EEG data, the weighted sum of the degree of deviation of the EEG power value of the current timestamp from the mean value of the EEG power value of the current brain region during the observation period of the target measurer and the degree of jitter of the EEG power value in the preset time period before the current moment is obtained as the EEG data activation value. The EEG data activation value of the current brain region i at the current time t0 is for:

[0068]

[0069] in represents the normalized EEG power value time vector of the EEG data sample v in the current brain region i, σ represents the hyperparameter, Indicates the degree of mean deviation after normalization, θ0, θ j represents the weight parameter, represents the jitter degree of EEG power value in the preset time period j before the current moment, and k represents the length of the preset time period;

[0070] A judgment is made based on the activation values ​​of the fMRI data and the EEG data, and the activated brain area sequences corresponding to the fMRI data and the EEG data are extracted.

[0071] Cycle verification is used to identify and capture subsequence pairs in the activated brain area sequences corresponding to fMRI data and EEG data, and the matching function f of the subsequence pairs is defined:

[0072]

[0073] Where x, y represent the fMRI data vector and EEG data vector to be compared, respectively; k represents the scaling factor, which means that the sampling rate of EEG data is a multiple of the sampling rate of fMRI data in this data sample; i, j represent the starting position and the ending position of the subsequence to be compared, respectively; σ represents the maximum difference threshold for matching; and m represents the number of subsequence items;

[0074] When the function value of the matching function is less than or equal to 0, the subsequences are matched, otherwise they are not matched. All similar subsequences of EEG data and fMRI data that meet the conditions are identified and captured to obtain subsequence pairs, and the captured subsequence pairs are aligned.

[0075] Figure 2 A flowchart for building an EEG-fMRI knowledge graph is shown.

[0076] According to an embodiment of the present invention, an EEG-fMRI knowledge graph is established based on brain region functional connections, specifically:

[0077] S202, obtaining EEG data, fMRI data and corresponding brain age detection data of healthy individuals of different ages, constructing data samples, extracting the activated brain region sequence in each data sample, establishing the association between EEG data and fMRI data corresponding to the activated brain regions according to the functional connection of the brain regions, and initializing the EEG-fMRI knowledge graph;

[0078] S204, in the EEG-fMRI knowledge graph, the activated brain regions corresponding to the EEG data and fMRI data are used as head nodes, the power values ​​of the brain regions are used as attributes of the nodes, and the brain age of the data samples are used as tail nodes, and the entity nodes and attributes are mapped using triples;

[0079] S206, obtaining knowledge related to brain regions and brain networks, EEG data and fMRI data through data retrieval methods, obtaining feature vectors of prior knowledge through structured processing, capturing deep features to construct entities to supplement the EEG-fMRI knowledge graph;

[0080] S208, reading the knowledge related to brain regions and brain networks, the semantic features of the knowledge related to EEG data and fMRI data, obtaining the attribution relationship of brain region nodes to brain networks, the functional connection relationship between brain regions or between brain networks, and the cross-complementary verification relationship between EEG and fMRI;

[0081] S210, construct a relationship table based on the extracted relationships, map the supplemented entities and the relationship table in the form of triples, and complete the EEG-fMRI knowledge graph in a graphical manner.

[0082] It should be noted that the representation of EEG data and fMRI data is constructed as an EEG-fMRI knowledge graph, with EEG and fMRI brain regions as head nodes, brain region power values ​​as node attributes, and tail nodes as brain ages of data samples. The connection relationship of brain region nodes in the knowledge graph is realized through a pre-calculated brain region functional connection matrix. In order to better restore and complete the relationship of the knowledge graph, a structure-related prompt framework is designed based on the original knowledge graph. The specific implementation includes the attribution relationship of brain region nodes to brain networks, the functional connection relationship between brain regions or between brain networks, and the cross-complementary verification relationship between EEG and fMRI. Knowledge query and retrieval are performed based on the constructed EEG-fMRI knowledge graph to provide rich domain prior knowledge for brain age prediction.

[0083] The aligned subsequence pairs are imported into the EEG-fMRI knowledge graph for pre-training, the core entities corresponding to the EEG data representation and the fMRI data representation in the aligned subsequence pairs are determined, the core entities are input into the EEG-fMRI knowledge graph, and are positioned in the EEG-fMRI knowledge graph; entities whose distances from the core entities meet a preset distance threshold are searched according to the positioning information of the core entities, and a knowledge graph subgraph containing the core entity and all entities and relationships that meet the preset distance threshold is obtained; the brain region-brain network affiliation relationship, the brain region and brain network functional connection relationship, and the EEG-fMRI cross-validation supplementary relationship are prompted by the knowledge graph subgraph, the entities in the knowledge graph subgraph corresponding to the EEG data representation and the fMRI data representation are fused, the fused entity representations are matched with the extracted relationships to generate semantic prompts, and a semantic sequence represented by the knowledge graph is output. By identifying the core entity, searching for the entity and all entities with a distance x from the entity in the EEG-fMRI knowledge graph, and displaying all entities and relationships in the form of knowledge graph subgraphs, we can better understand the detailed information of the relevant entities, achieve effective fusion and relationship mining of EEG data and fMRI data, and obtain EEG-fMRI composite activated brain areas with semantic cues.

[0084] Figure 3 A flowchart for obtaining brain age prediction results using the CNN-Transformer model is shown.

[0085] According to an embodiment of the present invention, the semantic sequence is imported into the CNN-Transformer model, and the brain age prediction result of the target measurement subject is output, specifically:

[0086] S302, constructing a CNN-Transformer model and training it, taking the semantic sequence as the model input, extracting features from the input semantic sequence using a convolution operation, expanding the number of channels for the extracted features, independently processing each channel using a depthwise separable convolution, suppressing unimportant features, and finally mapping the processed features back to the original number of output channels through a linear convolution, and obtaining a semantic local feature sequence through a CNN;

[0087] S304, importing the semantic local feature sequence into the Transformer encoder, converting each local feature into an embedding vector, performing position encoding embedding according to the position of the local feature in the sequence, importing the position-encoded embedding vector into the self-attention layer, and using a multi-head self-attention mechanism to process different local features in parallel;

[0088] S306, adding the output of the multi-head attention to the embedding vector of the original local feature, performing a normalization operation and then importing the result into a feedforward neural network, and obtaining the global feature through linear transformation and activation in the feedforward neural network;

[0089] S308, using global average pooling to compress the global feature representation into a vector of fixed dimension, and performing regression through a fully connected layer to obtain a brain age prediction result of the target person being measured.

[0090] It should be noted that the training data is obtained to train the constructed CNN-Transformer model. The training and verification data sets are divided according to 8:2, and the network structure of the CNN-Transformer model is iteratively trained and output. In the CNN network of the CNN-Transformer model, a 3*3 convolution kernel is used to extract features from the input semantic sequence, and the activation operation is performed through the GELU activation function. Finally, the key features are further extracted through the maximum pooling layer to reduce the dimension of the features, making the model more compact and efficient. The number of channels of the extracted features is expanded using a 1*1 convolution kernel, creating a richer feature space for subsequent deep processing; each channel is processed independently using a 3*3 depth-separable convolution, and the importance of each channel is dynamically adjusted, which enhances the model's ability to capture key features and suppresses unimportant features. Finally, the processed features are mapped back to the original number of output channels through a 1*1 linear convolution, and a semantic local feature sequence is output.

[0091] The Transformer encoder is composed of a multi-head attention mechanism and a feedforward neural network. The semantic local feature sequence is imported into the Transformer encoder, and each local feature is converted into d using the embedding matrix E. model dimensional embedding vector e i , e i =E[w i ], position encoding is performed according to the position P of the local feature in the sequence and embedded into X = E[w i ]+PE. A multi-head self-attention mechanism is used to calculate the relationship between each time step i and all other time steps j. Multiple attention heads are used to process different attention subspaces in parallel. The feedforward neural network is input into the residual connection and layer normalization layer by two linear transformations and a ReLU activation function to prevent the gradient from disappearing. The feedforward neural network is used to obtain semantic global features.

[0092] In order to predict brain age, the output of the encoder needs to be converted into a continuous prediction value. Global average pooling is used to compress the representation of the sequence into a vector of fixed dimension, and then regression is performed through a fully connected layer (or multi-layer perceptron). Perform global average pooling h pool , L is the length of the encoder output: Generate the target person's brain age prediction result through regression head in is the weight and b is the bias term.

[0093] It should be noted that in the CNN-Transformer model, the attention weights of the semantic local feature sequence are obtained. By analyzing the attention weight matrix, it is possible to determine which tokens in the input sequence contribute the most to the final prediction result. Since there are multiple attention heads and multiple encoder layers, these attention weights need to be aggregated to obtain the overall attention distribution. The CNN-Transformer model has N encoder layers and h attention heads per layer, so all attention weight matrices can be aggregated. Aggregate into a comprehensive attention weight matrix A total , The total attention weight of each token at all positions is calculated according to the comprehensive attention weight matrix, and the token with the greatest contribution to the prediction result of the target subject's brain age is identified according to the total attention weight; wherein the contribution calculation formula is: Among them C j represents the contribution of the jth token, and k represents the number of feature items output by the encoder. The corresponding brain region is identified according to the token with the largest contribution at different time steps. A sequence of brain region changes over time is constructed based on the identified brain regions, and the signal conduction pathway of resting brain activity is generated and visualized.

[0094] Figure 4 A flowchart of the brain age prediction system based on EEG-fMRI knowledge graph enhancement is shown.

[0095] The second embodiment of the present invention provides a brain age prediction system based on EEG-fMRI knowledge graph enhancement, the system comprising a data acquisition module, a feature extraction module, a knowledge graph module, a semantic sequence generation module, a training module, a brain age prediction module and a path observation module;

[0096] The data acquisition module is responsible for acquiring the EEG data and fMRI data of the target subject and performing preliminary data processing;

[0097] The feature extraction module is responsible for extracting the activated brain region sequence from the EEG data and fMRI data, identifying and capturing similar subsequences in the EEG and fMRI data, and performing alignment processing;

[0098] The knowledge graph module is responsible for constructing and maintaining the EEG-fMRI knowledge graph;

[0099] The semantic sequence generation module is responsible for converting the aligned subsequence pairs into semantic sequences using the EEG-fMRI knowledge graph;

[0100] The training module is responsible for training the CNN-Transformer model and for regularly adding new data for retraining to achieve model optimization;

[0101] The brain age prediction module is responsible for predicting the brain age of the target subject using the trained CNN-Transformer model and outputting the result;

[0102] The path observation module is responsible for visualizing and analyzing the signal conduction path of the target subject's brain activity.

[0103] The third embodiment of the present invention provides a computer-readable storage medium, which includes a brain age prediction method program based on EEG-fMRI knowledge graph enhancement. When the brain age prediction method program based on EEG-fMRI knowledge graph enhancement is executed by a processor, the steps of the brain age prediction method based on EEG-fMRI knowledge graph enhancement are implemented.

[0104] In the several embodiments provided in the present application, it should be understood that the disclosed method can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as: multiple modules or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0105] Those skilled in the art can understand that: all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiments; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), disks or optical disks, and other media that can store program codes.

[0106] Alternatively, if the above-mentioned integrated module of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0107] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A brain age prediction method based on EEG-fMRI knowledge graph enhancement, characterized in that: The following steps are involved: Obtain the EEG data and fMRI data of the target subject, and extract the activated brain area sequence from the EEG dataset and fMRI data respectively; identifying and capturing similar subsequences of EEG data and fMRI data in the activated brain region sequence, and aligning the captured subsequences; Establish an EEG-fMRI knowledge graph based on the functional connectivity of brain regions, and use the EEG-fMRI knowledge graph to fuse and mine the relationships of the aligned subsequence pairs to obtain semantic sequences; The semantic sequence is imported into the CNN-Transformer model, and the brain age prediction result of the target subject is output. By reversely outputting tokens, the observation of the signal conduction pathway of resting brain activity is achieved.

2. The method for predicting brain age based on EEG-fMRI knowledge graph enhancement according to claim 1, characterized in that: Obtain the EEG data and fMRI data of the target subject, and extract the activated brain area sequence from the EEG dataset and fMRI data, respectively, as follows: Obtain the EEG data and fMRI data of the target measurement subject. When judging whether the current brain region is activated at the current timestamp based on the fMRI data, obtain the degree to which the fMRI power value of the current timestamp deviates from the mean fMRI power value of the current brain region during the observation period of the target measurement subject as the fMRI data activation value. The fMRI data activation value of the current brain region i at the current time t0 is for: in represents the normalized fMRI power value time vector of the fMRI data sample v in the current brain region i, σ represents the hyperparameter, Indicates the degree of mean deviation after normalization; When judging whether the current brain region is activated at the current timestamp according to the EEG data, the weighted sum of the degree of deviation of the EEG power value of the current timestamp from the mean value of the EEG power value of the current brain region during the observation period of the target measurer and the degree of jitter of the EEG power value in the preset time period before the current moment is obtained as the EEG data activation value. The EEG data activation value of the current brain region i at the current time t0 is for: in represents the normalized EEG power value time vector of the EEG data sample v in the current brain region i, σ represents the hyperparameter, Indicates the degree of mean deviation after normalization, θ0, θ j represents the weight parameter, represents the jitter degree of EEG power value in the preset time period j before the current moment, and k represents the length of the preset time period; A judgment is made based on the activation values ​​of the fMRI data and the EEG data, and the activated brain area sequences corresponding to the fMRI data and the EEG data are extracted.

3. The method for predicting brain age based on EEG-fMRI knowledge graph enhancement according to claim 1, characterized in that: Identify and capture similar subsequences of EEG data and fMRI data, and align the captured subsequence pairs, specifically: Cycle verification is used to identify and capture subsequence pairs in the activated brain area sequences corresponding to fMRI data and EEG data, and the matching function f of the subsequence pairs is defined: Where x, y represent the fMRI data vector and EEG data vector to be compared, respectively; k represents the scale factor; i, j represent the starting position and the ending position of the subsequence to be compared, respectively; σ represents the maximum difference threshold of the match; and m represents the number of subsequence items; When the function value of the matching function is less than or equal to 0, the subsequences are matched, otherwise they are not matched. All similar subsequences of EEG data and fMRI data that meet the conditions are identified and captured to obtain subsequence pairs, and the captured subsequence pairs are aligned.

4. The method for predicting brain age based on EEG-fMRI knowledge graph enhancement according to claim 1, characterized in that: Establish an EEG-fMRI knowledge graph based on brain region functional connectivity, specifically: Obtain EEG data, fMRI data and corresponding brain age detection data of healthy individuals of different ages, construct data samples, extract the activated brain region sequence in each data sample, establish the association between EEG data and fMRI data corresponding to the activated brain regions according to the functional connection of brain regions, and initialize the EEG-fMRI knowledge graph; In the EEG-fMRI knowledge graph, the activated brain regions corresponding to EEG data and fMRI data are used as head nodes, the power values ​​of the brain regions are used as attributes of the nodes, and the brain age of the data samples is used as the tail node. The entity nodes and attributes are mapped in the form of triples. Through data retrieval methods, we can obtain knowledge related to brain regions and brain networks, EEG data, and fMRI data. Through structured processing, we can obtain feature vectors of prior knowledge, capture deep features, and construct entities to supplement the EEG-fMRI knowledge graph. Read the knowledge related to brain regions and brain networks, the semantic features of the knowledge related to EEG data and fMRI data, obtain the attribution relationship of brain region nodes to brain networks, the functional connection relationship between brain regions or between brain networks, and the cross-complementary verification relationship between EEG and fMRI; A relationship table is constructed based on the extracted relationships, and the supplemented entities are mapped to the relationship table in the form of triples to complete the EEG-fMRI knowledge graph in a graphical way.

5. The method for predicting brain age based on EEG-fMRI knowledge graph enhancement according to claim 1, characterized in that: The EEG-fMRI knowledge graph is used to fuse and mine the relationships of the aligned subsequence pairs to obtain semantic sequences, specifically: Importing the aligned subsequence pairs into the EEG-fMRI knowledge graph for pre-training, determining the core entities corresponding to the EEG data representation and the fMRI data representation in the aligned subsequence pairs, inputting the core entities into the EEG-fMRI knowledge graph, and locating them in the EEG-fMRI knowledge graph; According to the positioning information of the core entity, search for entities whose distance from the core entity meets a preset distance threshold, and obtain a knowledge graph subgraph including the core entity and all entities and relationships that meet the preset distance threshold; The knowledge graph subgraph prompts the brain region-brain network affiliation relationship, the brain region and brain network functional connection relationship, and the EEG-fMRI cross-validation supplementary relationship therein, and the entities in the knowledge graph subgraph corresponding to the EEG data representation and the fMRI data representation are fused, the fused entity representation is matched with the extracted relationship to generate semantic prompts, and the semantic sequence of the knowledge graph representation is output.

6. The method for predicting brain age based on EEG-fMRI knowledge graph enhancement according to claim 1, characterized in that: The semantic sequence is imported into the CNN-Transformer model, and the brain age prediction result of the target measurement subject is output, specifically: A CNN-Transformer model is constructed and trained, the semantic sequence is used as the model input, a convolution operation is used to extract features of the input semantic sequence, the number of channels of the extracted features is expanded, each channel is processed independently by using a depthwise separable convolution, unimportant features are suppressed, and finally the processed features are mapped back to the original number of output channels by a linear convolution, and a semantic local feature sequence is obtained by using a CNN; Importing the semantic local feature sequence into the Transformer encoder, converting each local feature into an embedding vector, performing position encoding embedding according to the position of the local feature in the sequence, importing the position-encoded embedding vector into the self-attention layer, and using a multi-head self-attention mechanism to process different local features in parallel; The output of the multi-head attention is added to the embedding vector of the original local feature, and then normalized and imported into a feed-forward neural network, where global features are obtained through linear transformation and activation; The global feature representation is compressed into a vector of fixed dimension by using global average pooling, and regression is performed through a fully connected layer to obtain the brain age prediction result of the target person being measured.

7. The method for predicting brain age based on EEG-fMRI knowledge graph enhancement according to claim 1, characterized in that: By outputting tokens in reverse, we can observe the signal transmission pathway of resting brain activity, specifically: In the CNN-Transformer model, the attention weights of the semantic local feature sequence are obtained, and the attention weights output by all attention heads of all encoder layers are averaged to obtain the comprehensive attention weight matrix; Calculating the total attention weight of each token at all positions according to the comprehensive attention weight matrix, and identifying the token that contributes most to the prediction result of the brain age of the target measured person according to the total attention weight; According to the brain regions corresponding to the tokens with the largest contribution at different time steps, a sequence of brain region changes over time is constructed to generate the signal conduction pathway of resting brain activity and perform visualization.

8. A brain age prediction system based on EEG-fMRI knowledge graph enhancement, characterized in that: Implementing the brain age prediction method based on EEG-fMRI knowledge graph enhancement as described in any one of claims 1 to 7, the system includes a data acquisition module, a feature extraction module, a knowledge graph module, a semantic sequence generation module, a training module, a brain age prediction module and a path observation module; The data acquisition module is responsible for acquiring the EEG data and fMRI data of the target subject and performing preliminary data processing; The feature extraction module is responsible for extracting the activated brain region sequence from the EEG data and fMRI data, identifying and capturing similar subsequences in the EEG and fMRI data, and performing alignment processing; The knowledge graph module is responsible for constructing and maintaining the EEG-fMRI knowledge graph; The semantic sequence generation module is responsible for converting the aligned subsequence pairs into semantic sequences using the EEG-fMRI knowledge graph; The training module is responsible for training the CNN-Transformer model and for regularly adding new data for retraining to achieve model optimization; The brain age prediction module is responsible for predicting the brain age of the target person using the trained CNN-Transformer model and outputting the result; The path observation module is responsible for visualizing and analyzing the signal conduction path of the target measured person's brain activity.

Images