Brain age prediction method based on EEG-fMRI knowledge graph enhancement

Abstract

The present invention discloses a brain age prediction method based on EEG-fMRI knowledge graph enhancement, which relates to the field of brain age prediction technology. The method comprises: for each target measurer, extracting activated brain region sequences based on their EEG data and fMRI data, then capturing similar subsequences of the two, aligning them, and using the EEG-fMRI knowledge graph established by brain region functional connectivity to prompt the brain region-brain network affiliation, brain region and brain network functional connectivity, and EEG-fMRI cross-validation supplementary relationships, merging them into semantic sequences, and finally inputting them into a CNN-Transformer model to output the corresponding brain age prediction results. By reversely outputting tokens, the method realizes the observation of the signal conduction pathway of resting brain activity. The present invention combines the temporal superiority of EEG data with the spatial superiority of fMRI data to improve the accuracy of brain age prediction.

Description

Technical Field

[0001] The present invention relates to the technical field of brain age prediction, and more specifically, to a brain age prediction method based on EEG-fMRI knowledge graph enhancement. Background Art

[0002] Brain age prediction is a key topic in neuroscience and medical research. Its primary goal is to estimate an individual's biological brain age by analyzing their physiological and functional brain data. Accurate brain age prediction not only facilitates early diagnosis and intervention for neuropsychiatric disorders but also provides a scientific basis for personalized medicine and brain health management. Knowledge graphs, by integrating and linking vast amounts of medical information and data, provide a multidimensional, structured framework for brain age prediction. On this basis, combined with machine learning and data mining techniques, knowledge graphs can enhance feature representation, improve model interpretability, and facilitate multimodal data fusion.

[0003] In the field of brain science, knowledge graphs can integrate information on various aspects, such as brain region function, structural connectivity, and disease-related genes, providing rich prior knowledge support for models. Existing brain age prediction methods have achieved some success, but they still suffer from the following shortcomings: limitations of a single data modality, a lack of models integrating prior knowledge, and insufficient interpretability. In recent years, the Transformer architecture has been widely used in various data analysis tasks due to its outstanding performance in fields such as natural language processing. In brain age prediction, the Transformer, through its self-attention mechanism, can effectively capture long-range dependencies and complex features in EEG and fMRI data. Therefore, how to construct a knowledge graph-enhanced Transformer architecture based on EEG and fMRI data for brain age prediction is a problem that needs to be solved. Summary of the Invention

[0004] In order to solve the above technical problems, the present invention proposes a brain age prediction method based on EEG-fMRI knowledge graph enhancement. By integrating multimodal neuroimaging data and domain prior knowledge, the accuracy, robustness and interpretability of brain age prediction are improved, thereby providing more reliable technical support for brain health assessment and early diagnosis of related diseases.

[0005] The present invention provides a brain age prediction method based on EEG-fMRI knowledge graph enhancement, comprising the following steps:

[0006] Obtain the EEG data and fMRI data of the target subject, and extract the activated brain area sequences from the EEG dataset and fMRI data respectively;

[0007] identifying and capturing similar subsequences of EEG data and fMRI data in the activated brain region sequence, and aligning the captured subsequences;

[0008] Establish an EEG-fMRI knowledge graph based on the functional connectivity of brain regions, and use the EEG-fMRI knowledge graph to fuse and mine the relationships of the aligned subsequence pairs to obtain semantic sequences;

[0009] The semantic sequence is imported into the CNN-Transformer model to output the brain age prediction result of the target subject. By reversely outputting tokens, the signal conduction pathway of resting brain activity is observed.

[0010] In this scheme, the EEG data and fMRI data of the target subject are obtained, and the activated brain area sequences are extracted from the EEG dataset and fMRI data respectively. Specifically:

[0011] Obtain the EEG data and fMRI data of the target measurement subject. When judging whether the current brain region is activated at the current timestamp based on the fMRI data, obtain the degree to which the fMRI power value of the current timestamp deviates from the mean fMRI power value of the current brain region during the observation period of the target measurement subject as the fMRI data activation value. The fMRI data activation value of the current brain region i at the current time t0 is for:

[0012]

[0013] in represents the normalized fMRI power value time vector of the fMRI data sample v in the current brain region i, σ represents the hyperparameter, Indicates the degree of mean deviation after normalization;

[0014] When judging whether the current brain region is activated at the current timestamp based on the EEG data, the weighted sum of the degree of deviation of the EEG power value of the current timestamp from the mean EEG power value of the current brain region during the observation period of the target measurer and the jitter degree of the EEG power value in the preset time period before the current moment is obtained as the EEG data activation value. The EEG data activation value of the current brain region i at the current time t0 is for:

[0015]

[0016] in represents the normalized EEG power value time vector of the EEG data sample v in the current brain region i, σ represents the hyperparameter, Indicates the degree of mean deviation after normalization, θ0, θ j represents the weight parameter, represents the jitter degree of EEG power value in the preset time period j before the current moment, and k represents the length of the preset time period;

[0017] A judgment is made based on the activation values ​​of the fMRI data and the EEG data, and an activated brain region sequence corresponding to the fMRI data and the EEG data is extracted.

[0018] In this solution, similar subsequences of EEG data and fMRI data are identified and captured, and the captured subsequence pairs are aligned. Specifically,

[0019] Cycle verification is used to identify and capture subsequence pairs in the activation brain area sequences corresponding to fMRI data and EEG data, and the matching function f of the subsequence pairs is defined as:

[0020]

[0021] Where x, y represent the fMRI data vector and EEG data vector to be compared, respectively; k represents the scaling factor; i, j represent the starting position and ending position of the subsequence to be compared, respectively; σ represents the maximum difference threshold for matching; and m represents the number of subsequence items.

[0022] When the function value of the matching function is less than or equal to 0, the subsequences are matched, otherwise they are not matched. All similar subsequences of EEG data and fMRI data that meet the conditions are identified and captured to obtain subsequence pairs, and the captured subsequence pairs are aligned.

[0023] In this scheme, an EEG-fMRI knowledge graph is established based on the functional connectivity of brain regions, specifically:

[0024] Obtain EEG data, fMRI data, and corresponding brain age detection data from healthy individuals of different ages, construct data samples, extract the sequence of activated brain regions in each data sample, establish the association between EEG data and fMRI data corresponding to the activated brain regions based on the functional connectivity of the brain regions, and initialize the EEG-fMRI knowledge graph;

[0025] In the EEG-fMRI knowledge graph, the activated brain regions corresponding to EEG data and fMRI data are used as head nodes, the brain region power values ​​are used as node attributes, and the brain age of the data samples is used as the tail node. The entity nodes and attributes are mapped using triples.

[0026] Through data retrieval methods, we acquire knowledge related to brain regions and brain networks, EEG data, and fMRI data. Through structured processing, we obtain feature vectors of prior knowledge, capture deep features, and construct entities to supplement the EEG-fMRI knowledge graph.

[0027] Read the semantic features of knowledge related to brain regions and brain networks, EEG data and fMRI data, and obtain the attribution relationship of brain region nodes to brain networks, the functional connectivity relationship between brain regions or between brain networks, and the cross-complementary verification relationship between EEG and fMRI;

[0028] A relational table is constructed based on the extracted relationships, and the supplemented entities are mapped to the relational table in the form of triples to complete the EEG-fMRI knowledge graph in a graphical way.

[0029] In this solution, the EEG-fMRI knowledge graph is used to fuse and mine the aligned subsequence pairs to obtain semantic sequences. Specifically:

[0030] Importing the aligned subsequence pairs into the EEG-fMRI knowledge graph for pre-training, determining core entities corresponding to the EEG data representation and the fMRI data representation in the aligned subsequence pairs, inputting the core entities into the EEG-fMRI knowledge graph, and locating them in the EEG-fMRI knowledge graph;

[0031] Searching for entities whose distance from the core entity meets a preset distance threshold according to the positioning information of the core entity, and obtaining a knowledge graph subgraph including the core entity and all entities and relationships that meet the preset distance threshold;

[0032] The knowledge graph subgraph prompts the brain region-brain network affiliation relationship, the brain region and brain network functional connection relationship, and the EEG-fMRI cross-validation supplementary relationship therein, and the entities in the knowledge graph subgraph corresponding to the EEG data representation and the fMRI data representation are fused, and the fused entity representation is matched with the extracted relationship to generate semantic prompts, and the semantic sequence of the knowledge graph representation is output.

[0033] In this solution, the semantic sequence is imported into the CNN-Transformer model to output the brain age prediction result of the target subject, specifically:

[0034] A CNN-Transformer model is constructed and trained. The semantic sequence is used as the model input. The input semantic sequence is subjected to feature extraction using convolution operations. The extracted features are expanded in number of channels. Each channel is processed independently using depthwise separable convolution to suppress unimportant features. Finally, the processed features are mapped back to the original number of output channels using linear convolution, and a semantic local feature sequence is obtained through CNN.

[0035] Importing the semantic local feature sequence into the Transformer encoder, converting each local feature into an embedding vector, performing position encoding embedding according to the position of the local feature in the sequence, importing the position-encoded embedding vector into the self-attention layer, and using a multi-head self-attention mechanism to process different local features in parallel;

[0036] The output of the multi-head attention is added to the embedding vector of the original local feature, and after normalization, it is introduced into the feedforward neural network, where the global feature is obtained through linear transformation and activation;

[0037] Global average pooling is used to compress the global feature representation into a vector of fixed dimension, and regression is performed through a fully connected layer to obtain the brain age prediction result of the target person being measured.

[0038] In this scheme, the signal transmission pathway of resting brain activity is observed by outputting tokens in reverse, specifically:

[0039] In the CNN-Transformer model, the attention weights of the semantic local feature sequence are obtained, and the attention weights output by all attention heads of all encoder layers are averaged to obtain the comprehensive attention weight matrix;

[0040] Calculating the total attention weight of each token at all positions according to the comprehensive attention weight matrix, and identifying the token that contributes most to the brain age prediction result of the target subject according to the total attention weight;

[0041] Based on the brain regions corresponding to the tokens with the largest contribution at different time steps, a sequence of brain region changes over time is constructed to generate the signal conduction pathway of resting-state brain activity and perform visualization.

[0042] The second aspect of the present invention provides a brain age prediction system based on EEG-fMRI knowledge graph enhancement, which includes a data acquisition module, a feature extraction module, a knowledge graph module, a semantic sequence generation module, a training module, a brain age prediction module and a path observation module;

[0043] The data acquisition module is responsible for acquiring the EEG data and fMRI data of the target subject and performing preliminary data processing;

[0044] The feature extraction module is responsible for extracting activated brain region sequences from EEG data and fMRI data, identifying and capturing similar subsequences in EEG and fMRI data, and performing alignment processing;

[0045] The knowledge graph module is responsible for constructing and maintaining the EEG-fMRI knowledge graph;

[0046] The semantic sequence generation module is responsible for converting the aligned subsequence pairs into semantic sequences using the EEG-fMRI knowledge graph;

[0047] The training module is responsible for training the CNN-Transformer model and regularly adding new data for retraining to achieve model optimization;

[0048] The brain age prediction module is responsible for using the trained CNN-Transformer model to predict the brain age of the target subject and output the result;

[0049] The path observation module is responsible for visualizing and analyzing the signal conduction path of the target measured person's brain activity.

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] To address the problem of multimodal fusion analysis and prediction of brain age, this proposal proposes a brain age prediction method and system based on an EEG-fMRI knowledge graph. For each healthy individual, activation sequences are extracted from both the EEG and fMRI data. Similar subsequences between the two are then captured and aligned. The EEG-fMRI knowledge graph, constructed using functional connectivity between brain regions, reveals region-network relationships, functional connectivity between regions and networks, and EEG-fMRI cross-validation complementarity. Ultimately, this is merged into a semantically structured sequence, similar to a "sentence" in natural language. This is then input into a CNN-Transformer model to produce the corresponding brain age prediction result. This approach combines the temporal advantages of EEG data with the spatial advantages of fMRI data to improve the accuracy of brain age prediction. Furthermore, by outputting tokens in reverse order, the signal transmission pathways of resting-state brain activity can be observed. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments or exemplary embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or exemplary descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained according to these drawings without paying any creative work.

[0053] Figure 1 A flowchart of the brain age prediction method based on EEG-fMRI knowledge graph enhancement is shown;

[0054] Figure 2 A flowchart for building an EEG-fMRI knowledge graph is shown;

[0055] Figure 3 A flowchart showing the use of the CNN-Transformer model to obtain brain age prediction results is shown;

[0056] Figure 4 A flowchart of the brain age prediction system based on EEG-fMRI knowledge graph enhancement is shown. DETAILED DESCRIPTION

[0057] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present application and the features therein can be combined with each other.

[0058] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0059] like Figure 1 As shown, the first embodiment of the present invention provides a brain age prediction method based on EEG-fMRI knowledge graph enhancement, including:

[0060] S102, obtaining EEG data and fMRI data of the target subject, and extracting activated brain region sequences from the EEG dataset and fMRI data respectively;

[0061] S104, identifying and capturing similar subsequences of EEG data and fMRI data in the activated brain region sequence, and aligning the captured subsequences;

[0062] S106, establishing an EEG-fMRI knowledge graph based on the functional connectivity of brain regions, and using the EEG-fMRI knowledge graph to fuse and mine the relationships of the aligned subsequence pairs to obtain a semantic sequence;

[0063] S108, importing the semantic sequence into the CNN-Transformer model, outputting the brain age prediction result of the target person being measured, and realizing the observation of the signal conduction pathway of the resting brain activity by reversely outputting the token.

[0064] It should be noted that, when obtaining the target measurement subject's EEG data and functional magnetic resonance imaging (fMRI) data, and judging whether the current brain region is activated at the current timestamp based on the fMRI data, the degree to which the current timestamp fMRI power value deviates from the mean fMRI power value of the current brain region during the target measurement subject's observation period is obtained as the fMRI data activation value, and the fMRI data activation value of the current brain region i at the current time t0 is for:

[0065]

[0066] in represents the normalized fMRI power value time vector of the fMRI data sample v in the current brain region i, σ represents the hyperparameter, Indicates the degree of mean deviation after normalization;

[0067] When judging whether the current brain region is activated at the current timestamp based on the EEG data, the weighted sum of the degree of deviation of the EEG power value of the current timestamp from the mean EEG power value of the current brain region during the observation period of the target measurer and the jitter degree of the EEG power value in the preset time period before the current moment is obtained as the EEG data activation value. The EEG data activation value of the current brain region i at the current time t0 is for:

[0068]

[0069] in represents the normalized EEG power value time vector of the EEG data sample v in the current brain region i, σ represents the hyperparameter, Indicates the degree of mean deviation after normalization, θ0, θ j represents the weight parameter, represents the jitter degree of EEG power value in the preset time period j before the current moment, and k represents the length of the preset time period;

[0070] A judgment is made based on the activation values ​​of the fMRI data and the EEG data, and an activated brain region sequence corresponding to the fMRI data and the EEG data is extracted.

[0071] Cycle verification is used to identify and capture subsequence pairs in the activation brain area sequences corresponding to fMRI data and EEG data, and the matching function f of the subsequence pairs is defined as:

[0072]

[0073] Where x, y represent the fMRI data vector and EEG data vector to be compared, respectively; k represents the scaling factor, which means that the EEG data sampling rate is a multiple of the fMRI data sampling rate in this data sample; i, j represent the starting position and ending position of the subsequence to be compared, respectively; σ represents the maximum difference threshold for matching; and m represents the number of subsequence items.

[0074] When the function value of the matching function is less than or equal to 0, the subsequences are matched, otherwise they are not matched. All similar subsequences of EEG data and fMRI data that meet the conditions are identified and captured to obtain subsequence pairs, and the captured subsequence pairs are aligned.

[0075] Figure 2 A flowchart for building an EEG-fMRI knowledge graph is shown.

[0076] According to an embodiment of the present invention, an EEG-fMRI knowledge graph is established based on the functional connectivity of brain regions, specifically:

[0077] S202, obtaining EEG data, fMRI data, and corresponding brain age detection data of healthy individuals of different ages, constructing data samples, extracting the sequence of activated brain regions in each data sample, establishing associations between EEG data and fMRI data corresponding to activated brain regions based on brain region functional connectivity, and initializing the EEG-fMRI knowledge graph;

[0078] S204, in the EEG-fMRI knowledge graph, the activated brain regions corresponding to the EEG data and fMRI data are used as head nodes, the brain region power values ​​are used as attributes of the nodes, and the brain age of the data samples is used as the tail node, and the entity nodes and attributes are mapped using triples;

[0079] S206, obtaining knowledge related to brain regions and brain networks, EEG data, and fMRI data through data retrieval methods, obtaining feature vectors of prior knowledge through structured processing, capturing deep features to construct entities to supplement the EEG-fMRI knowledge graph;

[0080] S208, reading the semantic features of the knowledge related to brain regions and brain networks, EEG data and fMRI data, obtaining the attribution relationship of brain region nodes to brain networks, the functional connectivity relationship between brain regions or between brain networks, and the cross-complementary verification relationship between EEG and fMRI;

[0081] S210, constructing a relationship table based on the extracted relationships, mapping the supplemented entities and the relationship table into triples, and completing the EEG-fMRI knowledge graph in a graphical manner.

[0082] It should be noted that the representation of EEG and fMRI data is constructed as an EEG-fMRI knowledge graph, with EEG and fMRI brain regions as head nodes, brain region power values ​​as node attributes, and tail nodes as the brain age of the data sample. The connection relationship between brain region nodes in the knowledge graph is realized through a pre-calculated brain region functional connectivity matrix. To better restore and complete the relationships in the knowledge graph, a structurally relevant prompt framework is designed based on the original knowledge graph. Specific implementations include the attribution relationship of brain region nodes to brain networks, the functional connectivity relationship between brain regions or between brain networks, and the cross-complementary verification relationship between EEG and fMRI. Knowledge query and retrieval are performed based on the constructed EEG-fMRI knowledge graph, providing rich domain prior knowledge for brain age prediction.

[0083] The aligned subsequence pairs are imported into the EEG-fMRI knowledge graph for pre-training, and the core entities corresponding to the EEG data representation and the fMRI data representation in the aligned subsequence pairs are determined. The core entities are input into the EEG-fMRI knowledge graph and located in the EEG-fMRI knowledge graph; based on the positioning information of the core entities, entities whose distance from the core entities meets a preset distance threshold are searched, and a knowledge graph subgraph containing the core entity and all entities and relationships that meet the preset distance threshold is obtained; the brain region-brain network affiliation relationship, the brain region and brain network functional connection relationship, and the EEG-fMRI cross-validation supplementary relationship are prompted by the knowledge graph subgraph, and the entities in the knowledge graph subgraph corresponding to the EEG data representation and the fMRI data representation are fused, the fused entity representations are matched with the extracted relationships to generate semantic hints, and a semantic sequence of the knowledge graph representation is output. By identifying the core entity, searching for the entity and all entities with a distance x from it in the EEG-fMRI knowledge graph, and displaying all entities and relationships in the form of knowledge graph subgraphs, we can better understand the detailed information of the relevant entities, achieve effective fusion and relationship mining of EEG data and fMRI data, and obtain EEG-fMRI composite activated brain areas with semantic cues.

[0084] Figure 3 A flowchart for obtaining brain age prediction results using the CNN-Transformer model is shown.

[0085] According to an embodiment of the present invention, the semantic sequence is imported into the CNN-Transformer model to output the brain age prediction result of the target measurement subject, specifically:

[0086] S302, constructing a CNN-Transformer model and training it, taking the semantic sequence as the model input, extracting features from the input semantic sequence using a convolution operation, expanding the number of channels of the extracted features, independently processing each channel using depthwise separable convolution to suppress unimportant features, and finally mapping the processed features back to the original number of output channels using linear convolution, and obtaining a semantic local feature sequence through CNN;

[0087] S304: importing the semantic local feature sequence into a Transformer encoder, converting each local feature into an embedding vector, performing position encoding embedding according to the position of the local feature in the sequence, importing the position-encoded embedding vector into a self-attention layer, and using a multi-head self-attention mechanism to process different local features in parallel;

[0088] S306, adding the output of the multi-head attention to the embedding vector of the original local feature, performing a normalization operation, and then importing it into a feedforward neural network, where a global feature is obtained through linear transformation and activation;

[0089] S308 , using global average pooling to compress the global feature representation into a vector of fixed dimension, and performing regression through a fully connected layer to obtain a brain age prediction result of the target person being measured.

[0090] It should be noted that training data is obtained to train the constructed CNN-Transformer model. The training and validation datasets are split in an 8:2 ratio. Iterative training is performed and the network structure of the CNN-Transformer model is output. The CNN network of the CNN-Transformer model uses 3*3 convolution kernels to extract features from the input semantic sequence. The GELU activation function is used for activation, and finally a maximum pooling layer is used to further extract key features, reducing the feature dimension and making the model more compact and efficient. 1*1 convolution kernels are used to expand the number of channels of the extracted features, creating a richer feature space for subsequent deep processing. 3*3 depthwise separable convolutions are used to independently process each channel and dynamically adjust the importance of each channel, enhancing the model's ability to capture key features and suppressing unimportant features. Finally, a 1*1 linear convolution is used to map the processed features back to the original number of output channels, outputting a sequence of semantic local features.

[0091] The Transformer encoder is composed of a multi-head attention mechanism and a feedforward neural network. The semantic local feature sequence is imported into the Transformer encoder, and each local feature is converted into d using the embedding matrix E. model dimensional embedding vector e i , e i =E[w i ], position encoding is performed according to the position P of the local feature in the sequence and embedded into X=E[w i ]+PE. A multi-head self-attention mechanism is used to calculate the relationship between each time step i and all other time steps j. Multiple attention heads are used to process different attention subspaces in parallel. A feedforward neural network is fed into a residual connection and layer normalization layer using two linear transformations and a ReLU activation function to prevent gradient vanishing. This feedforward neural network is used to obtain semantic global features.

[0092] In order to predict brain age, the output of the encoder needs to be converted into a continuous prediction value. Global average pooling is used to compress the sequence representation into a vector of fixed dimension, and then regression is performed through a fully connected layer (or multi-layer perceptron). Perform global average pooling h pool , L is the length of the encoder output: Generate the target person's brain age prediction results through the regression head in is the weight and b is the bias term.

[0093] It should be noted that in the CNN-Transformer model, the attention weights of the semantic local feature sequence are obtained. By analyzing the attention weight matrix, it is possible to determine which tokens in the input sequence contribute the most to the final prediction result. Since there are multiple attention heads and multiple encoder layers, these attention weights need to be aggregated to obtain the overall attention distribution. The CNN-Transformer model has N encoder layers and h attention heads per layer, so all the attention weight matrices can be aggregated. Aggregated into a comprehensive attention weight matrix A total , The total attention weight of each token at all positions is calculated based on the comprehensive attention weight matrix, and the token with the greatest contribution to the target subject's brain age prediction result is identified based on the total attention weight. The contribution calculation formula is: Among them C j represents the contribution of the jth token, and k represents the number of feature items output by the encoder. The brain regions corresponding to the most contributing tokens at different time steps are identified. A time-varying sequence of brain region changes is constructed based on the identified regions. This sequence generates and visualizes the signal conduction pathways of resting-state brain activity.

[0094] Figure 4 A flowchart of the brain age prediction system based on EEG-fMRI knowledge graph enhancement is shown.

[0095] A second embodiment of the present invention provides a brain age prediction system based on EEG-fMRI knowledge graph enhancement, the system comprising a data acquisition module, a feature extraction module, a knowledge graph module, a semantic sequence generation module, a training module, a brain age prediction module, and a path observation module;

[0096] The data acquisition module is responsible for acquiring the EEG data and fMRI data of the target subject and performing preliminary data processing;

[0097] The feature extraction module is responsible for extracting activated brain region sequences from EEG data and fMRI data, identifying and capturing similar subsequences in EEG and fMRI data, and performing alignment processing;

[0098] The knowledge graph module is responsible for constructing and maintaining the EEG-fMRI knowledge graph;

[0099] The semantic sequence generation module is responsible for converting the aligned subsequence pairs into semantic sequences using the EEG-fMRI knowledge graph;

[0100] The training module is responsible for training the CNN-Transformer model and regularly adding new data for retraining to achieve model optimization;

[0101] The brain age prediction module is responsible for using the trained CNN-Transformer model to predict the brain age of the target subject and output the result;

[0102] The path observation module is responsible for visualizing and analyzing the signal conduction path of the target measured person's brain activity.

[0103] A third embodiment of the present invention provides a computer-readable storage medium, which includes a brain age prediction method program based on EEG-fMRI knowledge graph enhancement. When the brain age prediction method program based on EEG-fMRI knowledge graph enhancement is executed by a processor, it implements the steps of the brain age prediction method based on EEG-fMRI knowledge graph enhancement.

[0104] In the several embodiments provided in this application, it should be understood that the disclosed methods can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of the modules is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0105] Those skilled in the art will appreciate that all or part of the steps of the above-mentioned method embodiments may be implemented by hardware associated with program instructions, and the aforementioned program may be stored in a computer-readable storage medium. When the program is executed, the program executes the steps of the above-mentioned method embodiments. The aforementioned storage medium includes various media that can store program codes, such as mobile storage devices, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0106] Alternatively, if the above-mentioned integrated module of the present invention is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0107] The above description is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A brain age prediction method based on EEG-fMRI knowledge graph enhancement, characterized in that: The following steps are involved: Obtain the EEG data and fMRI data of the target subject, and extract the activated brain area sequences from the EEG data and fMRI data respectively; identifying and capturing similar subsequences of EEG data and fMRI data in the activated brain region sequence, and aligning the captured subsequences; Establish an EEG-fMRI knowledge graph based on the functional connectivity of brain regions, and use the EEG-fMRI knowledge graph to fuse and mine the relationships of the aligned subsequence pairs to obtain semantic sequences; The semantic sequence is imported into the CNN-Transformer model to output the brain age prediction result of the target subject, and the signal transmission pathway of the resting brain activity is observed by reversely outputting tokens. Establish an EEG-fMRI knowledge graph based on brain region functional connectivity, specifically: Obtain EEG data, fMRI data, and corresponding brain age detection data from healthy individuals of different ages, construct data samples, extract the sequence of activated brain regions in each data sample, establish the association between EEG data and fMRI data corresponding to the activated brain regions based on the functional connectivity of the brain regions, and initialize the EEG-fMRI knowledge graph; In the EEG-fMRI knowledge graph, the activated brain regions corresponding to EEG data and fMRI data are used as head nodes, the brain region power values ​​are used as node attributes, and the brain age of the data samples is used as the tail node. The entity nodes and attributes are mapped using triples. Through data retrieval methods, we acquire knowledge related to brain regions and brain networks, EEG data, and fMRI data. Through structured processing, we obtain feature vectors of prior knowledge, capture deep features, and construct entities to supplement the EEG-fMRI knowledge graph. Read the semantic features of knowledge related to brain regions and brain networks, EEG data and fMRI data, and obtain the attribution relationship of brain region nodes to brain networks, the functional connectivity relationship between brain regions or between brain networks, and the cross-complementary verification relationship between EEG and fMRI; A relational table is constructed based on the extracted relationships, and the supplemented entities are mapped to the relational table in triple form to complete the EEG-fMRI knowledge graph in a graphical way.

2. The brain age prediction method based on EEG-fMRI knowledge graph enhancement according to claim 1 is characterized in that: Obtain the EEG data and fMRI data of the target subject, and extract the activated brain area sequences from the EEG data and fMRI data respectively, specifically: Obtain the EEG data and fMRI data of the target measurement subject, and determine whether the current brain region is activated at the current timestamp based on the fMRI data. The degree to which the fMRI power value of the current timestamp deviates from the mean fMRI power value of the current brain region during the observation period of the target measurement subject is taken as the fMRI data activation value. At the current moment fMRI data activation value for: ， in Represents an fMRI data sample In the current brain area The normalized fMRI power value time vector, represents the hyperparameter, Indicates the degree of mean deviation after normalization; When judging whether the current brain region is activated at the current timestamp based on the EEG data, the weighted sum of the degree of deviation of the EEG power value of the current timestamp from the mean EEG power value of the current brain region during the observation period of the target measurer and the jitter degree of the EEG power value in the preset time period before the current moment is obtained as the EEG data activation value. At the current moment EEG data activation value for: ， in Represents an EEG data sample In the current brain area The normalized EEG power value time vector, represents the hyperparameter, Indicates the degree of mean deviation after normalization, 、 represents the weight parameter, Indicates the preset time period before the current time The degree of jitter of the internal EEG power value, Indicates the length of the preset time period; A judgment is made based on the activation values ​​of the fMRI data and the EEG data, and an activated brain region sequence corresponding to the fMRI data and the EEG data is extracted.

3. The brain age prediction method based on EEG-fMRI knowledge graph enhancement according to claim 1, characterized in that: Identify and capture similar subsequences of EEG data and fMRI data, and align the captured subsequences. Specifically: Using periodic verification, we identify and capture subsequence pairs in the activation brain region sequences corresponding to fMRI data and EEG data, and define the matching function of the subsequence pairs. : ， in, Represent the fMRI data vector and EEG data vector to be compared, Represents the scale factor, which respectively represents the starting position and the ending position of the subsequence to be compared, Indicates the maximum difference threshold for matching, Indicates the number of subsequence items; When the function value of the matching function is less than or equal to 0, the subsequences are matched, otherwise they are not matched. All similar subsequences of EEG data and fMRI data that meet the conditions are identified and captured to obtain subsequence pairs, and the captured subsequence pairs are aligned.

4. The brain age prediction method based on EEG-fMRI knowledge graph enhancement according to claim 1, characterized in that: The aligned subsequence pairs are fused and their relationships are mined using the EEG-fMRI knowledge graph to obtain semantic sequences. Specifically: Importing the aligned subsequence pairs into the EEG-fMRI knowledge graph for pre-training, determining core entities corresponding to the EEG data representation and the fMRI data representation in the aligned subsequence pairs, inputting the core entities into the EEG-fMRI knowledge graph, and locating them in the EEG-fMRI knowledge graph; Searching for entities whose distance from the core entity meets a preset distance threshold according to the positioning information of the core entity, and obtaining a knowledge graph subgraph including the core entity and all entities and relationships that meet the preset distance threshold; The knowledge graph subgraph prompts the brain region-brain network affiliation relationship, the brain region and brain network functional connection relationship, and the EEG-fMRI cross-validation supplementary relationship therein, and the entities in the knowledge graph subgraph corresponding to the EEG data representation and the fMRI data representation are fused, and the fused entity representation is matched with the extracted relationship to generate semantic prompts, and the semantic sequence of the knowledge graph representation is output.

5. The brain age prediction method based on EEG-fMRI knowledge graph enhancement according to claim 1, characterized in that: The semantic sequence is imported into the CNN-Transformer model to output the brain age prediction result of the target subject, specifically: A CNN-Transformer model is constructed and trained. The semantic sequence is used as the model input. The input semantic sequence is subjected to feature extraction using convolution operations. The extracted features are expanded in number of channels. Each channel is processed independently using depthwise separable convolution to suppress unimportant features. Finally, the processed features are mapped back to the original number of output channels using linear convolution, and a semantic local feature sequence is obtained through CNN. Importing the semantic local feature sequence into the Transformer encoder, converting each local feature into an embedding vector, performing position encoding embedding according to the position of the local feature in the sequence, importing the position-encoded embedding vector into the self-attention layer, and using a multi-head self-attention mechanism to process different local features in parallel; The output of the multi-head attention is added to the embedding vector of the original local feature, and after normalization, it is introduced into the feedforward neural network, where the global feature is obtained through linear transformation and activation; Global average pooling is used to compress the global feature representation into a vector of fixed dimension, and regression is performed through a fully connected layer to obtain the brain age prediction result of the target person being measured.

6. The brain age prediction method based on EEG-fMRI knowledge graph enhancement according to claim 1, characterized in that: By outputting tokens in reverse, we can observe the signal transmission pathway of resting brain activity, specifically: In the CNN-Transformer model, the attention weights of the semantic local feature sequence are obtained, and the attention weights output by all attention heads of all encoder layers are averaged to obtain the comprehensive attention weight matrix; Calculating the total attention weight of each token at all positions according to the comprehensive attention weight matrix, and identifying the token that contributes most to the brain age prediction result of the target subject according to the total attention weight; Based on the brain regions corresponding to the tokens with the largest contribution at different time steps, a sequence of brain region changes over time is constructed to generate the signal conduction pathway of resting-state brain activity and perform visualization.

7. A brain age prediction system based on EEG-fMRI knowledge graph enhancement, characterized by: Implementing the brain age prediction method based on EEG-fMRI knowledge graph enhancement as described in any one of claims 1 to 6, the system includes a data acquisition module, a feature extraction module, a knowledge graph module, a semantic sequence generation module, a training module, a brain age prediction module and a path observation module; The data acquisition module is responsible for acquiring the EEG data and fMRI data of the target subject and performing preliminary data processing; The feature extraction module is responsible for extracting activated brain region sequences from EEG data and fMRI data, identifying and capturing similar subsequences in EEG and fMRI data, and performing alignment processing; The knowledge graph module is responsible for constructing and maintaining the EEG-fMRI knowledge graph; The semantic sequence generation module is responsible for converting the aligned subsequence pairs into semantic sequences using the EEG-fMRI knowledge graph; The training module is responsible for training the CNN-Transformer model and regularly adding new data for retraining to achieve model optimization; The brain age prediction module is responsible for using the trained CNN-Transformer model to predict the brain age of the target subject and output the result; The path observation module is responsible for visualizing and analyzing the signal conduction path of the target measured person's brain activity.

Images