Smell-induced EEG emotion recognition method and system based on space-time stratified attention mechanism

Abstract

The invention relates to the technical field of emotion recognition, in particular to an odor-induced EEG emotion recognition method and system based on a space-time stratified attention mechanism, and the method comprises the steps: presenting odor stimulation to a testee through an olfactory stimulation device, and synchronously collecting an EEG signal and a free association voice text of the testee; performing semantic analysis on the free association voice text, and extracting keywords and corresponding emotion dimension tags; carrying out space-time hierarchical feature extraction on the EEG signal; constructing a space-time hierarchical attention mechanism model; and on the basis of the weighted spatial-temporal features and semantic tags, node connection weights of the individualized knowledge graph are updated through a graph neural network, and an emotion recognition result is generated. According to the method, spatial attention and time attention are combined, key spatio-temporal characteristics are dynamically focused, the limitation of a fixed weight of a traditional method is avoided, and the pertinence and integrity of neural signal representation are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of emotion recognition technology, and in particular to an odor-induced EEG emotion recognition method and system based on a spatiotemporal layered attention mechanism. Background Art

[0002] Smell directly links emotions and memories by activating the limbic system, providing a unique path for non-invasive emotion recognition. Electroencephalogram (EEG) signals have become the core data source for decoding odor-induced emotions due to their high temporal resolution and non-invasiveness. This technology is in urgent need in the fields of mental health diagnosis, intelligent interaction, and neuroscience research. However, odor processing involves the coordination of multiple brain regions such as the olfactory cortex, prefrontal lobe, and hippocampus, as well as the temporal dynamic changes from transient neural responses to steady-state integration, which places high precision requirements on feature extraction and modeling.

[0003] Current odor-induced EEG emotion recognition technology mainly revolves around unimodal analysis and traditional modeling. At the unimodal level, the study focuses on the frequency characteristics or time domain components of multi-focused EEG signals, and directly processes the statistical characteristics of the whole brain or preset brain regions through classifiers such as support vector machines and convolutional neural networks, ignoring the specific neural activity differences in the core brain regions related to olfactory processing. In terms of multimodal fusion, a few studies have attempted to combine the physical parameters of odor with EEG signals, but the semantic information of free association texts has not been fully utilized. Emotional associations are only established through manual keyword annotation or simple word frequency statistics, and the temporal alignment and deep fusion of text semantics and EEG signals have not been achieved. In terms of spatiotemporal modeling, traditional methods use fixed brain region divisions and unified time windows to extract features, which fails to distinguish the stage-by-stage neural mechanisms of odor processing, resulting in the loss of key dynamic information.

[0004] The core limitations of existing solutions are reflected in three aspects: First, the representation of spatiotemporal features is coarse, and specific indicators are not designed for the collaborative mechanism of the olfactory cortex, prefrontal lobe, and hippocampus. The fixed weight fusion of brain regions and temporal features cannot adapt to individual differences in brain region activation patterns and neural response time courses; second, multimodal data are loosely correlated, and semantic parsing relies on simple dictionary matching. The pre-trained language model lacks deep capture of contextual semantics, and the voice timestamps are not accurately aligned with the EEG signals, resulting in a break in the temporal causal relationship of "odor-memory-emotion"; third, the lack of individualized modeling capabilities, and the failure to construct a knowledge graph containing odor molecules, memory fragments, and emotion labels, making it difficult to explicitly model individual-specific associative preferences. In addition, the "black box" model lacks interpretability and cannot reveal the specific contributions of brain regions and semantic cues to emotion classification, limiting the generalization application of the technology in complex scenarios.

[0005] Therefore, there is an urgent need for a method that combines spatiotemporal attention mechanism with knowledge graph to achieve breakthroughs in the accuracy and explainability of odor-induced emotion recognition. Summary of the Invention

[0006] In response to the above-mentioned shortcomings of the prior art, the present invention provides an odor-induced EEG emotion recognition method and system based on a spatiotemporal layered attention mechanism, which can effectively solve the problems mentioned in the prior art.

[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0008] The present invention provides an odor-induced EEG emotion recognition method based on a spatiotemporal layered attention mechanism, comprising the following steps:

[0009] S1. Present odor stimuli to subjects through an olfactory stimulation device, and synchronously collect EEG signals and the subject's free association speech text;

[0010] S2. Perform semantic analysis on the free association speech text, extract keywords and corresponding emotional dimension labels, and construct a personalized smell-memory-emotion triple database;

[0011] S3. performing spatiotemporal hierarchical feature extraction on the EEG signal, comprising:

[0012] Spatial hierarchical extraction: The brain regions are divided based on anatomical divisions, including the olfactory cortex, prefrontal lobe, and hippocampus, and the time-frequency characteristics of each brain region are extracted separately;

[0013] The time-frequency feature extraction includes:

[0014] The power spectral density of alpha waves in the olfactory cortex is calculated using the formula:

[0015]

[0016] Among them, P α is the power spectral density of α wave, is the normalization coefficient, N is the number of sampling points in the frequency band, and X(f) is the Fourier transform amplitude of the brain area signal;

[0017] The phase synchronization of gamma waves in the prefrontal cortex and hippocampus was calculated using the phase locking value, as follows:

[0018]

[0019] Among them, PLV γ is the phase locking value of the γ wave, T is the number of sampling points in the time window, is the γ wave phase of the two-channel EEG signal at time point t;

[0020] Temporal hierarchical extraction: Divide the time period after odor stimulation into the early response stage and the late integration stage, extracting transient features and steady-state features respectively;

[0021] S4. Build a spatiotemporal hierarchical attention mechanism model, including:

[0022] Spatial attention calculation: Dynamically assign weights based on the correlation between semantic keywords and the characteristics of each brain region. The formula is:

[0023]

[0024] in, is the spatial attention weight of the i-th brain region, σ is the Sigmoid activation function, W s 、v s is the trainable parameter matrix, is the trainable parameter matrix v s The transpose of is the feature vector of the i-th brain region, k j is the embedding vector of the jth semantic keyword, For the vector concatenation operation, the feature vector of the i-th brain region is and the embedding vector k of the jth semantic keyword j Splice into a high-dimensional vector;

[0025] Temporal attention calculation: Adaptively integrate the contribution of early and late EEG features to emotion classification. The formula is:

[0026]

[0027] in, is the time attention weight of the t-th time segment, softmax is the normalized exponential function, is the time-layered feature vector, W t 、u t is the trainable parameter matrix, is the trainable parameter matrix u t The transpose of

[0028] S5. Based on weighted spatiotemporal features and semantic labels, the node connection weights of the personalized knowledge graph are updated through a graph neural network to generate emotion recognition results. The nodes of the knowledge graph include odor molecules, memory fragments and emotion labels, and the edge weights represent the strength of the association between the three.

[0029] Furthermore, the semantic analysis in step S2 includes:

[0030] S201, preprocessing the free association speech text, including speech-to-text conversion, word segmentation and stop word filtering, and retaining the timestamp information of the words;

[0031] S202, using a pre-trained language model to encode the cleaned text, generating a word-level context embedding vector, and obtaining numerical labels of emotional valence and arousal through sentiment dictionary matching;

[0032] The sentiment dictionary matching includes:

[0033] For unregistered words, calculate the cosine similarity between their embedding vectors and the words in the sentiment dictionary, and select the average sentiment score of the top N words with the highest similarity as a supplement;

[0034] Aligning the emotional valence and arousal labels with the timestamps of the EEG signals by time windowing;

[0035] S203, extracting high-frequency keywords and mapping them to a psychological emotion dimension space to generate a semantic vector sequence that is time-aligned with the EEG signal, wherein the emotion dimension space includes three-dimensional features of pleasure, arousal, and dominance;

[0036] The extraction of the high-frequency keywords includes:

[0037] We filter words with a frequency ≥ 2 based on TF-IDF weights and remove entity words without sentimental meaning.

[0038] Map the filtered keywords to the PAD emotion space to generate a three-dimensional semantic vector p = (P, A, D), where:

[0039] Pleasure P is normalized to [-1,1] by emotional valence;

[0040] Arousal A is normalized to [0,1] by emotional arousal;

[0041] The dominance D is obtained by calculating the semantic similarity, and the formula is:

[0042]

[0043] Among them, k j is the embedding vector of the jth semantic keyword, w 控制 、w 被动 are the standard vectors of control and passive in the sentiment dictionary respectively.

[0044] Furthermore, the time alignment includes:

[0045] Based on the word timestamps output by speech recognition, the semantic vectors are associated with the corresponding time periods of the EEG signal;

[0046] For time periods without keyword coverage, linear interpolation of adjacent semantic vectors is used to fill in the gaps.

[0047] Furthermore, the extraction of transient features and steady-state features in step S3 includes:

[0048] In the early response stage, the N1 component latency and P2 component amplitude of the event-related potential (ERP) were extracted by the superposition averaging method.

[0049] In the late integration stage, the phase-amplitude coupling index of the θ wave phase and the γ wave amplitude is calculated using Morlet wavelet transform, and the formula is:

[0050]

[0051] Where PAC is the phase amplitude coupling index, N is the number of sampling points in the time window, and A γ (t) is the γ wave amplitude at time point t, is the phase of the θ wave at time point t.

[0052] Furthermore, the step S5 specifically includes:

[0053] S501: Input the weighted spatiotemporal features and semantic vector sequence into the personalized knowledge graph to initialize the graph node features, where:

[0054] Odor node: characterized by the physical and chemical property vector of the odor molecule;

[0055] Memory node: features the semantic vector p = (P, A, D) generated in step S2;

[0056] Emotional nodes: characterized by the one-hot encoding of the emotional dimension label;

[0057] S502. Calculate the connection weights between nodes through the multi-head attention mechanism. The formula is:

[0058] e ij =MLP([h i ;h j ;Δt ij ]);

[0059] Among them, e ij is the connection weight between nodes i and j, MLP is a multi-layer perceptron, which contains at least one hidden layer and ReLU activation function, h i and h j is the node feature vector, Δt ij is the time interval, [h i ;h j ;Δt ij ] is a vector concatenation operation, which combines the node features and time intervals into the input vector;

[0060] S503. Update node features based on the graph attention network. The formula is:

[0061]

[0062] Among them, h' i is the updated feature vector of node i, σ is the activation function, N(i) is the set of neighbor nodes of node i, α ij is the normalized attention coefficient, W is the weight matrix;

[0063] S504: Using the updated emotion node features as output, generate the emotion classification probability through the Softmax function.

[0064] Furthermore, the time interval Δt in step S502 ij The calculation includes:

[0065] If node i is a memory node and node j is an EEG feature node, then Δt ij The difference between the memory description timestamp and the starting point of the EEG time window;

[0066] If nodes i and j are both emotion nodes, then Δt ij =0.

[0067] Furthermore, the normalized attention coefficient α in step S503 ij The calculation formula is:

[0068]

[0069] Among them, a is a trainable parameter vector, LeakyReLU is a linear rectification unit with leakage, [Wh i Wh j ] is the node feature vector after splicing transformation.

[0070] An odor-evoked EEG emotion recognition system based on spatiotemporal hierarchical attention mechanism, including:

[0071] Data acquisition module: used to present odor stimulation to the subject through the olfactory stimulation device, synchronously collect EEG signals and the subject's free association speech text, and retain the timestamp information of the speech text;

[0072] Semantic parsing module: used to perform semantic parsing on the free association speech text, extract keywords and corresponding emotional dimension labels, and build an individualized smell-memory-emotion triple database;

[0073] Spatiotemporal feature extraction module: used for performing spatiotemporal hierarchical feature extraction on the EEG signal;

[0074] Spatiotemporal layered attention mechanism module: used to build a spatiotemporal layered attention mechanism model

[0075] Knowledge graph processing module: This module is used to update the node connection weights of the personalized knowledge graph through a graph neural network based on weighted spatiotemporal features and semantic labels to generate emotion recognition results. The nodes of the knowledge graph include odor molecules, memory fragments, and emotion labels, and the edge weights represent the strength of the association between the three.

[0076] Wherein, the semantic parsing module includes:

[0077] Preprocessing unit: performs speech-to-text conversion, word segmentation, stop word filtering on free association speech text, and retains timestamp information;

[0078] Semantic encoding and sentiment labeling unit: uses a pre-trained language model to generate word-level context embedding vectors, obtains sentiment valence and arousal numerical labels through sentiment dictionary matching, and supplements sentiment scores for unregistered words using cosine similarity;

[0079] Keyword and emotion dimension mapping unit: This unit filters high-frequency keywords based on TF-IDF, maps them to the PAD emotion space, and generates a semantic vector sequence that is time-aligned with the EEG signal.

[0080] Time alignment unit: associates the semantic vector with the corresponding time period of the EEG signal according to the speech timestamp, and uses linear interpolation of adjacent semantic vectors to fill in the time period without keyword coverage;

[0081] The knowledge graph processing module includes:

[0082] Node initialization unit: Odor nodes are characterized by the physical and chemical property vectors of odor molecules, memory nodes are characterized by the semantic vector p = (P, A, D), and emotion nodes are characterized by the one-hot encoding of the emotion dimension label;

[0083] Connection weight calculation unit: The connection weights between nodes are calculated through the multi-head attention mechanism. The formula is

[0084] The connection weights between nodes are calculated through the multi-head attention mechanism. The formula is:

[0085] e ij =MLP([h i ;h j ;Δt ij ]);

[0086] Among them, e ij is the connection weight between nodes i and j, MLP is a multi-layer perceptron, which contains at least one hidden layer and ReLU activation function, h i and h j is the node feature vector, Δt ij is the time interval, [h i ;h j ;Δt ij] is a vector concatenation operation, which combines the node features and time intervals into the input vector;

[0087] Node feature update unit: Updates node features based on the graph attention network. The formula is:

[0088]

[0089] Among them, h' i is the updated feature vector of node i, σ is the activation function, N(i) is the set of neighbor nodes of node i, α ij is the normalized attention coefficient, and W is the weight matrix.

[0090] Furthermore, the spatiotemporal feature extraction module includes:

[0091] Spatial hierarchical extraction unit: divides the brain regions based on anatomical divisions and extracts the time-frequency characteristics of each brain region;

[0092] Temporal hierarchical extraction unit: divides the time period after odor stimulation into early response stage and late integration stage, extracting transient features and steady-state features respectively;

[0093] The spatiotemporal layered attention mechanism module includes:

[0094] Spatial attention calculation unit: dynamically assigns weights based on the correlation between semantic keywords and the characteristics of each brain region;

[0095] Temporal attention computation unit: Adaptively fusing the contribution of early and late EEG features to emotion classification.

[0096] Compared with the prior art, the technical solution provided by the present invention has the following beneficial effects:

[0097] 1. This paper proposes a spatiotemporal hierarchical feature extraction method targeting the specific neural circuits involved in olfactory processing. Spatially, it focuses on the olfactory cortex, prefrontal cortex, and hippocampus to accurately capture activity in core brain regions associated with odor perception, emotion regulation, and memory. Temporally, it divides the early response phase into the late integration phase, covering the entire process from transient physiological reactions to steady-state cognitive processing. Combining spatial and temporal attention, this method dynamically focuses on key spatiotemporal features, avoiding the limitations of traditional fixed weighting methods and significantly improving the pertinence and completeness of neural signal representation.

[0098] 2. This invention generates a three-dimensional semantic vector containing pleasure, arousal, and dominance by preprocessing free-association speech text, pretraining language model encoding, and matching sentiment dictionaries. It then precisely aligns the timestamps with the EEG signal, and uses a linear interpolation strategy to fill in periods without keywords to ensure multimodal data synchronization. This mechanism transforms individual memory cues into quantifiable emotional dimension features, constructing a "smell-memory-emotion" triple database, achieving a deep fusion of language semantics and neural signals. This overcomes the bottlenecks of coarse semantic parsing and broken spatiotemporal associations in existing technologies, providing fine-grained semantic support for individualized emotion modeling.

[0099] 3. Construct a personalized knowledge graph containing odor molecules, memory fragments, and emotion labels. Node features are initialized using odor physical and chemical properties, semantic vectors, and emotion labels. A multi-head attention mechanism is used to calculate the connection weights between nodes and dynamically update the strength of the "odor-memory-emotion" association. The graph neural network model explicitly models individual-specific neural-semantic associations. Compared to traditional "black box" models, it can clearly reveal the contribution of brain region coordination and semantic cues to emotion classification. Furthermore, adaptive weight updates adapt to individual differences in brain activation patterns and memory recall, improving recognition accuracy while enhancing the model's interpretability and generalization capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0100] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive effort.

[0101] Figure 1 Schematic diagram of the method flow of the present invention;

[0102] Figure 2 Schematic diagram of the system structure of the present invention. DETAILED DESCRIPTION

[0103] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0104] The present invention will be further described below with reference to the embodiments.

[0105] Example 1: Reference Figure 1, an odor-induced EEG emotion recognition method based on spatiotemporal hierarchical attention mechanism, comprising the following steps:

[0106] S1. Present odor stimuli to subjects through an olfactory stimulation device, and synchronously collect EEG signals and the subject's free association speech text;

[0107] S2. Perform semantic analysis on the free association speech text, extract keywords and corresponding emotional dimension labels, and build a personalized smell-memory-emotion triple database;

[0108] S3. Extract spatiotemporal features from EEG signals, including:

[0109] Spatial hierarchical extraction: The brain regions are divided based on anatomical divisions, including the olfactory cortex, prefrontal lobe, and hippocampus, and the time-frequency characteristics of each brain region are extracted separately;

[0110] Among them, time-frequency feature extraction includes:

[0111] The power spectral density of alpha waves in the olfactory cortex is calculated using the formula:

[0112]

[0113] Among them, P α is the power spectral density of α wave, is the normalization coefficient, N is the number of sampling points in the frequency band, and X(f) is the Fourier transform amplitude of the brain area signal;

[0114] The phase synchronization of gamma waves in the prefrontal cortex and hippocampus was calculated using the phase locking value, as follows:

[0115]

[0116] Among them, PLV γ is the phase locking value of the γ wave, T is the number of sampling points in the time window, is the γ wave phase of the two-channel EEG signal at time point t;

[0117] Temporal hierarchical extraction: Divide the time period after odor stimulation into the early response stage and the late integration stage, extracting transient features and steady-state features respectively;

[0118] S4. Build a spatiotemporal hierarchical attention mechanism model, including:

[0119] Spatial attention calculation: Dynamically assign weights based on the correlation between semantic keywords and the characteristics of each brain region. The formula is:

[0120]

[0121] in, is the spatial attention weight of the i-th brain region, σ is the Sigmoid activation function, W s 、v s is the trainable parameter matrix, is the trainable parameter matrix v s The transpose of is the feature vector of the i-th brain region, k j is the embedding vector of the jth semantic keyword, For the vector concatenation operation, the feature vector of the i-th brain region is and the embedding vector k of the jth semantic keyword j Splice into a high-dimensional vector;

[0122] Temporal attention calculation: Adaptively integrate the contribution of early and late EEG features to emotion classification. The formula is:

[0123]

[0124] in, is the time attention weight of the t-th time segment, softmax is the normalized exponential function, is the time-layered feature vector, W t 、u t is the trainable parameter matrix, is the trainable parameter matrix u t The transpose of

[0125] S5. Based on weighted spatiotemporal features and semantic labels, the node connection weights of the personalized knowledge graph are updated through the graph neural network to generate emotion recognition results. The nodes of the knowledge graph include odor molecules, memory fragments and emotion labels, and the edge weights represent the strength of the association between the three.

[0126] Furthermore, the semantic analysis in step S2 includes:

[0127] S201, preprocessing the free association speech text, including speech-to-text conversion, word segmentation and stop word filtering, and retaining the timestamp information of the words;

[0128] S202, using a pre-trained language model to encode the cleaned text, generating a word-level context embedding vector, and obtaining numerical labels of emotional valence and arousal through sentiment dictionary matching;

[0129] Sentiment dictionary matching includes:

[0130] For unregistered words, calculate the cosine similarity between their embedding vectors and the words in the sentiment dictionary, and select the average sentiment score of the top N words with the highest similarity as a supplement;

[0131] Align the emotional valence and arousal labels with the timestamps of the EEG signals through time window division;

[0132] S203, extracting high-frequency keywords and mapping them to a psychological emotion dimension space to generate a semantic vector sequence that is time-aligned with the EEG signal, where the emotion dimension space includes three-dimensional features of pleasure, arousal, and dominance;

[0133] The extraction of high-frequency keywords includes:

[0134] We filter words with a frequency ≥ 2 based on TF-IDF weights and remove entity words without sentimental meaning.

[0135] Map the filtered keywords to the PAD emotion space to generate a three-dimensional semantic vector p = (P, A, D), where:

[0136] Pleasure P is normalized to [-1,1] by emotional valence;

[0137] Arousal A is normalized to [0,1] by emotional arousal;

[0138] The dominance D is obtained by calculating the semantic similarity, and the formula is:

[0139]

[0140] Among them, k j is the embedding vector of the jth semantic keyword, w 控制 、w 被动 are the standard vectors of control and passive in the sentiment dictionary respectively.

[0141] Furthermore, the time alignment includes:

[0142] Based on the word timestamps output by speech recognition, the semantic vectors are associated with the corresponding time periods of the EEG signal;

[0143] For time periods without keyword coverage, linear interpolation of adjacent semantic vectors is used to fill in the gaps.

[0144] Furthermore, the extraction of transient features and steady-state features in step S3 includes:

[0145] In the early response stage, the N1 component latency and P2 component amplitude of the event-related potential (ERP) were extracted by the superposition averaging method.

[0146] Late integration stage: Morlet wavelet transform is used to calculate the phase-amplitude coupling index of theta wave phase and gamma wave amplitude. The formula is:

[0147]

[0148] Where PAC is the phase amplitude coupling index, N is the number of sampling points in the time window, and A γ (t) is the γ wave amplitude at time point t, is the phase of the θ wave at time point t.

[0149] Furthermore, step S5 specifically includes:

[0150] S501: Input the weighted spatiotemporal features and semantic vector sequence into the personalized knowledge graph to initialize the graph node features, where:

[0151] Odor node: characterized by the physical and chemical property vector of the odor molecule;

[0152] Memory node: features the semantic vector p = (P, A, D) generated in step S2;

[0153] Emotional nodes: characterized by the one-hot encoding of the emotional dimension label;

[0154] S502. Calculate the connection weights between nodes through the multi-head attention mechanism. The formula is:

[0155] e ij =MLP([h i ;h j ;Δt ij ]);

[0156] Among them, e ij is the connection weight between nodes i and j, MLP is a multi-layer perceptron, which contains at least one hidden layer and ReLU activation function, h i and h j is the node feature vector, Δt ij is the time interval, [h i ;h j ;Δt ij ] is a vector concatenation operation, which combines the node features and time intervals into the input vector;

[0157] S503. Update node features based on the graph attention network. The formula is:

[0158]

[0159] Among them, h' i is the updated feature vector of node i, σ is the activation function, N(i) is the set of neighbor nodes of node i, α ij is the normalized attention coefficient, W is the weight matrix;

[0160] S504: Using the updated emotion node features as output, generate the emotion classification probability through the Softmax function.

[0161] Furthermore, in step S502, the time interval Δt ij The calculation includes:

[0162] If node i is a memory node and node j is an EEG feature node, then Δt ij The difference between the memory description timestamp and the starting point of the EEG time window;

[0163] If nodes i and j are both emotion nodes, then Δt ij =0.

[0164] Furthermore, in step S503, the normalized attention coefficient α ij The calculation formula is:

[0165]

[0166] Among them, a is a trainable parameter vector, LeakyReLU is a linear rectification unit with leakage, [Wh i Wh j ] is the node feature vector after splicing transformation.

[0167] In a specific example, 20 healthy subjects aged 18-25 were selected to conduct an emotion recognition experiment induced by the scent of lavender. The experimental equipment included a controllable concentration olfactory stimulation device, a 64-channel EEG acquisition system, and high-precision speech recording equipment. The EEG acquisition system had a sampling rate of 1000 Hz, with leads covering the O1 / O2 regions of the olfactory cortex, the F3 / F4 regions of the prefrontal cortex, and the T3 / T4 regions of the hippocampus. The speech recording equipment had a timestamp accuracy of ±50 ms. During the experiment, subjects were presented with a 50 ppm concentration of lavender for 5 seconds, and EEG signals and free association speech text were simultaneously collected. After smelling the scent, subjects spontaneously described their associations, for example, "Smelling lavender reminds me of my bedroom, feeling at ease and wanting to sleep." The speech processing system recorded the start and end times of each word in real time. For example, the timestamp for the word "peace of mind" was recorded as 3.2-3.8 seconds.

[0168] First, the speech text is preprocessed and converted to text using speech-to-text technology. After word segmentation and stop word filtering, the emotionally significant keywords "bedroom," "peace of mind," and "sleep" are retained, along with their timestamp information. The cleaned text is encoded using the pretrained language model BERT to generate word embedding vectors that incorporate contextual semantics. The vector representation of "peace of mind" captures its emotional tendencies in a specific context. Using sentiment lexicon matching technology, the emotional valence and arousal numerical labels of the keywords are obtained. For personalized words not included in the lexicon, such as "bedroom," the cosine similarity between their embedding vectors and similar words in the lexicon, such as "home," is calculated. The average sentiment score of the top N most similar words is taken as a supplement to ensure the integrity of the sentiment annotation.

[0169] The selected high-frequency keywords were then mapped to the PAD emotion space, commonly used in psychology. Pleasure was normalized to the [-1, 1] range using emotional valence, arousal to the [0, 1] range using emotional arousal, and dominance was calculated by calculating half the difference in semantic similarity between the keyword and the standard vectors for "control" and "passive." This metric reflects an individual's subjective sense of control over their emotions. The dominance of "peace of mind" was calculated by comparing its semantic distance to "control" and "passive." Ultimately, a three-dimensional sequence of semantic vectors was generated, aligned with the EEG signal timestamps. Time periods not covered by the keywords were filled in using linear interpolation of adjacent semantic vectors to ensure temporal synchronization of the multimodal data.

[0170] In the spatial dimension, specific time-frequency features were extracted from the core brain areas related to olfactory processing: the power spectral density of alpha waves in the 8-12 Hz frequency band was calculated for the O1 / O2 leads in the olfactory cortex area, which reflects the neural energy distribution during the olfactory perception stage; the phase locking value of gamma waves in the 30-80 Hz frequency band was calculated for the F3 / F4 areas of the prefrontal lobe and the T3 / T4 areas of the hippocampus, and the strength of neural functional connections in the process of associating emotion regulation with memory was captured by quantifying the phase synchronization of the two-channel signals.

[0171] In the time dimension, the differential features were extracted by dividing the early response stage of 0.5-1.5 seconds and the late integration stage of 3-5 seconds: in the early stage, the N1 component latency and P2 component amplitude of the event-related potential were extracted by the superposition averaging method. The N1 component latency reflects the neural response time of primary perception, and the P2 component amplitude reflects the neural activity intensity of attention allocation; in the late stage, Morlet wavelet transform was used to calculate the phase-amplitude coupling index of the 4-8Hz phase of the θ wave and the amplitude of the γ wave. This index characterizes the interaction of oscillation signals of different frequencies and reflects the neural integration process in high-level cognitive processing.

[0172] The spatiotemporal layered attention mechanism dynamically integrates multi-dimensional features: the spatial attention module concatenates the feature vectors of each brain region with the semantic keyword embedding vector, and calculates the weight of each brain region for specific keywords through a trainable parameter matrix and a Sigmoid activation function. For example, the keyword "peace of mind" will increase the weight of the hippocampus region, indicating the importance of memory association; the temporal attention module adaptively distributes the contribution of early and late features through the Softmax function. In the recognition of relaxed emotions, the weight of the integrated features in the middle and late stages is usually higher, achieving a focus on key spatiotemporal information.

[0173] The personalized knowledge graph uses odor molecules, memory fragments, and emotion labels as nodes, and the initial features are set as the odor physical and chemical property vector, three-dimensional semantic vector, and one-hot encoded emotion label. The connection weights between nodes are calculated through a multi-head attention mechanism. This process integrates node features and time interval information, such as the difference between the memory description time and the EEG time window, and dynamically updates the "odor-memory-emotion" association strength. The graph neural network updates the emotion node representation by aggregating the features of neighboring nodes, and finally outputs the emotion classification probability through the Softmax function. In this embodiment, the association path of "lavender smell → peace of mind memory → relaxing emotion" is explicitly modeled through the knowledge graph, with an identification accuracy of 92%, which is significantly higher than the 78% of the traditional single-modal method. The key nodes and connections can be visualized through the graph to clearly explain the neural-semantic association mechanism of "enhanced hippocampal gamma wave synchronization and peace of mind memory jointly driving the judgment of relaxing emotions."

[0174] This embodiment achieves accurate decoding from odor stimulation to emotional labels through spatiotemporal alignment of multimodal data, hierarchical feature extraction, and personalized knowledge graph modeling. Experimental results show that the spatiotemporal hierarchical attention mechanism effectively captures the stage-by-stage neural dynamics and brain region coordination patterns of olfactory processing, the semantic parsing module successfully converts individual memory clues into quantifiable emotional dimension features, and the knowledge graph technology explicitly models the individual's unique odor-emotion association path, providing interpretable support for emotion recognition. This complete process verifies the effectiveness and innovation of the present invention in odor-induced emotion recognition, and lays a practical foundation for the application of related technologies in the fields of mental health, intelligent interaction, etc.

[0175] Example 2: Reference Figure 2 , an odor-evoked EEG emotion recognition system based on spatiotemporal hierarchical attention mechanism, including:

[0176] Data acquisition module: used to present odor stimulation to the subject through the olfactory stimulation device, synchronously collect EEG signals and the subject's free association speech text, and retain the timestamp information of the speech text;

[0177] Semantic parsing module: used to perform semantic parsing on free association speech text, extract keywords and corresponding emotional dimension labels, and build a personalized smell-memory-emotion triple database;

[0178] Spatiotemporal feature extraction module: used to extract spatiotemporal hierarchical features of EEG signals;

[0179] Spatiotemporal layered attention mechanism module: used to build a spatiotemporal layered attention mechanism model

[0180] Knowledge graph processing module: This module updates the node connection weights of the personalized knowledge graph through a graph neural network based on weighted spatiotemporal features and semantic labels to generate emotion recognition results. The nodes of the knowledge graph include odor molecules, memory fragments, and emotion labels, and the edge weights represent the strength of the association between the three.

[0181] Among them, the semantic parsing module includes:

[0182] Preprocessing unit: performs speech-to-text conversion, word segmentation, stop word filtering on free association speech text, and retains timestamp information;

[0183] Semantic encoding and sentiment labeling unit: uses a pre-trained language model to generate word-level context embedding vectors, obtains sentiment valence and arousal numerical labels through sentiment dictionary matching, and supplements sentiment scores for unregistered words using cosine similarity;

[0184] Keyword and emotion dimension mapping unit: This unit filters high-frequency keywords based on TF-IDF, maps them to the PAD emotion space, and generates a semantic vector sequence that is time-aligned with the EEG signal.

[0185] Time alignment unit: associates the semantic vector with the corresponding time period of the EEG signal according to the speech timestamp, and uses linear interpolation of adjacent semantic vectors to fill in the time period without keyword coverage;

[0186] The knowledge graph processing module includes:

[0187] Node initialization unit: Odor nodes are characterized by the physical and chemical property vectors of odor molecules, memory nodes are characterized by the semantic vector p = (P, A, D), and emotion nodes are characterized by the one-hot encoding of the emotion dimension label;

[0188] Connection weight calculation unit: The connection weights between nodes are calculated through the multi-head attention mechanism. The formula is

[0189] The connection weights between nodes are calculated through the multi-head attention mechanism. The formula is:

[0190] e ij =MLP([h i ;h j ;Δt ij ]);

[0191] Among them, e ij is the connection weight between nodes i and j, MLP is a multi-layer perceptron, which contains at least one hidden layer and ReLU activation function, h i and h j is the node feature vector, Δt ij is the time interval, [h i ;h j ;Δt ij ] is a vector concatenation operation, which combines the node features and time intervals into the input vector;

[0192] Node feature update unit: Updates node features based on the graph attention network. The formula is:

[0193]

[0194] Among them, h' i is the updated feature vector of node i, σ is the activation function, N(i) is the set of neighbor nodes of node i, α ij is the normalized attention coefficient, and W is the weight matrix.

[0195] Furthermore, the spatiotemporal feature extraction module includes:

[0196] Spatial hierarchical extraction unit: divides the brain regions based on anatomical divisions and extracts the time-frequency characteristics of each brain region;

[0197] Temporal hierarchical extraction unit: divides the time period after odor stimulation into early response stage and late integration stage, extracting transient features and steady-state features respectively;

[0198] The spatiotemporal layered attention mechanism module includes:

[0199] Spatial attention calculation unit: dynamically assigns weights based on the correlation between semantic keywords and the characteristics of each brain region;

[0200] Temporal attention computation unit: Adaptively fusing the contribution of early and late EEG features to emotion classification.

[0201] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the various embodiments of the present invention.

Claims

1. An odor-induced EEG emotion recognition method based on spatiotemporal layered attention mechanism, characterized by: The following steps are involved: S1. Present odor stimuli to subjects through an olfactory stimulation device, and synchronously collect EEG signals and the subject's free association speech text; S2. Perform semantic analysis on the free association speech text, extract keywords and corresponding emotional dimension labels, and construct a personalized smell-memory-emotion triple database; S3. performing spatiotemporal hierarchical feature extraction on the EEG signal, comprising: Spatial hierarchical extraction: The brain regions are divided based on anatomical divisions, including the olfactory cortex, prefrontal lobe, and hippocampus, and the time-frequency characteristics of each brain region are extracted separately; The time-frequency feature extraction includes: The power spectral density of alpha waves in the olfactory cortex is calculated using the formula: Among them, P α is the power spectral density of α wave, is the normalization coefficient, N is the number of sampling points in the frequency band, and X(f) is the Fourier transform amplitude of the brain area signal; The phase synchronization of gamma waves in the prefrontal cortex and hippocampus was calculated using the phase locking value, as follows: Among them, PLV γ is the phase locking value of the γ wave, T is the number of sampling points in the time window, is the γ wave phase of the two-channel EEG signal at time point t; Temporal hierarchical extraction: Divide the time period after odor stimulation into the early response stage and the late integration stage, extracting transient features and steady-state features respectively; S4. Build a spatiotemporal hierarchical attention mechanism model, including: Spatial attention calculation: Dynamically assign weights based on the correlation between semantic keywords and the characteristics of each brain region. The formula is: in, is the spatial attention weight of the i-th brain region, σ is the Sigmoid activation function, W s 、v s is the trainable parameter matrix, is the trainable parameter matrix v s The transpose of is the feature vector of the i-th brain region, k j is the embedding vector of the jth semantic keyword, For the vector concatenation operation, the feature vector of the i-th brain region is and the embedding vector k of the jth semantic keyword j Splice into a high-dimensional vector; Temporal attention calculation: Adaptively integrate the contribution of early and late EEG features to emotion classification. The formula is: in, is the time attention weight of the t-th time segment, softmax is the normalized exponential function, is the time-layered feature vector, W t 、u t is the trainable parameter matrix, is the trainable parameter matrix u t The transpose of S5. Based on weighted spatiotemporal features and semantic labels, the node connection weights of the personalized knowledge graph are updated through a graph neural network to generate emotion recognition results. The nodes of the knowledge graph include odor molecules, memory fragments and emotion labels, and the edge weights represent the strength of the association between the three.

2. The odor-induced EEG emotion recognition method based on spatiotemporal layered attention mechanism according to claim 1 is characterized in that: The semantic analysis in step S2 includes: S201, preprocessing the free association speech text, including speech-to-text conversion, word segmentation and stop word filtering, and retaining the timestamp information of the words; S202, using a pre-trained language model to encode the cleaned text, generating a word-level context embedding vector, and obtaining numerical labels of emotional valence and arousal through sentiment dictionary matching; The sentiment dictionary matching includes: For unregistered words, calculate the cosine similarity between their embedding vectors and the words in the sentiment dictionary, and select the average sentiment score of the top N words with the highest similarity as a supplement; Aligning the emotional valence and arousal labels with the timestamps of the EEG signals by time windowing; S203, extracting high-frequency keywords and mapping them to a psychological emotion dimension space to generate a semantic vector sequence that is time-aligned with the EEG signal, wherein the emotion dimension space includes three-dimensional features of pleasure, arousal, and dominance; The extraction of the high-frequency keywords includes: We filter words with a frequency ≥ 2 based on TF-IDF weights and remove entity words without sentimental meaning. Map the filtered keywords to the PAD emotion space to generate a three-dimensional semantic vector p = (P, A, D), where: Pleasure P is normalized to [-1,1] by emotional valence; Arousal A is normalized to [0,1] by emotional arousal; The dominance D is obtained by calculating the semantic similarity, and the formula is: Among them, k j is the embedding vector of the jth semantic keyword, w 控制 、w 被动 are the standard vectors of control and passive in the sentiment dictionary respectively.

3. The odor-induced EEG emotion recognition method based on spatiotemporal layered attention mechanism according to claim 2 is characterized in that: The time alignment includes: Based on the word timestamps output by speech recognition, the semantic vectors are associated with the corresponding time periods of the EEG signal; For time periods without keyword coverage, linear interpolation of adjacent semantic vectors is used to fill in the gaps.

4. The odor-induced EEG emotion recognition method based on spatiotemporal layered attention mechanism according to claim 1 is characterized in that: The transient feature and steady-state feature extraction in step S3 includes: In the early response stage, the N1 component latency and P2 component amplitude of the event-related potential (ERP) were extracted by the superposition averaging method. In the late integration stage, the phase-amplitude coupling index of the θ wave phase and the γ wave amplitude is calculated using Morlet wavelet transform, and the formula is: Where PAC is the phase amplitude coupling index, N is the number of sampling points in the time window, and A γ (t) is the γ wave amplitude at time point t, is the phase of the θ wave at time point t.

5. The odor-induced EEG emotion recognition method based on spatiotemporal layered attention mechanism according to claim 1 is characterized in that: The step S5 specifically includes: S501: Input the weighted spatiotemporal features and semantic vector sequence into the personalized knowledge graph to initialize the graph node features, where: Odor node: characterized by the physical and chemical property vector of the odor molecule; Memory node: features the semantic vector p = (P, A, D) generated in step S2; Emotional nodes: characterized by the one-hot encoding of the emotional dimension label; S502. Calculate the connection weights between nodes through the multi-head attention mechanism. The formula is: e ij =MLP([h i ;h j ;Δt ij ]); Among them, e ij is the connection weight between nodes i and j, MLP is a multi-layer perceptron, which contains at least one hidden layer and ReLU activation function, h i and h j is the node feature vector, Δt ij is the time interval, [h i ;h j ;Δt ij ] is a vector concatenation operation, which combines the node features and time intervals into the input vector; S503. Update node features based on the graph attention network. The formula is: Among them, h' i is the updated feature vector of node i, σ is the activation function, N(i) is the set of neighbor nodes of node i, α ij is the normalized attention coefficient, W is the weight matrix; S504: Using the updated emotion node features as output, generate the emotion classification probability through the Softmax function.

6. The odor-induced EEG emotion recognition method based on spatiotemporal layered attention mechanism according to claim 5 is characterized in that: The time interval Δt in step S502 ij The calculation includes: If node i is a memory node and node j is an EEG feature node, then Δt ij The difference between the memory description timestamp and the starting point of the EEG time window; If nodes i and j are both emotion nodes, then Δt ij =0.

7. The odor-induced EEG emotion recognition method based on spatiotemporal layered attention mechanism according to claim 5 is characterized in that: The normalized attention coefficient α in step S503 ij The calculation formula is: Among them, a is a trainable parameter vector, LeakyReLU is a linear rectification unit with leakage, [Wh i Wh j ] is the node feature vector after splicing transformation.

8. An odor-induced EEG emotion recognition system based on spatiotemporal layered attention mechanism, characterized by: include: Data acquisition module: used to present odor stimulation to the subject through the olfactory stimulation device, synchronously collect EEG signals and the subject's free association speech text, and retain the timestamp information of the speech text; Semantic parsing module: used to perform semantic parsing on the free association speech text, extract keywords and corresponding emotional dimension labels, and build an individualized smell-memory-emotion triple database; Spatiotemporal feature extraction module: used for performing spatiotemporal hierarchical feature extraction on the EEG signal; Spatiotemporal layered attention mechanism module: used to build a spatiotemporal layered attention mechanism model Knowledge graph processing module: This module is used to update the node connection weights of the personalized knowledge graph through a graph neural network based on weighted spatiotemporal features and semantic labels to generate emotion recognition results. The nodes of the knowledge graph include odor molecules, memory fragments, and emotion labels, and the edge weights represent the strength of the association between the three. Wherein, the semantic parsing module includes: Preprocessing unit: performs speech-to-text conversion, word segmentation, stop word filtering on free association speech text, and retains timestamp information; Semantic encoding and sentiment labeling unit: uses a pre-trained language model to generate word-level context embedding vectors, obtains sentiment valence and arousal numerical labels through sentiment dictionary matching, and supplements sentiment scores for unregistered words using cosine similarity; Keyword and emotion dimension mapping unit: This unit filters high-frequency keywords based on TF-IDF, maps them to the PAD emotion space, and generates a semantic vector sequence that is time-aligned with the EEG signal. Time alignment unit: associates the semantic vector with the corresponding time period of the EEG signal according to the speech timestamp, and uses linear interpolation of adjacent semantic vectors to fill in the time period without keyword coverage; The knowledge graph processing module includes: Node initialization unit: Odor nodes are characterized by the physical and chemical property vectors of odor molecules, memory nodes are characterized by the semantic vector p = (P, A, D), and emotion nodes are characterized by the one-hot encoding of the emotion dimension label; Connection weight calculation unit: The connection weights between nodes are calculated through the multi-head attention mechanism. The formula is The connection weights between nodes are calculated through the multi-head attention mechanism. The formula is: e ij =MLP([h i ;h j ;Δt ij ]); Among them, e ij is the connection weight between nodes i and j, MLP is a multi-layer perceptron, which contains at least one hidden layer and ReLU activation function, h i and h j is the node feature vector, Δt ij is the time interval, [h i ;h j ;Δt ij ] is a vector concatenation operation, which combines the node features and time intervals into the input vector; Node feature update unit: Updates node features based on the graph attention network. The formula is: Among them, h' i is the updated feature vector of node i, σ is the activation function, N(i) is the set of neighbor nodes of node i, α ij is the normalized attention coefficient, and W is the weight matrix.

9. An odor-induced EEG emotion recognition system based on spatiotemporal layered attention mechanism according to claim 8, characterized in that: The spatiotemporal feature extraction module includes: Spatial hierarchical extraction unit: divides the brain regions based on anatomical divisions and extracts the time-frequency characteristics of each brain region; Temporal hierarchical extraction unit: divides the time period after odor stimulation into early response stage and late integration stage, extracting transient features and steady-state features respectively; The spatiotemporal layered attention mechanism module includes: Spatial attention calculation unit: dynamically assigns weights based on the correlation between semantic keywords and the characteristics of each brain region; Temporal attention computation unit: Adaptively fusing the contribution of early and late EEG features to emotion classification.

Images