Cross-subject electroencephalogram emotion recognition model training method, model training system and emotion recognition method based on space interaction characteristics

Abstract

The invention discloses a cross-subject electroencephalogram emotion recognition model training method, a cross-subject electroencephalogram emotion recognition model training system and a cross-subject electroencephalogram emotion recognition method based on spatial interaction characteristics. The model training method comprises the step of providing an electroencephalogram signal emotion recognition method in combination with differential warm-up learning (DWL), a high and low frequency window (HFLFW) and a space interactive Colmogov-Arnod network (SIK). Firstly, electroencephalogram signals are converted into electroencephalogram feature maps through three-dimensional mapping, then a model is warmed up before training through differential warm-up learning, and the ability of distinguishing physiological differences of subjects is learned in advance. And processing local refinement and global structure features of the electroencephalogram feature map through high and low frequency windows respectively. And finally, capturing spatial interaction characteristics of electroencephalogram signals of different channels among different testees through a spatial interaction KAN so as to enhance the modeling ability of the model for different electroencephalogram modes.

Description

Technical Field

[0001] The present invention belongs to the field of brain-computer interface and machine learning, and more specifically, to a cross-subject EEG signal emotion recognition model training method based on spatial interaction features, a model training system and an emotion recognition method. Background Art

[0002] Emotions, as the basis of human interaction with the world, affect our decision-making, behavioral activities and interpersonal relationships. In the field of medical monitoring, emotion recognition helps to detect negative emotions in a timely manner, thereby reducing the risk of mental illness caused by excessive exposure to negative emotions. In smart home systems, combining IoT technology and EEG emotion recognition, future smart home systems will be able to automatically adjust environmental settings (such as light brightness, music playlists, etc.) according to the emotional state of the residents to create a more comfortable living atmosphere. In the field of brain-computer interfaces, emotion recognition can help computers better understand the user's current emotional state and provide relevant biofeedback in a timely manner. Therefore, accurately identifying and understanding the user's emotional state is crucial to the development of more intelligent and personalized interactive systems.

[0003] Generally speaking, emotions are mainly identified through physiological and non-physiological signals. Non-physiological signals such as voice intonation, body posture and facial expression can be controlled and hidden, which often lead to classifier diagnosis errors subjectively. Therefore, we often use physiological signals such as electrocardiogram (ECG) and electroencephalogram (EEG), which can reflect emotions more objectively and accurately. With the continuous development of brain science and neuroengineering, and the growing demand for natural human-computer interaction, EEG emotion recognition technology has gradually become a research hotspot. However, most of the current emotion recognition research on EEG signals faces a common challenge-the cross-subject problem. That is, the same model is difficult to be applied to emotion classification tasks between different individuals at the same time, because the physiological differences between different people lead to different EEG patterns. Taking traditional machine learning methods as an example, they first pre-process the EEG signals, such as downsampling and bandpass filtering. Then, specific types of noise are removed by artifact processing technology. Subsequently, artificial feature extraction methods are used to extract features from the time domain, frequency domain, time-frequency domain and spatial domain. This method is not only complicated and time-consuming, but also manual feature extraction often cannot fully capture the nonlinear high-dimensional characteristics of EEG signals, causing the model to perform poorly in cross-subject recognition tasks.

[0004] Since the EEG data set has a small amount of data and there are physiological differences between individuals, existing methods lack special designs for processing individual physiological differences on small sample data sets, which further exacerbates the difficulty of emotion recognition effects on cross-subject tasks. At the same time, existing methods mostly use time domain or frequency domain information in EEG signals without considering the spatial frequency domain, which limits the model's understanding of complex EEG detail changes. EEG signals are often collected using multiple electrode channels, and existing methods only extract features within a single channel without considering the spatial interaction characteristics of different electrode channels. These limiting factors work together to cause emotion recognition technology to face many challenges in practical applications, especially in terms of the accuracy of emotion recognition across individuals.

[0005] The invention patent with the prior art publication number CN117150397A proposes a cross-subject EEG emotion recognition method and system based on multi-branch sample selection. By building a multi-branch network framework, each subject is regarded as a source domain, and then each source domain and target domain are aligned in pairs, so that the target domain can learn useful features from different source domains. A sample selection module is used to align each source domain and target domain in pairs. The module uses the maximum cosine similarity method to select the sample closest to the target domain sample from each source domain, and continuously optimizes with the iteration of the model. This method only extracts features within a single channel and does not consider the spatial interaction characteristics of different electrode channels. Summary of the invention

[0006] In order to overcome the problems faced by the electroencephalogram (EEG) emotion recognition methods in the prior art, especially the insufficient accuracy of cross-subject emotion recognition and the problem of neglecting the spatial interaction information between different electrode channels, the present invention provides a cross-subject EEG signal emotion recognition model training method based on spatial interaction features, a model training system and an emotion recognition method.

[0007] The primary purpose of the present invention is to solve the above technical problems. The technical solution of the present invention is as follows:

[0008] The first aspect of the present invention provides a method for training a cross-subject EEG signal emotion recognition model based on spatial interaction features, comprising the following steps:

[0009] Extracting a three-dimensional mapping matrix of the electroencephalogram signal data, and performing random masking on the three-dimensional mapping matrix to obtain a masked three-dimensional mapping matrix;

[0010] Perform differential warm-up training, input the three-dimensional mapping matrix and the mask three-dimensional mapping matrix into an emotion recognition model that combines a high-frequency window module, a low-frequency window module, and a spatial interaction neural network module, output respective spatial interaction features, and update some module parameters in the emotion recognition model through a loss function to obtain a warm-up emotion recognition model;

[0011] Formal training is carried out, the three-dimensional mapping matrix is ​​input into the warm-up emotion recognition model, the emotion category result prediction value is output, and the parameters of the entire model are updated through the loss function to obtain the trained emotion recognition model.

[0012] Furthermore, the extraction of the three-dimensional mapping matrix of the EEG signal data comprises the following steps:

[0013] Using the time window segmentation method, the input EEG signal data is segmented according to the window length of T seconds to obtain the emotional EEG signals in multiple time windows;

[0014] The baseline EEG signal is removed from the emotional EEG signal to obtain a de-baselined emotional EEG signal;

[0015] Decompose the baseline-free emotional EEG signal into n sub-band signals using a preset filter;

[0016] The differential entropy is calculated for each sub-band signal to obtain a matrix containing the differential entropy characteristics of each frequency band;

[0017] The differential entropy features of each frequency band are placed into a blank matrix according to the corresponding electrode channel position to obtain a three-dimensional mapping matrix.

[0018] Furthermore, the preset filter is a Butterworth filter.

[0019] Furthermore, the emotion recognition model also includes a feature fusion module and a fully connected layer module; the method for processing a three-dimensional mapping matrix using the emotion recognition model includes the following steps:

[0020] Linearly map the three-dimensional mapping matrix and add position encoding in high and wide dimensions;

[0021] Divide the three-dimensional mapping matrix after adding the position code into a plurality of sub-blocks;

[0022] Input the sub-block into the high-frequency window module, perform feature extraction using local multi-head self-attention, and output high-frequency features; input the sub-block into the low-frequency window module, perform average pooling in the sub-block to obtain n pixels, perform feature extraction using cross-attention operation, and output low-frequency features;

[0023] Input the high-frequency features and low-frequency features into the feature fusion module for feature concatenation and linear mapping to obtain fused features;

[0024] Using a batch splicing method to splice multiple fused features to obtain a horizontal tensor unit and a vertical tensor unit, and adding position encoding information to the horizontal tensor unit and the vertical tensor unit to enhance the position feature;

[0025] The horizontal tensor unit and the vertical tensor unit are spliced ​​to obtain a combined tensor containing rich spatial interaction features;

[0026] The combined tensor is input into the spatial interaction neural network module to extract the spatial interaction features between the multi-channel EEG signals, and the spatial interaction features are input into the fully connected layer module to output the predicted emotion category results.

[0027] Furthermore, in the differential warm-up training stage, a small amount of sample data is used to perform the training, including the following steps:

[0028] Using a random mask matrix to perform random masking on the input three-dimensional mapping matrix to generate a masked three-dimensional mapping matrix;

[0029] The original three-dimensional mapping matrix and the mask three-dimensional mapping matrix are processed by the high-frequency window module, the low-frequency window module, the feature fusion module and the spatial interaction neural network module respectively to obtain their respective spatial interaction features;

[0030] The loss function is calculated using their respective spatial interaction features and the model parameters except for the fully connected layer are updated.

[0031] Furthermore, in the formal training stage, the following steps are included: the three-dimensional mapping matrix is ​​input into the high-frequency window module, the low-frequency window module, the feature fusion module and the spatial interaction neural network module for processing, the spatial interaction features are obtained and the emotion category result prediction value is obtained through the fully connected layer, the loss function is calculated with the target value, and the parameters of the entire model are updated, including the parameters of the fully connected layer.

[0032] Furthermore, the loss function used in the differential warm-up training stage is the mean square error between the respective spatial interaction features; and the loss function used in the formal training stage is the cross entropy loss function.

[0033] Furthermore, the spatial interactive neural network is a spatial interactive Kolmogorov-Arnold network.

[0034] The second aspect of the present invention provides a cross-subject EEG signal emotion recognition model training system based on spatial interaction features, including a memory and a processor, wherein the memory includes a cross-subject EEG signal emotion recognition model training method program based on spatial interaction features, and when the cross-subject EEG signal emotion recognition model training method program based on spatial interaction features is executed by the processor, the steps of a cross-subject EEG signal emotion recognition model training method based on spatial interaction features are implemented.

[0035] The third aspect of the present invention provides a method for cross-subject EEG signal emotion recognition based on spatial interaction features, which uses a cross-subject EEG signal emotion recognition model training method based on spatial interaction features to train a cross-subject EEG signal emotion recognition model based on spatial interaction features, and uses the trained cross-subject EEG signal emotion recognition model based on spatial interaction features to perform emotion recognition on EEG signals to obtain emotion recognition results.

[0036] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0037] The present invention proposes a warm-up learning method before formal training: differential warm-up learning (DWL), which uses small sample data to simulate different EEG patterns caused by physiological differences across subjects, so that the model can learn in advance to distinguish inter-individual differences and intra-class differences in EEG signals; a high-frequency and low-frequency window (HFLFW) module is proposed, which captures local refinement features and global structural features of input data through high-frequency and low-frequency windows respectively to extract spatial frequency information of the image; a spatial interaction neural network model is proposed, which uses a special batch-level expansion design and the nonlinear modeling ability of the neural network model to effectively capture the spatial interaction characteristics of EEG emotion signals in different channels between different subjects, so as to enhance the modeling ability of the model for different EEG patterns. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to make the purpose and technical solution of the present invention clearer, the present invention provides the following drawings and descriptions:

[0039] Figure 1 A flow chart of a method provided by an embodiment of the present invention;

[0040] Figure 2 A three-dimensional mapping matrix mapping process diagram provided by an embodiment of the present invention;

[0041] Figure 3 A schematic diagram of a differential learning process provided by an embodiment of the present invention;

[0042] Figure 4 A flow chart of high and low frequency window feature extraction provided by an embodiment of the present invention;

[0043] Figure 5 A diagram comparing the Kolmogorov-Arnold network and the multilayer perceptron structure provided by an embodiment of the present invention;

[0044] Figure 6 A flowchart of a spatial interactive Kolmogorov-Arnold network provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0046] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0047] Embodiment 1:

[0048] The present invention provides a method for training a cross-subject EEG signal emotion recognition model based on spatial interaction features. Figure 1 The figure shows a flow chart of a method for training a cross-subject EEG signal emotion recognition model based on spatial interaction features. The specific steps are as follows:

[0049] S1: extracting a three-dimensional mapping matrix of the electroencephalogram signal data, and performing random masking on the three-dimensional mapping matrix to obtain a masked three-dimensional mapping matrix.

[0050] The specific process is:

[0051] Using the time window segmentation method, the input EEG signal data is segmented according to the window length of T seconds to obtain the emotional EEG signals in multiple time windows. For each input S∈R C′×T′ , where C′ and T′ represent the number of channels and time steps of the EEG signal, respectively. A time window of T seconds is used to cut each input, as shown in the following formula:

[0052] S->S i ∈R C×C {i=1,2,...,N}

[0053] Among them, C is the number of EEG signal channels, T is the time window length, and N is the number of signal segments;

[0054] The baseline EEG signal is removed from the emotional EEG signal to eliminate the individual differences in emotional state and obtain the baseline-free emotional EEG signal. Generally, before the EEG signal is collected, a baseline signal within 5 seconds is recorded before the experiment is carried out. These pre-trial baselines record the emotional state of the subject before the stimulation, and there are individual differences. The purpose of recording the baseline signal is to ensure that the EEG signal in the subsequent experimental process is only affected by the current emotion and eliminate some individual differences. Therefore, baseline removal is required. represents the baseline signal for segmentation, To represent the segmented emotional EEG signal, first calculate the average baseline signal to represent the average pre-stimulus emotional pattern of all subjects:

[0055]

[0056] in, Represents the average baseline signal, and K is the total number of baseline signals. Remove the average baseline signal from the emotional EEG signal:

[0057]

[0058] in To remove the baseline EEG emotion signal, a 30th-order Butterworth filter is used to decompose the signal into four frequency bands: theta band (4-7Hz), alpha band (8-12Hz), beta band (13-30Hz) and gamma band (31-45Hz). The specific formula is as follows:

[0059]

[0060] Among them, H f (S) represents Butterworth filter, represents the subband after applying the filter, where Ω represents the band label (theta, alpha, beta and gamma bands), and P is the subband length. Here it can be understood that each input data with dimension C×T, after being decomposed by the filter band, obtains four outputs with dimension C×P, each of which represents a subband.

[0061] The differential entropy is calculated for each sub-band signal to obtain a matrix containing the differential entropy characteristics of each band. The expression for calculating the differential entropy of each sub-band signal is:

[0062]

[0063] Among them, X i (Ω)∈R C×1 Indicates subband The differential entropy, σ 2Represents the variance of the signal. Here, the data of each sub-band is converted into a value after the calculation of differential entropy. It was originally C×P, but became C×1. That is to say, the initial C×T input is eventually converted into four C×1 outputs. Finally, a three-dimensional mapping is performed. Since each EEG channel corresponds to a specific brain area, in order to capture spatial frequency information, each sub-band is mapped to a two-dimensional spatial representation of size 9x9. Then, along the channel dimension (if a data dimension is (H, W, C), then H and W represent the height and width of the image, respectively, and C is the channel dimension of the image), the two-dimensional spatial representations of the four sub-bands are connected to obtain a three-dimensional representation of the signal. The mapping process is as follows Figure 2 As shown. FP1 and other symbols in the figure represent the names of electrode channels. The matrix can be regarded as a square approximation of the human brain. The corresponding electrode channels are placed in the corresponding parts of the human brain to collect EEG signals. The electrode channel installation method in the figure is based on the positions of 32 channels selected by the international 10-20 system. Figure 2 On the left is an original collected input EEG signal, and each row of signals represents the EEG signal of a channel. In the middle is the vector obtained after time window segmentation, baseline removal, frequency band decomposition and differential entropy calculation. Each row of signals on the far left will become four sub-band signals after frequency band decomposition, and then the differential entropy is calculated on each sub-band signal, so one channel can get four differential entropy data. The rightmost is the final mapping result, and a differential entropy is placed at its corresponding mapping position. There are 32 channels in total, and each channel has four differential entropy data. Therefore, the final three-dimensional mapping has four channels, and each channel has 32 positions with values, and the remaining positions are set to zero. Convert each input into a matrix of size (9,9,4), and fill the remaining matrix areas without corresponding electrodes with zeros to obtain a three-dimensional mapping matrix.

[0064] S2: Perform differential warm-up training, input the three-dimensional mapping matrix and the mask three-dimensional mapping matrix into the emotion recognition model that combines the high-frequency window module, the low-frequency window module and the spatial interaction Kolmogorov-Arnold network module, output the respective spatial interaction features, update some module parameters in the emotion recognition model through the loss function, and obtain the emotion recognition model after warm-up.

[0065] More specifically, the emotion recognition model further includes a feature fusion module and a fully connected layer module; the method for processing a three-dimensional mapping matrix using the emotion recognition model includes the following steps:

[0066] Linearly map the three-dimensional mapping matrix and add position encoding in high and wide dimensions. In order to emphasize the position information of the EEG signal channel in the three-dimensional feature map, first convert the input data X i ∈R H×W×4 Map to X i ∈RH×W×32 , and then add the position information to the EEG feature map through the sinusoidal encoder:

[0067]

[0068] Where dim is the input embedding dimension, which is 32 in this case. PE(t,:) represents the sinusoidal encoder, t represents different positions, and i is the index variable used to traverse the embedding vector dimension. The zero-filled area also supplements the positional encoding to simulate the information of adjacent areas. Let the encoded data be X pos,i ∈R H×W×32 :

[0069] X pos,i =X i +PE(t,:)

[0070] Divide the three-dimensional mapping matrix into multiple sub-blocks according to the 3x3 local window;

[0071] Local multi-head self-attention operations are performed within the sub-block to extract fine-grained high-frequency features. Compared with directly inputting all patches (a patch is a high-resolution image such as 1000*1000. If each pixel is directly input as a unit, it will result in huge memory consumption. Some models will input 5*5 pixels as a unit, so 5*5 can be regarded as a patch.) into the feature extractor for attention operation, the high-frequency window divides the input data into multiple local sub-blocks (for example, 3×3) and performs local multi-head self-attention operations within the sub-block to extract fine-grained high-frequency features and short-distance dependencies.

[0072] Average pooling is performed in the sub-block to obtain n pixels. The low-frequency features are extracted using the cross-attention operation. The low-frequency window is first average pooled in each local window to obtain the input X i The low-frequency signal of the average pooling is then mapped to the key Sum Where L = H × W, w is the size of the low-frequency window. In order to reduce the information loss caused by pooling, the original input data X pos,i Mapping to query V∈R L×dim Then, the low-frequency features are captured through a multi-head cross-attention mechanism as follows:

[0073]

[0074] CrossAttention(X) = Concat(Z 1 ,Z 2 ,...Z h )W O

[0075] in is the dimension of k heads (multi-head attention), Z i For the output of each attention head, the matrix W i Q , W i K , W i V Used to project the query vector, key vector, and value vector into different subspaces. CrossAttention(X) represents cross attention. is the weight matrix that merges the connection tensors into the final output dimension;

[0076] The high-frequency features and low-frequency features are concatenated and linearly mapped to obtain fused features. In this embodiment, the input data block H = W, so the outputs of the two windows are H∈R L×dim and W∈R L×dim Therefore, the high frequency window output is X H ∈R H×W×dim , the low frequency window output is X L ∈R H×W×dim . Connect X along the channel dimension H and X l , and then apply linear mapping to get the fusion features:

[0077] X O =Linear(Concat C [X H ;X L ])

[0078] Where X O ∈R H×W×4 Concat is a fusion feature. C Indicates connection along the channel dimension. The whole process of this module is as follows Figure 4 shown.

[0079] Using a batch splicing method to splice multiple fused features to obtain a horizontal tensor unit and a vertical tensor unit, and adding position encoding information to the horizontal tensor unit and the vertical tensor unit to enhance the position feature;

[0080] The horizontal tensor units and the vertical tensor units are spliced ​​at corresponding positions to form a combined tensor containing rich spatial interaction features;

[0081] The combined tensor is input into the preset neural network to extract the spatial interaction features between the multi-channel EEG signals, which are input into the fully connected layer to obtain the predicted emotion category results.

[0082] The updating of the parameters of the high-frequency and low-frequency window modules and the parameters of the spatial interaction Kolmogorov-Arnold network model is carried out in stages, including: a differential warm-up stage and a formal training stage.

[0083] Like the Multilayer Perceptron (MLP), the Kolmogorov-Arnold Network (KAN) has a fully connected structure. However, the KAN moves the learnable activation function from the neurons to the weights, such as Figure 5 As shown. Compared with MLP, KAN has stronger nonlinear representation ability and better interpretability. Therefore, this module uses KAN as a feature extractor, and sets B as the batch size (Bacthsize), that is, the model updates the parameters each time it processes B samples. For a four-dimensional input tensor (B, H, W, C), the module first performs batch-level horizontal splicing along the width (W) dimension, that is, continuously splicing B samples along the W direction, horizontally integrating slices of the same height of different samples to obtain a horizontal tensor unit (1, H, W × B, C), and then performs batch-level vertical splicing along the height (H) dimension, continuously splicing B samples along the H direction to obtain a vertical tensor unit (1, H × B, W, C), and then places the horizontal tensor unit and the vertical tensor unit in the fourth quadrant of the Cartesian coordinate system. The horizontal tensor unit is along the positive direction of the x-axis and the vertical tensor unit is along the negative direction of the y-axis. For each column of pixels in the horizontal tensor unit, draw a vertical wire extending along the negative direction of the y-axis. The vertical tensor unit starts from the first sample of the horizontal tensor unit and moves one sample length in the positive x-axis direction each time. The column pixels that touch the line are vertically connected to the column pixels corresponding to the horizontal tensor unit, and then passed to KAN to extract the associated spatial information between different EEG channels within the same individual and between different individuals. In order to enhance the position information, a sinusoidal encoder is constructed along the positive x-axis and negative y-axis to simulate the position information of the tensor unit. The workflow of the spatial interactive Kolmogorov-Arnold network (SIK) is as follows Figure 6 shown.

[0084] Let the column pixel of the horizontal tensor unit corresponding to the vertical wire be P i h ∈R H×1×C {i=1,2,...,W×B}. Let the column pixels of the vertical tensor unit be First, the two pixels are vertically spliced ​​and then sent to the network to extract spatial interaction features. Therefore, the expression of the spatial interaction Kolmogorov-Arnold network is:

[0085]

[0086]

[0087] in, Represents the concatenation of vertical tensor units and column pixels of horizontal tensor units. H and Concat H Represents connections along the width and height dimensions, respectively. N is the number of times KAN is applied. The output of the module maintains the same dimension as the input, resulting in a four-dimensional tensor of size (B, H, W, C). Finally, the output of the spatial interaction KAN is passed through a KAN-based fully connected layer to generate predicted emotion categories. In this example, since the data collection is scored 1-9 from four perspectives: valence, arousal, dominance, and liking, 1-5 is defined as low (valence, arousal, dominance, and liking), and 6-9 is defined as high (valence, arousal, dominance, and liking). At this time, it becomes a binary classification task, and the output dimension of the fully connected layer is 2, representing high or low prediction results of the emotion, respectively.

[0088] S3: Conduct formal training, input the three-dimensional mapping matrix into the warmed-up emotion recognition model, output the predicted value of the emotion category result, update the parameters of the entire model through the loss function, and obtain the trained emotion recognition model.

[0089] For training, the model will first randomly mask some pixels of each image in the dataset to simulate the physiological differences between different individuals, and then use these data and the original unmasked data to pass into the subsequent two modules. At this time, in the output of the last spatial interaction KAN, they are not passed through the fully connected layer, but directly use their output to calculate the mean square error loss, and then preliminarily update the parameters of the model. This stage can be regarded as a warm-up stage. After the warm-up is completed, formal training begins. The data will pass directly through the subsequent two modules. At this time, the output will pass through the fully connected layer to obtain the predicted value, and then the cross entropy loss will be calculated with the original value to update the model parameters. For the inference stage, the user only needs to input his own EEG data, and the model can automatically and accurately identify the subject's emotions at this time based on the cross-subject emotion recognition ability it has learned.

[0090] In the differential warm-up phase, by using small sample data to simulate different EEG patterns caused by physiological differences across subjects, the model can learn to distinguish between individual differences and intra-class differences in EEG signals in advance, including:

[0091] The input three-dimensional mapping matrix is ​​randomly masked using a random mask matrix to generate a masked data block. Due to individual physiological differences, the EEG signal shows significant inter-individual and intra-class differences. In order to improve the performance of the model in the cross-subject emotion recognition task, warm-up learning is first performed directly on the original small sample dataset to improve the model's ability to distinguish physiological differences between subjects. In order to simulate physiological differences between subjects, P% of non-zero padded pixels are first randomly masked in the input data block of size (H, W, C). In particular, for each input Xi ∈R H×W×4 , randomly generate a mask matrix M∈R H×W×4 ,in:

[0092]

[0093] Then we can calculate the masked data, that is, the data X that can simulate the physiological differences of different subjects. mask ∈R H ×W×4 :

[0094] X mask =X i ⊙M

[0095] where ⊙ represents element-wise multiplication.

[0096] The original three-dimensional mapping matrix and the mask three-dimensional mapping matrix are processed by the high-frequency window module, the low-frequency window module, the feature fusion module and the spatial interaction Kolmogorov-Arnold network module respectively to obtain their respective spatial interaction features;

[0097] Calculate the mean square error between spatial interaction features and use this error as the loss function to update model parameters except the fully connected layer. Calculate the mean square error between spatial interaction features as the loss function for updating subsequent module parameters in the warm-up phase. Note that the parameters of the fully connected layer used for classification are not updated here:

[0098] L = MSE(X o ,X omask )

[0099]

[0100] Differential warm-up learning enhances the randomness and diversity of data by masking pixels. The module predicts the original image in the presence of missing pixels, which enables the model to accurately predict the original image content in the face of missing information, thereby driving the network to deeply learn the subtle physiological differences between individuals contained in the EEG signals during the pre-training stage. This mechanism not only enhances the model's ability to learn complex features, but also promotes richer and more robust feature representation. The whole process is as follows: Figure 3 shown.

[0101] In the formal training phase, the following steps are included: the three-dimensional mapping matrix is ​​input into the high-frequency window module, the low-frequency window module, the feature fusion module and the spatial interaction Kolmogorov-Arnold network module for processing, the spatial interaction features are obtained and the predicted value of the emotion category result is obtained through the fully connected layer, and the parameters of the whole model, including the parameters of the fully connected layer, are updated by comparing with the cross entropy loss function of the target value. The expression is as follows:

[0102] L = -[ylog(p)+(1-y)log(1-p)]

[0103] Among them, the probability distribution of the model output is p (the probability of the positive class), and the true label is y.

[0104] Embodiment 2:

[0105] The present embodiment provides a system for training a cross-subject EEG signal emotion recognition model based on spatial interaction features, comprising a memory and a processor, wherein the memory comprises a program for training a cross-subject EEG signal emotion recognition model based on spatial interaction features, and when the program for training a cross-subject EEG signal emotion recognition model based on spatial interaction features is executed by the processor, the steps of a cross-subject EEG signal emotion recognition model training method based on spatial interaction features as described in Example 1 are implemented.

[0106] Embodiment 3:

[0107] The present embodiment provides a method for cross-subject EEG signal emotion recognition based on spatial interaction features, and uses the cross-subject EEG signal emotion recognition model training method based on spatial interaction features described in Example 1 to train the cross-subject EEG signal emotion recognition model based on spatial interaction features. The trained cross-subject EEG signal emotion recognition model based on spatial interaction features is used to perform emotion recognition on EEG signals to obtain emotion recognition results.

[0108] The effectiveness of this method was verified by conducting cross-subject experiments with existing methods (support vector machine SVM, decision tree DT, ViT, FBCNet, ATCNet, STFFNN). The DEAP dataset was used, which recorded 40 emotional EEG signals of 32 subjects. Each EEG signal was 60 seconds long and was cut into 1 second pieces, so there were 32*40*60=76800 data in total. Then a five-fold cross validation was performed, that is, the 76800 data were divided into five blocks, one block was used for testing each time, and the remaining four blocks were used for training until each block was used as a test set, and then the average accuracy of the five times was recorded as the final result. Valence and arousal are two different emotions. Subjects will score valence and arousal from 1 to 9 to express their emotions. 1-4 is considered low valence / low arousal, and 5-9 is considered high valence / high arousal, mainly for low and high classification. The following Table 1 records the cross-subject comparison results, and the numbers in the table are the percentage of classification accuracy.

[0109] Table 1

[0110] method Potency wake Support Vector Machine (SVM) 64.31 63.34 Decision Tree DT 60.67 61.52 ViT 64.47 65.06 FBCNet 70.63 67.68 ATCNet 76.73 79.02 STFFNN 80.17 81.82 This method 94.78 94.61

[0111] In addition, a non-cross-subject experiment was conducted: the data of 32 individuals were divided into 1 second, and each individual had 40*60=2400 data. For each individual, a five-fold cross validation was performed, and then the average accuracy of the 32 individuals was calculated. The experimental results are shown in Table 2 below, and the numbers in the table are the classification accuracy percentages.

[0112] Table 2

[0113] method Potency wake ViT 67.23 69.68 FBCNet 74.23 75.85 ATCNet 93.72 92.59 This method 96.20 96.04

[0114] Experimental results show that our scheme outperforms existing schemes in terms of valence and arousal, and maintains high performance under different experimental settings. This consistency demonstrates the stability and reliability of the model.

[0115] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A method for training a cross-subject EEG signal emotion recognition model based on spatial interaction features, characterized in that: The steps include: Extracting a three-dimensional mapping matrix of the electroencephalogram signal data, and performing random masking on the three-dimensional mapping matrix to obtain a masked three-dimensional mapping matrix; Perform differential warm-up training, input the three-dimensional mapping matrix and the mask three-dimensional mapping matrix into an emotion recognition model that combines a high-frequency window module, a low-frequency window module, and a spatial interaction neural network module, output respective spatial interaction features, and update some module parameters in the emotion recognition model through a loss function to obtain a warm-up emotion recognition model; Formal training is carried out, the three-dimensional mapping matrix is ​​input into the warm-up emotion recognition model, the emotion category result prediction value is output, and the parameters of the entire model are updated through the loss function to obtain the trained emotion recognition model.

2. According to the method for training a cross-subject EEG signal emotion recognition model based on spatial interaction features according to claim 1, it is characterized in that: The method of extracting the three-dimensional mapping matrix of the electroencephalogram signal data comprises the following steps: Using the time window segmentation method, the input EEG signal data is segmented according to the window length of T seconds to obtain the emotional EEG signals in multiple time windows; The baseline EEG signal is removed from the emotional EEG signal to obtain a de-baselined emotional EEG signal; Decompose the baseline-free emotional EEG signal into n sub-band signals using a preset filter; The differential entropy is calculated for each sub-band signal to obtain a matrix containing the differential entropy characteristics of each frequency band; The differential entropy features of each frequency band are placed into a blank matrix according to the corresponding electrode channel position to obtain a three-dimensional mapping matrix.

3. The method for training a cross-subject EEG signal emotion recognition model based on spatial interaction features according to claim 2, characterized in that: The preset filter is a Butterworth filter.

4. The method for training a cross-subject EEG signal emotion recognition model based on spatial interaction features according to claim 1, characterized in that: The emotion recognition model also includes a feature fusion module and a fully connected layer module; The method for processing a three-dimensional mapping matrix using an emotion recognition model comprises the following steps: Linearly map the three-dimensional mapping matrix and add position encoding in high and wide dimensions; Divide the three-dimensional mapping matrix after adding the position code into a plurality of sub-blocks; Input the sub-block into the high-frequency window module, perform feature extraction using local multi-head self-attention, and output high-frequency features; input the sub-block into the low-frequency window module, perform average pooling in the sub-block to obtain n pixels, perform feature extraction using cross-attention operation, and output low-frequency features; Input the high-frequency features and low-frequency features into the feature fusion module for feature concatenation and linear mapping to obtain fused features; Using a batch splicing method to splice multiple fused features to obtain a horizontal tensor unit and a vertical tensor unit, and adding position encoding information to the horizontal tensor unit and the vertical tensor unit to enhance the position feature; The horizontal tensor unit and the vertical tensor unit are spliced ​​to obtain a combined tensor containing rich spatial interaction features; The combined tensor is input into the spatial interaction neural network module to extract the spatial interaction features between the multi-channel EEG signals, and the spatial interaction features are input into the fully connected layer module to output the predicted emotion category results.

5. The method for training a cross-subject EEG signal emotion recognition model based on spatial interaction features according to claim 4, characterized in that: In the differential warm-up training phase, a small amount of sample data is used, including the following steps: Using a random mask matrix to perform random masking on the input three-dimensional mapping matrix to generate a masked three-dimensional mapping matrix; The original three-dimensional mapping matrix and the mask three-dimensional mapping matrix are processed by the high-frequency window module, the low-frequency window module, the feature fusion module and the spatial interaction neural network module respectively to obtain their respective spatial interaction features; The loss function is calculated using their respective spatial interaction features and the model parameters except for the fully connected layer are updated.

6. The method for training a cross-subject EEG signal emotion recognition model based on spatial interaction features according to claim 4, characterized in that: In the formal training phase, the following steps are included: the three-dimensional mapping matrix is ​​input into the high-frequency window module, the low-frequency window module, the feature fusion module and the spatial interaction neural network module for processing, the spatial interaction features are obtained and the emotion category result prediction value is obtained through the fully connected layer, the loss function is calculated with the target value, and the parameters of the entire model, including the parameters of the fully connected layer, are updated.

7. A method for training a cross-subject EEG signal emotion recognition model based on spatial interaction features according to claim 5 or 6, characterized in that: The loss function used in the differential warm-up training stage is the mean square error between the respective spatial interaction features; the loss function used in the formal training stage is the cross entropy loss function.

8. A method for training a cross-subject EEG signal emotion recognition model based on spatial interaction features according to claim 5 or 6, characterized in that: The spatial interaction neural network is a spatial interaction Kolmogorov-Arnold network.

9. A cross-subject EEG signal emotion recognition model training system based on spatial interaction features, characterized in that: The system includes: a memory and a processor, wherein the memory includes a program for a method for training a cross-subject EEG signal emotion recognition model based on spatial interaction features, and when the program for training a cross-subject EEG signal emotion recognition model based on spatial interaction features is executed by the processor, the steps of a method for training a cross-subject EEG signal emotion recognition model based on spatial interaction features as described in any one of claims 1 to 8 are implemented.

10. A method for cross-subject EEG signal emotion recognition based on spatial interaction features, characterized in that: The method adopts the model trained by claims 1-8, and uses the model to perform emotion recognition on EEG signals to obtain emotion recognition results.

Images