Sentiment Monitoring Method Based on Online Independent Component Analysis

Abstract

The present invention discloses a method for emotion monitoring based on online independent component analysis, comprising the following steps: step 1, filtering, removing bad leads and removing baseline operations on real-time EEG signals obtained from a signal acquisition device; step 2, setting a sliding window of 1s and a step size of 0.5s, segmenting the above-mentioned EEG signals, feeding the segmented data into a generative adversarial network model to generate an artifact signal, and inputting the segmented data and the segmented signal into an online independent component analysis to fully remove artifacts; step 3, extracting the spatial and temporal features of the EEG signals respectively using spatial mapping technology and time series modeling from the processed data; step 4, deeply fusing the temporal and spatial features and inputting them into a trained softmax classifier for emotion monitoring. The present invention performs real-time anti-aliasing operations on real-time EEG signals, ensuring that the anti-aliasing delay is low and the anti-aliasing quality remains at a high level.

Description

Technical Field

[0001] The present invention relates to the technical field of real-time electroencephalogram (EEG) signal processing, and in particular to an emotion monitoring method based on online ICA algorithm signal processing. Background Art

[0002] In recent years, people have become increasingly concerned about their health and emotions. As an emerging biosignal processing method, emotion monitoring technology has been widely researched and applied in fields such as psychology, medicine, and human-computer interaction. Identifying emotional states not only helps people understand their own emotional changes, but also better understand their health.

[0003] Electroencephalography (EEG), a biosignal of brain activity, can accurately reflect a person's emotional state. By collecting real-time EEG signals and analyzing their waveform characteristics, a person's emotional state can be accurately determined. However, EEG signal collection is subject to interference from numerous external and internal sources, such as electrooculogram (EOG) and electromyography (EMG). Removing these interference signals requires more effective processing.

[0004] Independent Component Analysis (ICA) is a powerful blind source separation algorithm that can effectively remove interference signals from EEG signals and accurately judge emotions.

[0005] However, current ICA algorithms rely on offline processing, typically processing EEG signals after they are acquired. This introduces a slight delay, making it difficult to implement emotion monitoring. In practice, real-time monitoring can provide a more timely reflection of the human body's state, making it suitable for applications requiring more precise emotion monitoring, such as healthcare and human-computer interaction. Summary of the Invention

[0006] In response to the shortcomings of the above-mentioned existing technologies, the present invention proposes an emotion monitoring method based on online independent component analysis, which can perform real-time signal processing and emotion monitoring while collecting EEG signals, eliminating the delay problem of traditional ICA processing methods; at the same time, combined with the ICA algorithm optimized by GAN, it can effectively remove artifacts and extract the required EEG signals, which can effectively improve the accuracy and robustness of emotion monitoring.

[0007] To achieve the above object, the technical solution of the present invention is: comprising the following steps:

[0008] S1, obtains the user's real-time brain signal and performs preliminary preprocessing, including filtering, removing bad signals and removing baselines;

[0009] S2, the pre-processed brain signal is segmented according to the sliding window, and the segmented data is fed into the generative adversarial network model to generate similar artifact signals and perform independent component analysis together with the segmented data to remove artifacts;

[0010] S3, using convolutional neural networks and self-attention mechanisms to extract spatial and temporal features of EEG signals from the obtained data;

[0011] S4, deeply fuses the extracted spatial and temporal features and inputs them into the softmax classifier for sentiment classification to achieve online real-time sentiment monitoring.

[0012] Preferably, step S1, preliminary pre-processing includes the following:

[0013] (1) Filtering is performed using a 1-45 Hz bandpass filter;

[0014] (2) Based on the amplitude and waveform stability of the EEG signal, the bad conductor is marked;

[0015] (3) Select a certain section of static state signal, calculate the baseline signal, and subtract the baseline value;

[0016]

[0017] T is the time of a certain period of rest, and the subsequent EEG signals are subtracted from the baseline value.

[0018] Preferably, in step S2, the size of the sliding window is set to 1s, the step size is set to 0.5s, and the segmented EEG signal is:

[0019] X={x1,x2,...,x t}

[0020] T represents the tth sliding window;

[0021] Using real EEG signals, GAN is used to generate artifact signals. The generator is trained to generate artifact signals that are statistically similar to the real signals. The generated artifact signals are compared with the real EEG signals. ICA is used to remove the artifacts of electrooculogram and electromyography signals from these signals to improve the quality and efficiency of artifact removal.

[0022] Preferably, in step S3, first, the EEG signal X processed in step S2 is converted into {x1, x2, ..., x t}, where x t represents the t-th sliding window;

[0023] The segmented EEG signals are input into the Generative Adversarial Network model, where the generator generates artifact signals as much as possible. The discriminator distinguishes between the artifact signals generated by the generator and the real EEG signals. On this basis, the generator is continuously optimized to generate artifact signals that are close to the real ones. The generated artifact signals and the segmented original EEG signals are subjected to independent component analysis together, so that the signals processed by ICA can effectively remove endogenous signals (such as electrooculogram, electromyography, etc.) and other interference signals. The independent component analysis algorithm optimized by GAN can further extract the real EEG signals and improve robustness.

[0024] Preferably, in step S3, the one-dimensional EEG signal processed in step 2 is mapped into a two-dimensional matrix according to the electrode position of the EEG signal acquisition device:

[0025]

[0026] m represents the mth electrode;

[0027] Use convolutional neural network to process the two-dimensional signal frame X t For spatial feature extraction, use a 3×3 convolution kernel and a 2×2 convolution kernel, without using pooling operations to retain more spatial feature information:

[0028] F1=Conv(X,W1,b1)

[0029] F2=Conv(X,W2,b2)

[0030] W is the convolutional layer, b is the bias term, and F is the output feature map;

[0031] The self-attention mechanism is used to extract the temporal features of the EEG signal processed in step 2, and the weighted sum is used to obtain the final temporal feature vector Ft. The EEG signal is mapped to three different matrices with different weights:

[0032] Query matrix (Q): Q = x t w q ;

[0033] Key matrix (K): K = x t w k ;

[0034] Value matrix (V) V = x t w v ;

[0035] where w q , w k , w vis the weight matrix obtained through learning;

[0036] Then calculate the similarity of the sliding window, that is, the attention score:

[0037]

[0038] And perform normalization:

[0039]

[0040] The final output matrix V is weighted summed to obtain the final time eigenvector:

[0041] Output=Attention Weights×V.

[0042] Preferably, in step S3, the first convolution kernel is:

[0043] F1=ReLU(W1*X(t)+b1)

[0044] Among them, * represents the convolution operation, ReLU is the activation function, F1 is the feature matrix after convolution, and b1 is the bias term;

[0045] The second convolution kernel:

[0046] F2=ReLU(W2*F1+b2)

[0047] At the same time, the pooling operation is not used to ensure the maximum retention of spatial features, and finally the extracted spatial feature matrix F2 is flattened into a one-dimensional vector F s .

[0048] Preferably, in step S4, the spatial and temporal features extracted from CNN and self-attention mechanism are F s and F t And the temporal feature F of the EEG signal is extracted through the self-attention mechanism t , the spatial and temporal features are spliced ​​to form a new fusion feature vector F f , F f =[F s ||F t ],|| represents the feature concatenation operation;

[0049] The fusion feature is expressed as:

[0050] F f =concat(F s ,F t )

[0051] Finally, F f Input softmax classifier for sentiment classification:

[0052]

[0053] θ k Expressed as the weight of the k-th category sentiment, k is the number of categories, y k It represents the predicted probability of the k-th emotion, and finally selects the emotion with the largest predicted probability as the output.

[0054] The technical principles and beneficial effects of the present invention are as follows:

[0055] The online ICA algorithm proposed in this paper can perform real-time signal processing and emotion monitoring while collecting EEG signals, eliminating the delay problem of traditional ICA processing methods; at the same time, combined with the ICA algorithm optimized by GAN, it can effectively remove artifacts and extract the required EEG signals, which can effectively improve the accuracy and robustness of emotion monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is the algorithm flow chart of the present invention;

[0057] Figure 2 This is a schematic diagram of the GAN-based optimized ICA algorithm framework of the present invention. DETAILED DESCRIPTION

[0058] The technical solutions of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. It is apparent that the embodiments described are merely preferred embodiments of the present invention, not all embodiments. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are intended to fall within the scope of protection of the present invention.

[0059] Example

[0060] like Figure 1 As shown, the present invention has the following steps:

[0061] S1, obtains the user's real-time brain signal and performs preliminary preprocessing, including filtering, removing bad signals and removing baselines;

[0062] S2, the pre-processed brain signal is segmented according to the sliding window, and the segmented data is fed into the generative adversarial network model to generate similar artifact signals and perform independent component analysis together with the segmented data to remove artifacts;

[0063] S3, using convolutional neural networks and self-attention mechanisms to extract spatial and temporal features of EEG signals from the obtained data;

[0064] S4, deeply fuses the extracted spatial and temporal features and inputs them into the softmax classifier for sentiment classification to achieve online real-time sentiment monitoring.

[0065] Preferably, step S1, preliminary pre-processing includes the following:

[0066] (1) Filtering is performed using a 1-45 Hz bandpass filter;

[0067] (2) Based on the amplitude and waveform stability of the EEG signal, the bad conductor is marked;

[0068] (3) Select a certain section of static state signal, calculate the baseline signal, and subtract the baseline value;

[0069]

[0070] T is the time of a certain period of rest, and the subsequent EEG signals are subtracted from the baseline value.

[0071] Preferably, in step S2, the size of the sliding window is set to 1s, the step size is set to 0.5s, and the segmented EEG signal is:

[0072] X={x1,x2,...,x t}

[0073] T represents the tth sliding window;

[0074] Using real EEG signals, GAN is used to generate artifact signals. The generator is trained to generate artifact signals that are statistically similar to the real signals. The generated artifact signals are compared with the real EEG signals. ICA is used to remove the artifacts of electrooculogram and electromyography signals from these signals to improve the quality and efficiency of artifact removal.

[0075] Preferably, in step S3, first, the EEG signal X processed in step S2 is converted into {x1, x2, ..., x t}, where x t represents the t-th sliding window;

[0076] The segmented EEG signals are input into the Generative Adversarial Network model, where the generator generates artifact signals as much as possible. The discriminator distinguishes between the artifact signals generated by the generator and the real EEG signals. On this basis, the generator is continuously optimized to generate artifact signals that are close to the real ones. The generated artifact signals and the segmented original EEG signals are subjected to independent component analysis together, so that the signals processed by ICA can effectively remove endogenous signals (such as electrooculogram, electromyography, etc.) and other interference signals. The independent component analysis algorithm optimized by GAN can further extract the real EEG signals and improve robustness.

[0077] Preferably, in step S3, the one-dimensional EEG signal processed in step 2 is mapped into a two-dimensional matrix according to the electrode position of the EEG signal acquisition device:

[0078]

[0079] m represents the mth electrode;

[0080] Use convolutional neural network to process the two-dimensional signal frame X t For spatial feature extraction, use a 3×3 convolution kernel and a 2×2 convolution kernel, without using pooling operations to retain more spatial feature information:

[0081] F1=Conv(X,W1,b1)

[0082] F2=Conv(x,W2,b2)

[0083] W is the convolutional layer, b is the bias term, and F is the output feature map;

[0084] Use the self-attention mechanism to extract the temporal features of the EEG signal processed in step 2, and perform weighted summation to obtain the final temporal feature vector F t , the EEG signal is mapped to three different matrices with different weights:

[0085] Query matrix (Q): Q = x t w q ;

[0086] Key matrix (K): K = x t w k ;

[0087] Value matrix (V) V = x t w v ;

[0088] where w q , w k , w v is the weight matrix obtained through learning;

[0089] Then calculate the similarity of the sliding window, that is, the attention score:

[0090]

[0091] And perform normalization:

[0092]

[0093] The final output matrix V is weighted summed to obtain the final time eigenvector:

[0094] Output=AttentionWeights×V.

[0095] Preferably, in step S3, the first convolution kernel is:

[0096] F1=ReLU(W1*X(t)+b1)

[0097] Among them, * represents the convolution operation, ReLU is the activation function, F1 is the feature matrix after convolution, and b1 is the bias term;

[0098] The second convolution kernel:

[0099] F2=ReLU(W2*F1+b2)

[0100] At the same time, the pooling operation is not used to ensure the maximum retention of spatial features, and finally the extracted spatial feature matrix F2 is flattened into a one-dimensional vector F s .

[0101] Preferably, in step S4, the spatial and temporal features extracted from CNN and self-attention mechanism are F S and F t The temporal feature Ft of the EEG signal is extracted through the self-attention mechanism, and the spatial and temporal features are spliced ​​to form a new fusion feature vector F f , F f =[F s ||F t ],|| represents the feature concatenation operation;

[0102] The fusion feature is expressed as:

[0103] F f =concat(F S ,F t )

[0104] Finally, F f Input softmax classifier for sentiment classification:

[0105]

[0106] θ k Expressed as the weight of the k-th category sentiment, k is the number of categories, y k It represents the predicted probability of the k-th emotion, and finally selects the emotion with the largest predicted probability as the output.

[0107] Implementation Process

[0108] The specific steps of the emotion monitoring method based on the online ICA algorithm of the present invention are as follows:

[0109] Step S1: Collect real-time EEG signals and perform preliminary preprocessing. Specifically:

[0110] Real-time EEG signal acquired t , assuming that the signal amplitude at each time point is x t , where t represents the time point. Further, preliminary preprocessing is required:

[0111] S11. Use a 1-45 Hz bandpass filter H(ω), whose frequency response is defined as:

[0112] Y(ω)=X(ω)·H(ω)

[0113] Where X(ω) is the spectrum of the original EEG signal and Y(ω) is the spectrum of the filtered signal.

[0114] S12. Based on the amplitude and waveform stability of the EEG signal, a bad conductor is marked. If the amplitude of the signal is greater than the threshold ∈ over a period of time, it can be marked as a bad conductor:

[0115] ||x(t)|| ∞ ≥∈.

[0116] S13, baseline correction, collect the user's baseline signal b in the resting state, the corrected signal is:

[0117] x′ t , x'(t)=x(t)-b

[0118] Step S2: Generate adversarial network (GAN) to optimize ICA algorithm, see Figure 2 Schematic diagram of the GAN-based optimized ICA algorithm framework, specifically:

[0119] S21. To ensure efficient segmentation of real-time signal streams while maintaining the quality of the segmented signals, a sliding window approach is employed. However, the selection of the sliding window size and step size is also crucial. A window that is too small may not capture sufficient signal features, resulting in poor separation results; a window that is too large may increase the computational burden and introduce signal delay issues; a step size that is too small increases the algorithm's computational complexity; and a step size that is too large may result in discontinuous signal segmentation, affecting the final sentiment classification.

[0120] Finally, the size was chosen:

[0121] T win =1s

[0122] The step size is:

[0123] Δt=0.5s.

[0124] Then the final segmentation signal is:

[0125] X t ={x(t),x(t+1),...,x(t+T win -1)},t∈Z

[0126] where X t Represented as the signal of the t-th window.

[0127] S22. The generative adversarial network consists of a generator G and a discriminator D. The purpose of the generator is to generate a pseudo signal S(t) that is close to the real one (such as electrooculogram, electromyography, etc.).

[0128] S(t)=G(z)

[0129] Where G(z) represents the output of the generator and z represents random noise.

[0130] The purpose of the discriminator is to distinguish between the generated artifact signal and the real signal. The discriminator will perform a binary judgment on each signal x(t) and output the result D(x), which represents the probability that the signal is a real signal.

[0131]

[0132] Where W D represents the weight of the discriminator, b D is the bias term.

[0133] At the same time, there are certain challenges. Because GAN training is an adversarial process involving continuous iterations of the generator and discriminator, GAN training typically requires a significant amount of computing resources and time. To ensure the real-time performance of the algorithm, the GAN model needs to be pre-trained on large-scale datasets, eliminating the need to train from scratch. Based on the signal, only the loss function needs to be fine-tuned to ensure that the GAN training process converges quickly. The following is the GAN loss function:

[0134] L G = -E[logD(G(z))]

[0135] L D =-E[logD(x(t)))]-E[log(1-D(G(z)))]

[0136] During S23 and GAN training, the generated artifact signal S(t) is input into the ICA algorithm together with the original EEG signal. The artifact signal is removed by blind source separation to obtain the optimized independent component S optimizied (t).

[0137] X(t)=W optimized ·S optimizied (t)

[0138] Where W optimized Represented as the optimized unmixing matrix. By adversarially training the generator and discriminator, the blind source separation process of the ICA algorithm can be effectively optimized, artifacts in EEG signals can be removed, and the accuracy and robustness of emotion monitoring can be improved.

[0139] S3. Use CNN to extract the spatial features of EEG signals, and use the self-attention mechanism to extract the temporal features of EEG signals.

[0140] S31. Spatial feature extraction. Assume that the device worn by the user has m electrodes to collect EEG signals. Map the EEG signals x(t) to the corresponding electrode positions to form a two-dimensional matrix:

[0141]

[0142] Two convolution kernels K1 = 3 × 3 and K2 = 2 × 2 are used to extract spatial features of different scales. At the same time, in order to retain more spatial information, the pooling layer is not used. The ReLU activation function is used, and the final feature matrix F spatial for:

[0143] F spatial =ReLU(X conv )

[0144] Finally, a flattening operation is required to convert the two-dimensional feature vector into a one-dimensional vector F spatial

[0145] F spatial =Flatten(F spatial )

[0146] S32, temporal feature extraction, uses the self-attention mechanism to extract temporal features. The signal x(t) is mapped to the query matrix Q, key matrix K and value matrix V:

[0147] Q = x t w q

[0148] K=x t w k

[0149] V=x t w v

[0150] where w q 、w k 、w v It is the weight matrix obtained by pre-learning. Then the attention score AttentionScore is calculated and normalized and weighted summed to obtain the final time feature F time ,

[0151]

[0152] F time =Attention Weights×V

[0153] S4, deeply fuse the time and space features to form the final feature vector F,

[0154] F=[F spatial ||F time ]

[0155] Among them, || represents the concatenation of features. Finally, the feature vector F is input into the Softmax classifier to obtain the predicted probability P(K) of each emotion category:

[0156]

[0157] where θ k It is expressed as the weight of the k-th category emotion, k is the number of categories, and P(K) represents the predicted value of the K-th category emotion.

[0158] Measure the probability, and finally, select the emotion category y with the highest probability k As a result of emotion monitoring:

[0159] y k =arg max k P(K)

[0160] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. The sentiment monitoring method based on online independent component analysis is characterized by: The steps include: S1, obtains the user's real-time brain signal and performs preliminary preprocessing, including filtering, removing bad signals and removing baselines; S2, the pre-processed brain signal is segmented according to the sliding window, and the segmented data is fed into the generative adversarial network model to generate similar artifact signals and perform independent component analysis together with the segmented data to remove artifacts; S3, using convolutional neural networks and self-attention mechanisms to extract spatial and temporal features of EEG signals from the obtained data; S4, deeply fuses the extracted spatial and temporal features and inputs them into the softmax classifier for sentiment classification to achieve online real-time sentiment monitoring.

2. The sentiment monitoring method based on online independent component analysis according to claim 1, characterized in that: Step S1, preliminary preprocessing includes the following: (1) Filtering is performed using a 1-45 Hz bandpass filter; (2) Based on the amplitude and waveform stability of the EEG signal, the bad conductor is marked; (3) Select a certain section of static state signal, calculate the baseline signal, and subtract the baseline value; T is the time of a certain period of rest, and the subsequent EEG signals are subtracted from the baseline value.

3. The sentiment monitoring method based on online independent component analysis according to claim 1, characterized in that: In step S2, the sliding window size is set to 1s, the step size is set to 0.5s, and the segmented EEG signal is: X={x1,x2,...,x t } T represents the tth sliding window; Using real EEG signals, GAN is used to generate artifact signals. The generator is trained to generate artifact signals that are statistically similar to the real signals. The generated artifact signals are compared with the real EEG signals. ICA is used to remove the artifacts of electrooculogram and electromyography signals from these signals to improve the quality and efficiency of artifact removal.

4. The sentiment monitoring method based on online independent component analysis according to claim 1, characterized in that: Step S3: First, the EEG signal X processed in step S2 is converted into {x1, x2, ..., x t }, where x t represents the t-th sliding window; The segmented EEG signals are fed into a Generative Adversarial Network model, where the generator generates artifact signals as much as possible. The discriminator distinguishes between the artifact signals generated by the generator and the real EEG signals. On this basis, the generator is continuously optimized to generate artifact signals that are close to the real ones. The generated artifact signal and the segmented original EEG signal are subjected to independent component analysis together, so that the signal after ICA processing can effectively remove endogenous signals (such as electrooculogram, electromyography, etc.) and other interference signals; the independent component analysis algorithm optimized by GAN can further extract the real EEG signal and improve the robustness.

5. The sentiment monitoring method based on online independent component analysis according to claim 1, characterized in that: Step S3: Map the one-dimensional EEG signal processed in step 2 into a two-dimensional matrix according to the electrode position of the EEG signal acquisition device: m represents the mth electrode; Use convolutional neural network to process the two-dimensional signal frame X t For spatial feature extraction, use a 3×3 convolution kernel and a 2×2 convolution kernel, without using pooling operations to retain more spatial feature information: F1=Conv(X,W1,b1) F2=Conv(X,W2,b2) W is the convolutional layer, b is the bias term, and F is the output feature map; Use the self-attention mechanism to extract the temporal features of the EEG signal processed in step 2, and perform weighted summation to obtain the final temporal feature vector F t , the EEG signal is mapped to three different matrices with different weights: Query matrix (Q): Q = x t w q ; Key matrix (K): K = x t w k ; Value matrix (V) V = x t w v ; where w q , w k , w v is the weight matrix obtained through learning; Then calculate the similarity of the sliding window, that is, the attention score: And perform normalization: The final output matrix V is weighted summed to obtain the final time eigenvector: Output=Attention Weights×V.

6. The sentiment monitoring method based on online independent component analysis according to claim 1, characterized in that: Step S3, the first convolution kernel: F1=ReLU(W1*X(t)+b1) Among them, * represents the convolution operation, ReLU is the activation function, F1 is the feature matrix after convolution, and b1 is the bias term; The second convolution kernel: F2=ReLU(W2*F1+b2) At the same time, the pooling operation is not used to ensure the maximum retention of spatial features, and finally the extracted spatial feature matrix F2 is flattened into a one-dimensional vector F s .

7. The sentiment monitoring method based on online independent component analysis according to claim 1, characterized in that: Step S4: The spatial and temporal features extracted from CNN and self-attention mechanism are F s and F t And the temporal feature F of the EEG signal is extracted through the self-attention mechanism t , the spatial and temporal features are spliced ​​to form a new fusion feature vector F f , F f =[F s ||F t ],|| represents the feature concatenation operation; The fusion feature is expressed as: F f =concat(F s ,F t ) Finally, F f Input softmax classifier for sentiment classification: θ k Expressed as the weight of the k-th category sentiment, k is the number of categories, y k It represents the predicted probability of the k-th emotion, and finally selects the emotion with the largest predicted probability as the output.

Images