Reinforcement learning sentiment analysis method and device for reading evoked electroencephalogram

Abstract

The invention discloses a reinforcement learning sentiment analysis method and device for reading evoked electroencephalogram, and relates to the technical field of natural language understanding and cognitive neuroscience. The method comprises the steps of obtaining evoked electroencephalogram fragments corresponding to text segmented words based on reading eye movement information; inputting electroencephalogram fragments to an electroencephalogram word segmentation device, a quantizer and an encoder by taking a complete semantic sentence as a unit, and pre-training the modules through a self-supervised reconstruction normal form; an intelligent agent is activated by using the discrete electroencephalogram vectors of the quantizer, a key electroencephalogram vector set is screened, and sentiment classification is deduced based on the set; and a reward function for strengthening the class, diversity and characterization of the electroencephalogram vector set is constructed, and a selection strategy of the intelligent agent is optimized. By adopting the method, a new normal form can be provided for emotion recognition of the natural reading brain-computer interface.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language understanding and cognitive neuroscience, and in particular to a method and device for sentiment analysis based on enhanced learning of brain waves induced by reading. Background Art

[0002] With the development of human-computer interaction technology, the technology of understanding and identifying human emotional states has received widespread attention. Affective computing is an important branch of this field, and its goal is to perceive, understand and express human emotions through computer technology. Among the many emotion recognition methods, electroencephalography (EEG) has become a research hotspot in recent years because of its characteristics of directly reflecting brain neural activity. Compared with traditional emotion recognition methods such as facial expressions and voice, EEG signals have the advantages of anti-masquerade and real-time performance. However, due to the complexity of EEG signals and their strong noise characteristics, accurate analysis and decoding of EEG signals face many challenges.

[0003] At present, most of the research on EEG emotion recognition focuses on emotion classification using EEG signals obtained through video stimulation. Video stimulation has become a common choice in emotion recognition research because it can provide rich multimodal information, such as visual, auditory, and emotional color. However, this approach mainly focuses on video stimulation, resulting in a large gap in emotion recognition research based on natural reading text, especially the lack of an end-to-end emotion recognition system. In the existing technology, although some studies have attempted to combine EEG signals with natural text, there are still some key issues that have not been resolved.

[0004] Currently, some studies have attempted to use the method of generating text from EEG in emotion recognition, that is, generating intermediate text through EEG signals, and then performing emotion classification based on the generated text. The idea of ​​this type of method is to decode EEG signals into text, and then use text classification technology to perform emotion analysis. However, the limitation of this method is that there is a clear semantic and structural gap between the generated text and the actual input text, resulting in the generated text being unable to truly reflect the emotional information of the text stimulus. This gap directly affects the accuracy of emotion recognition, making the emotion recognition method based on generated text often perform poorly in practical applications.

[0005] In addition, since existing emotion recognition models usually rely on an indirect text generation step instead of directly extracting emotional features from EEG signals, this makes the emotion recognition process inefficient and inaccurate. This indirect method of generating text adds additional computational complexity and makes the training process of emotion recognition models more cumbersome. In many cases, the correlation between the generated text and the emotion category is weak, which affects the overall performance of the emotion recognition model. Summary of the invention

[0006] In order to solve the technical problems of low accuracy and high computational complexity of emotion recognition in the prior art, the present invention provides a method and device for emotion analysis based on reinforcement learning of reading-induced EEG. The technical solution is as follows:

[0007] On the one hand, a method for sentiment analysis based on reinforcement learning of reading-induced EEG is provided, which is implemented by a device for sentiment analysis based on reinforcement learning of reading-induced EEG, and comprises:

[0008] S1, obtaining EEG signal training samples, and training the initial EEG word segmenter according to the EEG signal training samples and the EEG signal decoder to obtain a trained EEG word segmenter;

[0009] S2, obtaining two-dimensional EEG signal data, wherein the EEG signal data includes an EEG signal and corresponding text data;

[0010] S3, dividing the EEG signal data into blocks to obtain a plurality of EEG signal segments; wherein each EEG signal segment includes an EEG signal block and a corresponding text word;

[0011] S4, inputting the multiple EEG signal segments into the trained EEG word segmenter for encoding to obtain multiple EEG encoding features;

[0012] S5, quantizing the multiple EEG encoding features into discrete EEG vectors through a quantizer module;

[0013] S6. Extract key features from discrete EEG vectors to obtain the importance score of word embedding;

[0014] S7, inputting the importance score of the word embedding into the intelligent agent, and sampling the multiple EEG signal segments based on action selection through the selection strategy optimized by the intelligent agent's reward function to obtain a keyword set;

[0015] S8. Input the keyword set into the emotion classification module to obtain the emotion classification corresponding to the EEG signal data.

[0016] On the other hand, a device for analyzing the sentiment of reading-induced EEG through reinforcement learning is provided, and the device is applied to a method for analyzing the sentiment of reading-induced EEG through reinforcement learning. The device comprises:

[0017] A training unit is used to obtain EEG signal training samples, and train the initial EEG word segmenter according to the EEG signal training samples and the EEG signal decoder to obtain a trained EEG word segmenter;

[0018] An acquisition unit, used for acquiring two-dimensional EEG signal data, wherein the EEG signal data includes the EEG signal and corresponding text data;

[0019] A block division unit, used for dividing the EEG signal data into blocks to obtain a plurality of EEG signal segments; wherein each EEG signal segment includes an EEG signal block and a corresponding text word;

[0020] An encoding unit, used for inputting a plurality of EEG signal segments into a trained EEG word segmenter for encoding to obtain a plurality of EEG encoding features;

[0021] A quantization unit, used for quantizing a plurality of EEG encoding features into discrete EEG vectors through a quantizer module;

[0022] The feature extraction unit is used to extract key features from discrete EEG vectors and obtain the importance score of word embedding;

[0023] A selection unit, configured to input the importance scores of word embeddings into the intelligent agent, and perform action-selection-based sampling on the plurality of EEG signal segments through a selection strategy optimized by a reward function of the intelligent agent to obtain a keyword set;

[0024] The analysis unit is used to input the keyword set into the emotion classification module to obtain the emotion classification corresponding to the EEG signal data.

[0025] On the other hand, a device for reinforcement learning of sentiment analysis of reading-induced EEG is provided, and the device for reinforcement learning of sentiment analysis of reading-induced EEG comprises: a processor; a memory, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by the processor, any one of the above-mentioned methods for reinforcement learning of sentiment analysis of reading-induced EEG is implemented.

[0026] On the other hand, a computer-readable storage medium is provided, in which at least one instruction is stored, and the at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned reading-induced EEG reinforcement learning sentiment analysis methods.

[0027] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0028] The present invention directly extracts emotion-related features from EEG signals, eliminating the intermediate text generation step in traditional methods. Through this innovative end-to-end framework, emotion recognition can be performed more efficiently and accurately, thus overcoming the limitations of existing technologies and improving the effect and robustness of emotion recognition.

[0029] The accuracy and efficiency of EEG signal analysis have been significantly improved through multi-level technologies such as signal segmentation, feature encoding, quantization, and reinforcement learning. By introducing time and space embedding, the model can fully capture the spatiotemporal dependencies of EEG signals, enhance the understanding of brain activity, and perform well in tasks such as emotion recognition and emotion monitoring. In addition, quantization technology converts EEG signals into discrete vectors, reducing computing resource consumption while improving the robustness of signal processing. The reinforcement learning module further optimizes the feature extraction process, making keyword selection more accurate, thereby improving the accuracy of emotion classification and emotion recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0031] Figure 1 This is a flow chart of a method for sentiment analysis based on enhanced learning of reading-induced EEG provided by an embodiment of the present invention;

[0032] Figure 2 This is a flowchart of a method for sentiment analysis based on enhanced learning of reading-induced EEG provided by an embodiment of the present invention;

[0033] Figure 3 It is a block diagram of a device for emotion analysis based on enhanced learning of reading-induced EEG provided by an embodiment of the present invention;

[0034] Figure 4 It is a structural schematic diagram of a reading-induced EEG reinforcement learning emotion analysis device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0035] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0036] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0037] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0038] In the embodiments of the present invention, sometimes the subscripts such as W 1 It may be written in non-subscript form such as W1. When the difference is not emphasized, the meaning is the same.

[0039] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0040] The embodiment of the present invention provides a method for analyzing the emotion of reading-induced EEG through reinforcement learning. The method can be implemented by a device for analyzing the emotion of reading-induced EEG through reinforcement learning. The device for analyzing the emotion of reading-induced EEG through reinforcement learning can be a terminal or a server. Figure 1 The flowchart of the reinforcement learning sentiment analysis method for reading induced EEG is shown in Figure 2 The flowchart of the method for sentiment analysis based on the reinforcement learning of reading-induced EEG is shown. The processing flow of the method may include the following steps:

[0041] S1. Obtain EEG signal training samples, and train an initial EEG word segmenter according to the EEG signal training samples and an EEG signal decoder to obtain a trained EEG word segmenter.

[0042] In a feasible implementation, unlike images with high signal-to-noise ratio, EEG signals have low signal-to-noise ratio and obvious randomness, non-stationarity and nonlinear characteristics, which makes it difficult to reconstruct the original signal well. It is difficult to converge by directly reconstructing the original EEG signal. The embodiment of the present invention reconstructs the amplitude and phase from discrete neural markers to train the EEG word segmenter.

[0043] Optionally, the training process of S1 may include S11-S17:

[0044] S11, obtaining an EEG signal training sample, where the EEG signal training sample includes EEG signal training data, amplitude true value, phase true value, and EEG feature true value;

[0045] S12, dividing the EEG signal training data into blocks to obtain multiple EEG signal training segments;

[0046] S13, inputting the multiple EEG signal training segments into the initial EEG word segmenter for encoding, and obtaining multiple predicted EEG encoding features;

[0047] S14, quantizing the plurality of predicted EEG encoding features into discrete predicted EEG vectors through a quantizer module;

[0048] In a feasible implementation, for an EEG sample x containing c channels and an EEG block x with sampling point t c,k , apply Discrete Fourier Transform (DFT):

[0049]

[0050] Where m∈[1,N], j is an imaginary unit. According to Euler's formula, we can get The amplitude A m and phase φ m :

[0051]

[0052] where Re and Im represent the real and imaginary parts of the complex number.

[0053] S15. Input the discrete predicted EEG vector into the EEG signal decoder to obtain the predicted amplitude and predicted phase.

[0054] In one feasible implementation, after being quantized by a quantizer, the normalized discrete EEG embedding is input into a neural decoder comprising multiple Transformer blocks. The decoder output passes through an average pooling layer and two prediction heads to regress and predict the spectral amplitude and phase respectively.

[0055] S16. Calculate the mean square error loss function based on the predicted EEG coding features, the true value of the EEG features, the predicted amplitude, the true value of the amplitude, the predicted phase, the true value of the phase and the following formula (1):

[0056]

[0057]

[0058] Among them, D is all EEG signal training data, sg indicates stopping gradient calculation, the first two items are the error between the predicted amplitude and the actual amplitude and the error between the predicted phase and the actual phase, and the last two items are the error between the EEG codebook and the encoded features. Since the conversion between EEG embedding and EEG codebook is discrete, the gradient cannot be back-propagated between the EEG codebook and the encoded features, so this part is only forward-transferred during the back-propagation process, and the gradient is 0 during the reverse process.

[0059] S17. Perform iterative training of forward propagation feedback according to the mean square error loss function. When the number of iterations reaches a preset threshold, stop the training to obtain a trained EEG word segmenter.

[0060] S2. Acquire two-dimensional EEG signal data, where the EEG signal data includes the EEG signal and corresponding text data.

[0061] In one feasible implementation, EEG data is obtained by using a multi-channel EEG device to collect EEG signals, with each channel recording the electrical activity of different brain regions. The obtained EEG signal is defined as a two-dimensional data representation, denoted as Where C is the number of EEG channels and T is the number of time frames. This representation can effectively capture the dynamic changes of EEG signals over time and the spatial information between different electrode channels.

[0062] S3, dividing the EEG signal data into blocks to obtain a plurality of EEG signal segments, wherein each EEG signal segment includes an EEG signal block and a corresponding text word.

[0063] Optionally, the specific operation steps of S3 may be as follows:

[0064] The EEG signal E is divided into multiple time windows of fixed size, and each EEG signal data is divided into segments, where each time window is regarded as an EEG segment containing several time frames, expressed as Where t is the length of the time window, there is no overlap between time windows, and the stride is s, that is, the starting position of each time window is the offset of the previous time window;

[0065] The segmented EEG signal fragments are expressed as the following formula (2):

[0066]

[0067] in, represents the signal segment of the jth electrode channel in the kth time window, and w is the length of each time window.

[0068] In a feasible implementation, the EEG signal is usually a long time series with a huge amount of data and rich information. However, directly inputting the entire time series into the model may result in excessive computation, so block processing is a commonly used signal processing technique. Inspired by the image patch embedding method, this embodiment regards the EEG signal as two-dimensional data, and the local features in the EEG signal can be better captured through block processing. Specifically, the EEG signal E is divided into multiple time windows of fixed size. Each time window can be regarded as an EEG segment containing several time frames, expressed as Where t is the length of the time window (e.g. 100ms, 200ms, etc.). In order to effectively capture the dynamic changes in the signal, the time windows are usually non-overlapping, with a stride of s, which is the offset of the starting position of each time window relative to the previous time window. In this way, each EEG sample will be divided into The segmented EEG segments are expressed as above formula (2). Each time window captures the local changes of the EEG signal at a certain moment, which helps to capture the brain activity pattern in a short time.

[0069] S4. Input multiple EEG signal segments into the trained EEG word segmenter for encoding to obtain multiple EEG encoding features.

[0070] Optionally, the EEG word segmenter may include a temporal encoder module, a temporal and spatial embedding module, and an improved Transformer encoder module;

[0071] The specific operation steps of S4 may include S41-S43:

[0072] S41, inputting multiple EEG signal segments into a time encoder module, and obtaining corresponding temporal coding features for each segment;

[0073] S42, optimizing the time embedding list according to the temporal coding feature, adding the optimized time embedding list, the initialized space embedding list and the EEG signal fragment to obtain an embedding sequence;

[0074] S43. Input the embedded sequence into the improved Transformer encoder module to obtain multiple EEG encoding features.

[0075] Optionally, S43 inputs the embedded sequence into the improved Transformer encoder module to obtain multiple EEG encoding features, including:

[0076] S431, input the embedded sequence into the improved Transformer encoder module, and obtain the query vector Q, the key vector K and the numerical vector V by linearly transforming the embedded sequence;

[0077] S432, perform layer normalization on the query vector Q and the key vector K respectively, and omit the deviation term in the QKV calculation;

[0078] S433. Transpose the normalized key vector K, perform dot product on the normalized query vector Q and the transposed key vector K to obtain the correlation between Q and K, scale the dot product result, apply the softmax function to convert the scaled correlation into a probability distribution to obtain weights, apply the weights to the numerical vector V, obtain the weighted summation result, and then obtain multiple EEG coding features.

[0079] In a feasible implementation, the temporal encoder is composed of multiple temporal convolutional blocks, which are composed of a one-dimensional convolutional layer, a group normalization layer, and a GELU activation function. This architecture can capture the local characteristics of EEG signals in the temporal dimension. For each block, the corresponding temporal coding feature is obtained:

[0080]

[0081] Furthermore, in order to enable the model to fully understand the temporal and spatial information of the block embedding, this embodiment initializes two learnable embedding lists: the temporal embedding list and the spatial embedding list S E =se 1 ,se 2 ,...,se |C| , the embedding vectors of these two lists are both d-dimensional, and d is 512 in this embodiment. max represents the maximum number of time blocks, which is a hyperparameter and needs to satisfy The spatial embedding list is used to represent the spatial position of each EEG electrode channel. C is the number of electrode channels, which is 105 in this embodiment. After embedding, the EEG feature is expressed as:

[0082]

[0083] Time Embedding k Able to provide the model with representation of temporal information and spatial embedding This provides the spatial information of the electrode position, embedding the two items and the features obtained in the previous step Add them together to get e emb These embeddings are optimized as learnable parameters during training, allowing the model to gradually learn temporal and spatial correlations and effectively incorporate them into the patch embedding representation.

[0084] Furthermore, the embedding sequence will be directly input into the Transformer encoder. In order to make the training of Transformer more stable and efficient, appropriate modifications are made on its basis. First, layer normalization is added to the query and key before the dot product attention mechanism, which avoids the attention logic value from being too large, where dhead is the dimension of a head in the multi-head attention, and LN represents layerNorm. Next, the bias term in the QKV calculation is omitted, which can speed up training without degrading performance. For downstream tasks, average pooling is used on the output embedding, followed by a task-specific prediction head. The calculation process is as follows:

[0085]

[0086] First, the query matrix Q and the key matrix K are dot-producted to obtain the correlation between them. Then, the dot product result is scaled to stabilize the calculation. Next, a softmax function is applied to convert the correlation into a probability distribution, representing the importance of each position. Finally, these weights are applied to the value matrix V to obtain the weighted summation result, thereby generating a new representation for each position. This process enables the model to capture dependencies between different input positions.

[0087] S5. Through the quantizer module, multiple EEG encoding features are quantized into discrete EEG vectors.

[0088] Optionally, S5 quantizes multiple EEG encoding features into discrete EEG vectors through a quantizer module, including:

[0089] Define a neural codebook V′={v i |i=1,...,K}∈R KD , where K is the number of discrete EEG vectors and D is the dimension of each vector;

[0090] In the neural codebook V′, the nearest vector of each EEG coding feature pi is found as the potential discrete representation of the EEG coding feature. This process is expressed as the following formula (3):

[0091]

[0092] Among them, l 2 Indicates e 2 Normalization, z i is the discretized EEG vector after the quantizer.

[0093] S6. Extract key features from discrete EEG vectors to obtain the importance score of word embedding.

[0094] In a feasible implementation, reinforcement learning is used to extract key features from the discrete EEG vector obtained in step S5, and the agent learns the importance score P of each word embedding in emotion discrimination through a multi-layer Transformer network:

[0095] P = {p t ∣p t ∈(0,1),t=1,...,T}

[0096] Where T represents the total number of words in the text, p t Represents the importance score of the tth word.

[0097] S7. The importance scores of the word embeddings are input into the intelligent agent, and the action-selection-based sampling is performed on multiple EEG signal segments through the selection strategy optimized by the intelligent agent's reward function to obtain a keyword set.

[0098] In a feasible implementation, according to the selected keyword set, corresponding rewards are provided to the agent to encourage it to select more representative keywords. The specific reward mechanism can be designed to be based on the emotional intensity and relevance of the keywords, so as to guide the agent to continuously optimize its selection strategy in subsequent iterations. The reward function is designed with enhanced rewards, discrete rewards and representative rewards. The key features in emotion recognition are screened through the reinforcement learning mechanism, and enhanced rewards, discrete reward functions and representative rewards are designed to improve the accuracy and robustness of emotion classification.

[0099] Optionally, the reward function includes an enhanced reward, a discrete reward, and a representative reward. The following describes the three reward functions respectively:

[0100] 1) Enhanced reward calculation method, including:

[0101] According to the following formula (4), the contribution of the selected keyword set to sentiment classification is measured by comparing the prediction probability at the sentence level:

[0102]

[0103] Among them, c is the true label of the word, p c represents the probability of predicting a word set as class c. 0 represents the sentence-level prediction of all words. The calculation process is to calculate the accuracy of the average EEG prediction of all word-level words, p 1 The sentence-level prediction of the keyword set is to calculate the prediction accuracy of each word-level EEG after taking action. is the feature vector of the tth word, a t Indicates whether the agent chooses a word at time t, a t =1 means selection, a t =0 means not selected.

[0104] In one feasible implementation, when the enhancement reward r ren When the value of is positive, it means that the selected keyword set has a positive contribution to the accuracy of sentiment classification, otherwise it indicates a negative contribution. Specifically, if the predicted action turns from the wrong label to the correct label, the agent will be given a positive reward r = Ω, otherwise it will be punished r = -Ω. In the embodiment of the present invention, Ω is set to 10 as the scale of positive and negative rewards.

[0105] 2) The calculation method of discrete rewards, including:

[0106] In order to ensure that the selected keyword set is uniform, the embodiment of the present invention uses a decentralized reward function to achieve better selection results. First, a standard function that reflects the distance within and between classes is constructed. A set with a smaller intra-class distance and a larger inter-class distance (i.e., the classification result should have a higher degree of discrimination) is selected through clustering results. A distribution with a higher decentralized reward shows significantly better clustering than a distribution with a lower decentralized reward.

[0107] The intra-class deviation matrix is ​​generated by calculating the distance from each sample point to the cluster center according to the following formula (5):

[0108]

[0109] in, is the sample mean vector of class w, n s represents the number of samples of class w, c represents the number of categories of classification, represents the i-th data sample in the j-th class, and the formula is to calculate the intra-class covariance matrix of the j-th class By calculating the deviation of each sample from the corresponding class mean (i.e. ) and average all samples to get

[0110] The inter-class distance criterion function generates the inter-class deviation matrix S B , defined as follows (6):

[0111]

[0112] Among them, c represents the number of classes to be classified, is the mean vector of all samples to be classified, represents the number of samples of class w, represents the total number of samples, Represents the class center of each class, through the mean vector of each class j and the global mean The difference between them is taken as the outer product and weighted summation to get S B ;

[0113] Discrete reward R based on intra-class distance criterion function and inter-class distance criterion function dis , defined as follows (7):

[0114]

[0115] where Tr[**] is the trace of the computation matrix.

[0116] 3) The calculation method of representative rewards includes:

[0117] Representative reward R repIt measures the extent to which the selected keyword set S can represent the characteristics of the original sentence. By maximizing Rrep, the agent is encouraged to select keywords close to the cluster center of the feature space.

[0118] According to the following formula (8), the extent to which the selected keyword set represents the characteristics of the original sentence is measured:

[0119]

[0120] Among them, x t represents the feature of the tth word in the sentence, T represents the number of words in the entire sentence, Y represents the index set of the keyword set, min t′∈Y Indicates that the feature most similar to the current word is selected from the selected keyword. This formula ensures that the selected keyword can accurately represent the sentence semantics by calculating the Euclidean distance between the original sentence feature and the selected keyword feature.

[0121] In a feasible implementation, after the reward function is determined, the reward function generates a reward value by evaluating the performance of the keyword set S screened by the action set in the sentiment classification task. The reward value consists of two parts: an enhanced reward and a discrete reward. The generated reward value is used as a feedback signal to train the Transformer network through the back propagation mechanism to optimize the importance score p. This process improves the subsequent action selection strategy, thereby gradually improving the accuracy and efficiency of keyword screening. The agent samples according to the importance score P and generates an action set A:

[0122] A={a t ∣a t ∈{0,1},t=1,…,T}

[0123] These actions are applied to word embeddings to filter out the set of keywords S:

[0124]

[0125] After obtaining the keyword set, it will be used as new input for sentiment classification.

[0126] S8. Input the keyword set into the emotion classification module to obtain the emotion classification corresponding to the EEG signal data.

[0127] In a feasible implementation, the sentiment classification module can be a fully connected (FC) layer followed by an activation function (such as ReLU or Softmax). In this module, the keyword set is processed by the fully connected layer and nonlinearly transformed by the activation function, thereby realizing the sentiment classification of the EEG signal data. The sentiment classification module can output the corresponding sentiment classification result based on the features in the keyword set.

[0128] The following is an explanation of the performance verification:

[0129] The embodiment of the present invention has been verified on the public EEG text dataset ZuCo containing 309 data. In the end-to-end EEG emotion classification results, the precision rate was 46.94%, the recall rate was 41.69%, the F1 score was 44.19%, and the accuracy rate was 42.60%. After using reinforcement learning to extract key fragments, the precision rate was 45.63%, the recall rate was 43.24%, the F1 score was 44.87%, and the accuracy rate was 43.38%. This embodiment has a reward function in the fields of brain-computer interface, intelligent health monitoring, sentiment analysis and neuroscience research. The performance of the keyword set SSS screened out by the action set is evaluated in the sentiment classification task, specifically including calculating the matching degree between the sentiment classification result and the target sentiment label. The reward value is used as a feedback signal to adjust the parameters of the Bernoulli distribution, thereby optimizing the subsequent action selection strategy and gradually improving the accuracy and effectiveness of keyword screening. It has a wide range of application prospects and can promote technological innovation and development in related industries.

[0130] The present invention directly extracts emotion-related features from EEG signals, eliminating the intermediate text generation step in traditional methods. Through this innovative end-to-end framework, emotion recognition can be performed more efficiently and accurately, thus overcoming the limitations of existing technologies and improving the effect and robustness of emotion recognition.

[0131] The accuracy and efficiency of EEG signal analysis have been significantly improved through multi-level technologies such as signal segmentation, feature encoding, quantization, and reinforcement learning. By introducing time and space embedding, the model can fully capture the spatiotemporal dependencies of EEG signals, enhance the understanding of brain activity, and perform well in tasks such as emotion recognition and emotion monitoring. In addition, quantization technology converts EEG signals into discrete vectors, reducing computing resource consumption while improving the robustness of signal processing. The reinforcement learning module further optimizes the feature extraction process, making keyword selection more accurate, thereby improving the accuracy of emotion classification and emotion recognition.

[0132] Figure 3 1 is a block diagram of a device for analyzing the emotion of reading-induced EEG through reinforcement learning according to an exemplary embodiment, wherein the device is used for analyzing the emotion of reading-induced EEG through reinforcement learning. Figure 3 The device includes a training unit 310, an acquisition unit 320, a block unit 330, an encoding unit 340, a quantization unit 350, a feature extraction unit 360, a selection unit 370 and an analysis unit 380. Among them:

[0133] The training unit 310 is used to obtain EEG signal training samples, and train the initial EEG word segmenter according to the EEG signal training samples and the EEG signal decoder to obtain a trained EEG word segmenter;

[0134] An acquisition unit 320, configured to acquire two-dimensional EEG signal data, wherein the EEG signal data includes an EEG signal and corresponding text data;

[0135] A block division unit 330 is used to divide the EEG signal data into blocks to obtain a plurality of EEG signal segments; wherein each EEG signal segment includes an EEG signal block and a corresponding text word;

[0136] The encoding unit 340 is used to input the multiple EEG signal segments into the trained EEG word segmenter for encoding to obtain multiple EEG encoding features;

[0137] A quantization unit 350, configured to quantize a plurality of EEG coding features into discrete EEG vectors through a quantizer module;

[0138] A feature extraction unit 360 is used to extract key features from discrete EEG vectors to obtain importance scores of word embeddings;

[0139] A selection unit 370, configured to input the importance scores of the word embeddings into the agent, and perform action-selection-based sampling on the plurality of EEG signal segments through a selection strategy optimized by a reward function of the agent to obtain a keyword set;

[0140] The analysis unit 380 is used to input the keyword set into the emotion classification module to obtain the emotion classification corresponding to the EEG signal data.

[0141] Optionally, the training unit 310 is used to:

[0142] S11, obtaining an EEG signal training sample, wherein the EEG signal training sample includes EEG signal training data, amplitude true value, phase true value and EEG feature true value;

[0143] S12, dividing the EEG signal training data into blocks to obtain multiple EEG signal training segments;

[0144] S13, inputting the multiple EEG signal training segments into the initial EEG word segmenter for encoding, and obtaining multiple predicted EEG encoding features;

[0145] S14, quantizing the plurality of predicted EEG encoding features into discrete predicted EEG vectors through a quantizer module;

[0146] S15, inputting the discrete predicted EEG vector into an EEG signal decoder to obtain a predicted amplitude and a predicted phase;

[0147] S16. Calculate the mean square error loss function based on the predicted EEG coding features, the true value of the EEG features, the predicted amplitude, the true value of the amplitude, the predicted phase, the true value of the phase and the following formula (1):

[0148]

[0149] in,

[0150]

[0151] S17. Perform iterative training of forward propagation feedback according to the mean square error loss function. When the number of iterations reaches a preset threshold, stop the training to obtain a trained EEG word segmenter.

[0152] Optionally, the block division unit 330 is used to:

[0153] The EEG signal E is divided into multiple time windows of fixed size, and each EEG signal data is divided into segments, where each time window is regarded as an EEG segment containing several time frames, expressed as Where t is the length of the time window, there is no overlap between time windows, and the stride is s, that is, the starting position of each time window is the offset of the previous time window;

[0154] The segmented EEG signal fragments are expressed as the following formula (2):

[0155]

[0156] in, represents the signal segment of the jth electrode channel in the kth time window, and w is the length of each time window.

[0157] Optionally, the EEG word segmenter includes a time encoder module, a time and space embedding module and an improved Transformer encoder module;

[0158] The encoding unit 340 is used to:

[0159] S41, inputting multiple EEG signal segments into a time encoder module, and obtaining corresponding temporal coding features for each segment;

[0160] S42, optimizing the time embedding list according to the temporal coding feature, adding the optimized time embedding list, the initialized space embedding list and the EEG signal fragment to obtain an embedding sequence;

[0161] S43. Input the embedded sequence into the improved Transformer encoder module to obtain multiple EEG encoding features.

[0162] Optionally, the encoding unit 240 is configured to:

[0163] S431, input the embedded sequence into the improved Transformer encoder module, and obtain the query vector Q, the key vector K and the numerical vector V by linearly transforming the embedded sequence;

[0164] S432, perform layer normalization on the query vector Q and the key vector K respectively, and omit the deviation term in the QKV calculation;

[0165] S433. Transpose the normalized key vector K, perform dot product on the normalized query vector Q and the transposed key vector K to obtain the correlation between Q and K, scale the dot product result, apply the softmax function to convert the scaled correlation into a probability distribution to obtain weights, apply the weights to the numerical vector V, obtain the weighted summation result, and then obtain multiple EEG coding features.

[0166] Optionally, the quantization unit 350 is configured to:

[0167] Define a neural codebook V′={v i |i=1,...,K}∈R KD , where K is the number of discrete EEG vectors and D is the dimension of each vector;

[0168] The nearest vector of each EEG coding feature p is found in the neural codebook V′ as the potential discrete representation of the EEG coding feature. This process is expressed as the following formula (3):

[0169] Among them, e 2 Indicates e 2 Normalization, z i is the discretized EEG vector after the quantizer.

[0170] Optionally, the reward function includes an enhanced reward and a discrete reward;

[0171] The calculation method of the enhancement reward includes:

[0172] According to the following formula (4), the contribution of the selected keyword set to sentiment classification is measured by comparing the prediction probability at the sentence level:

[0173]

[0174] Among them, c is the true label of the word, p c represents the probability of predicting a word set as class c. 0 represents the sentence-level prediction of all words. The calculation process is to calculate the accuracy of the average EEG prediction of all word-level words, p1 The sentence-level prediction of the keyword set is to calculate the prediction accuracy of each word-level EEG after taking action. is the feature vector of the tth word, a t Indicates whether the agent chooses a word at time t, a t =1 means selection, a t =0 means no selection;

[0175] The calculation method of the discrete reward includes:

[0176] The intra-class deviation matrix is ​​generated by calculating the distance from each sample point to the cluster center according to the following formula (5):

[0177]

[0178] in, is the sample mean vector of class w, n s represents the number of samples of class w, c represents the number of categories of classification, represents the i-th data sample in the j-th class, and the formula is to calculate the intra-class covariance matrix of the j-th class By calculating the deviation of each sample from the corresponding class mean (i.e. ) and average all samples to get

[0179] The inter-class distance criterion function generates the inter-class deviation matrix S B , defined as follows (6):

[0180]

[0181] Among them, c represents the number of classes to be classified, is the mean vector of all samples to be classified, represents the number of samples of class w, represents the total number of samples, Represents the class center of each class, through the mean vector of each class j and the global mean The difference between them is taken as the outer product and weighted summation to get S B ;

[0182] Discrete reward R based on intra-class distance criterion function and inter-class distance criterion function dis , defined as follows (7):

[0183]

[0184] Where Tr[**] is the trace of the computation matrix;

[0185] The calculation method of the representative reward includes:

[0186] According to the following formula (8), the extent to which the selected keyword set represents the characteristics of the original sentence is measured:

[0187]

[0188] Among them, x t represents the feature of the tth word in the sentence, T represents the number of words in the entire sentence, Y represents the index set of the keyword set, min t′∈Y Indicates selecting the feature most similar to the current word from the selected keywords.

[0189] The present invention directly extracts emotion-related features from EEG signals, eliminating the intermediate text generation step in traditional methods. Through this innovative end-to-end framework, emotion recognition can be performed more efficiently and accurately, thus overcoming the limitations of existing technologies and improving the effect and robustness of emotion recognition.

[0190] The accuracy and efficiency of EEG signal analysis have been significantly improved through multi-level technologies such as signal segmentation, feature encoding, quantization, and reinforcement learning. By introducing time and space embedding, the model can fully capture the spatiotemporal dependencies of EEG signals, enhance the understanding of brain activity, and perform well in tasks such as emotion recognition and emotion monitoring. In addition, quantization technology converts EEG signals into discrete vectors, reducing computing resource consumption while improving the robustness of signal processing. The reinforcement learning module further optimizes the feature extraction process, making keyword selection more accurate, thereby improving the accuracy of emotion classification and emotion recognition.

[0191] Figure 4 is a structural diagram of a device for reinforcing learning and analyzing emotion induced by reading EEG provided by an embodiment of the present invention, such as Figure 4 As shown, the reading-induced EEG reinforcement learning sentiment analysis device may include the above Figure 3 The device for analyzing the emotion through reinforcement learning of reading-induced EEG is shown. Optionally, the device for analyzing the emotion through reinforcement learning of reading-induced EEG 410 may include a first processor 2001 .

[0192] Optionally, the reading induced EEG reinforcement learning emotion analysis device 410 may also include a memory 2002 and a transceiver 2003 .

[0193] The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.

[0194] Combine the following Figure 4 The components of the reading-induced EEG reinforcement learning emotion analysis device 410 are specifically introduced:

[0195] The first processor 2001 is the control center of the reading induced EEG reinforcement learning emotion analysis device 410, which can be a processor or a general term for multiple processing elements. For example, the first processor 2001 is one or more central processing units (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiment of the present invention, such as one or more microprocessors (digital signal processor, DSP), or one or more field programmable gate arrays (field programmable gate array, FPGA).

[0196] Optionally, the first processor 2001 can perform various functions of the reading induced EEG reinforcement learning emotion analysis device 410 by running or executing a software program stored in the memory 2002 and calling data stored in the memory 2002.

[0197] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Figure 4 CPU0 and CPU1 are shown in FIG.

[0198] In a specific implementation, as an embodiment, the reading induced EEG reinforcement learning emotion analysis device 410 may also include multiple processors, such as Figure 4 The first processor 2001 and the second processor 2004 are shown in FIG. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0199] The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled to be executed by the first processor 2001. The specific implementation method can refer to the above method embodiment, which will not be repeated here.

[0200] Optionally, the memory 2002 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001, or may exist independently, and may be read through the interface circuit ( Figure 4 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0201] The transceiver 2003 is used to communicate with a network device or a terminal device.

[0202] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 4 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0203] Optionally, the transceiver 2003 can be integrated with the first processor 2001, or can exist independently, and read the interface circuit ( Figure 4 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0204] It should be noted that Figure 4 The structure of the reading induced EEG reinforcement learning sentiment analysis device 410 shown in the figure does not constitute a limitation on the router. The actual knowledge structure recognition device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0205] In addition, the technical effects of the reading-induced EEG reinforcement learning emotion analysis device 410 can refer to the technical effects of the reading-induced EEG reinforcement learning emotion analysis method described in the above method embodiment, and will not be repeated here.

[0206] It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0207] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0208] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0209] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0210] In the present invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0211] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0212] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0213] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0214] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0215] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0216] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0217] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0218] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for sentiment analysis based on reinforcement learning of reading-induced EEG, characterized in that: The method comprises: S1, obtaining EEG signal training samples, and training the initial EEG word segmenter according to the EEG signal training samples and the EEG signal decoder to obtain a trained EEG word segmenter; S2, obtaining two-dimensional EEG signal data, wherein the EEG signal data includes an EEG signal and corresponding text data; S3, dividing the EEG signal data into blocks to obtain a plurality of EEG signal segments; wherein each EEG signal segment includes an EEG signal block and a corresponding text word; S4, inputting the multiple EEG signal segments into the trained EEG word segmenter for encoding to obtain multiple EEG encoding features; S5, quantizing the multiple EEG encoding features into discrete EEG vectors through a quantizer module; S6. Extract key features from discrete EEG vectors to obtain the importance score of word embedding; S7, inputting the importance score of the word embedding into the intelligent agent, and sampling the multiple EEG signal segments based on action selection through the selection strategy optimized by the intelligent agent's reward function to obtain a keyword set; S8. Input the keyword set into the emotion classification module to obtain the emotion classification corresponding to the EEG signal data.

2. The method for analyzing the emotion of reading-induced EEG by reinforcement learning according to claim 1, characterized in that: The step S1 of acquiring an EEG signal training sample, and training an initial EEG word segmenter according to the EEG signal training sample and an EEG signal decoder to obtain a trained EEG word segmenter includes: S11, obtaining an EEG signal training sample, wherein the EEG signal training sample includes EEG signal training data, amplitude true value, phase true value and EEG feature true value; S12, dividing the EEG signal training data into blocks to obtain multiple EEG signal training segments; S13, inputting the multiple EEG signal training segments into the initial EEG word segmenter for encoding, and obtaining multiple predicted EEG encoding features; S14, quantizing the plurality of predicted EEG encoding features into discrete predicted EEG vectors through a quantizer module; S15, inputting the discrete predicted EEG vector into an EEG signal decoder to obtain a predicted amplitude and a predicted phase; S16. Calculate the mean square error loss function based on the predicted EEG coding features, the true value of the EEG features, the predicted amplitude, the true value of the amplitude, the predicted phase, the true value of the phase and the following formula (1): S17. Perform iterative training of forward propagation feedback according to the mean square error loss function. When the number of iterations reaches a preset threshold, stop the training to obtain a trained EEG word segmenter.

3. The method for analyzing the emotion of reading-induced EEG by reinforcement learning according to claim 1, characterized in that: The step S3 divides the EEG signal data into blocks to obtain multiple EEG signal segments, including: The EEG signal E is divided into multiple time windows of fixed size, and each EEG signal data is divided into segments, where each time window is regarded as an EEG segment containing several time frames, expressed as Where t is the length of the time window, there is no overlap between time windows, and the stride is s, that is, the starting position of each time window is the offset of the previous time window; The segmented EEG signal fragments are expressed as the following formula (2): in, represents the signal segment of the jth electrode channel in the kth time window, and w is the length of each time window.

4. The method for analyzing the emotion of reading-induced EEG by reinforcement learning according to claim 1, characterized in that: The EEG word segmenter includes a time encoder module, a time and space embedding module and an improved Transformer encoder module; The step S4 inputs the multiple EEG signal segments into the trained EEG word segmenter for encoding, and obtains multiple EEG encoding features, including: S41, inputting multiple EEG signal segments into a time encoder module, and obtaining corresponding temporal coding features for each segment; S42, optimizing the time embedding list according to the temporal coding feature, adding the optimized time embedding list, the initialized space embedding list and the EEG signal fragment to obtain an embedding sequence; S43. Input the embedded sequence into the improved Transformer encoder module to obtain multiple EEG encoding features.

5. The method for analyzing the emotion of reading-induced EEG by reinforcement learning according to claim 4 is characterized in that: The S43 inputs the embedded sequence into the improved Transformer encoder module to obtain multiple EEG encoding features, including: S431, input the embedded sequence into the improved Transformer encoder module, and obtain the query vector Q, the key vector K and the numerical vector V by linearly transforming the embedded sequence; S432, perform layer normalization on the query vector Q and the key vector K respectively, and omit the deviation term in the QKV calculation; S433. Transpose the normalized key vector K, perform dot product on the normalized query vector Q and the transposed key vector K to obtain the correlation between Q and K, scale the dot product result, apply the softmax function to convert the scaled correlation into a probability distribution to obtain weights, apply the weights to the numerical vector V, obtain the weighted summation result, and then obtain multiple EEG coding features.

6. The method for analyzing the emotion of reading-induced EEG by reinforcement learning according to claim 1, characterized in that: The S5 quantizes multiple EEG encoding features into discrete EEG vectors through a quantizer module, including: Define a neural codebook V′={v i |i=1,...,K}∈R KD , where K is the number of discrete EEG vectors and D is the dimension of each vector; In the neural codebook V′, the nearest vector of each EEG coding feature pi is found as the potential discrete representation of the EEG coding feature. This process is expressed as the following formula (3): in, express Normalization, z i is the discretized EEG vector after the quantizer.

7. The method for analyzing the emotion of reading-induced EEG by reinforcement learning according to claim 1, characterized in that: The reward function includes an enhanced reward, a discrete reward, and a representative reward; The calculation method of the enhancement reward includes: According to the following formula (4), the contribution of the selected keyword set to sentiment classification is measured by comparing the prediction probability at the sentence level: Among them, c is the true label of the word, p c represents the probability of predicting a word set as class c. 0 represents the sentence-level prediction of all words. The calculation process is to calculate the accuracy of the average EEG prediction of all word-level words, p 1 The sentence-level prediction of the keyword set is to calculate the prediction accuracy of each word-level EEG after taking action. is the feature vector of the tth word, a t Indicates whether the agent chooses a word at time t, a t =1 means selection, a t =0 means no selection; The calculation method of the discrete reward includes: The intra-class deviation matrix is ​​generated by calculating the distance from each sample point to the cluster center according to the following formula (5): in, is the sample mean vector of class w, n s represents the number of samples of class w, c represents the number of categories of classification, represents the i-th data sample in the j-th class, and the formula is to calculate the intra-class covariance matrix of the j-th class By calculating the deviation of each sample from the corresponding class mean (i.e. ) and average all samples to get The inter-class distance criterion function generates the inter-class deviation matrix S B , defined as follows (6): Among them, c represents the number of classes to be classified, is the mean vector of all samples to be classified, represents the number of samples of class w, represents the total number of samples, Represents the class center of each class, through the mean vector of each class j and the global mean The difference between them is taken as the outer product and weighted summation to get S B ; Discrete reward R based on intra-class distance criterion function and inter-class distance criterion function dis , defined as follows (7): Where Tr[**] is the trace of the computation matrix; The calculation method of the representative reward includes: According to the following formula (8), the extent to which the selected keyword set represents the characteristics of the original sentence is measured: Among them, x t represents the feature of the tth word in the sentence, T represents the number of words in the entire sentence, Y represents the index set of the keyword set, min t′∈Y Indicates selecting the feature most similar to the current word from the selected keywords.

8. A device for reinforcement learning of emotion analysis of reading-induced EEG, the device for reinforcement learning of emotion analysis of reading-induced EEG being used to implement the method for reinforcement learning of emotion analysis of reading-induced EEG as claimed in any one of claims 1 to 7, characterized in that: The device comprises: A training unit is used to obtain EEG signal training samples, and train the initial EEG word segmenter according to the EEG signal training samples and the EEG signal decoder to obtain a trained EEG word segmenter; An acquisition unit, used for acquiring two-dimensional EEG signal data, wherein the EEG signal data includes the EEG signal and corresponding text data; A block division unit, used for dividing the EEG signal data into blocks to obtain a plurality of EEG signal segments; wherein each EEG signal segment includes an EEG signal block and a corresponding text word; An encoding unit, used for inputting a plurality of EEG signal segments into a trained EEG word segmenter for encoding to obtain a plurality of EEG encoding features; A quantization unit, used for quantizing a plurality of EEG encoding features into discrete EEG vectors through a quantizer module; The feature extraction unit is used to extract key features from discrete EEG vectors and obtain the importance score of word embedding; A selection unit, configured to input the importance scores of word embeddings into the intelligent agent, and perform action-selection-based sampling on the plurality of EEG signal segments through a selection strategy optimized by a reward function of the intelligent agent to obtain a keyword set; The analysis unit is used to input the keyword set into the emotion classification module to obtain the emotion classification corresponding to the EEG signal data.

9. A device for reinforcing learning and sentiment analysis of reading-induced EEG, characterized in that: The reading-induced EEG reinforcement learning emotion analysis device comprises: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, which can be called by a processor to execute the method according to any one of claims 1 to 7.

Images