Electroencephalogram decision result prediction model construction method, prediction method and device
By constructing a decision model that incorporates emotional information and using multi-task learning technology to predict decision results, the problem of insufficient generalization and accuracy in the existing technology is solved, and more efficient decision results prediction is achieved.
Patent Information
- Application Number
- CN202410107816.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-01-25
- Publication Date
- 2025-07-25
AI Technical Summary
The existing decision-making model based on EEG-based emotions has poor generalization and accuracy, and has failed to effectively integrate emotional information.
Build a decision model that incorporates emotional information, and use the three tasks of emotion, decision-making and supervised comparison to achieve single trial prediction of decision results using multi-task learning technology, including time learning module, graph learning module and multi-task learning module, and combine deep learning models to extract decision-making and emotional characteristics.
It improves the accuracy and generalization performance of decision-making results prediction, and can effectively consider the impact of emotional factors on decision-making.
Smart Images
Figure CN120372270A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular, to a method, a prediction method, and a device for constructing an electroencephalogram decision result prediction model. Background Art
[0002] Decision-making is an indispensable part of human life, covering a wide range of fields, including personal life, career development, financial investment, and business strategy, etc. However, decision-making is not just a rational process, and emotions also play a crucial role in it. In many cases, the decisions of individuals or organizations are often directly affected by their emotional states.
[0003] The emotional center of the brain is closely connected to the decision-making network, and there is a complex and delicate interaction between the two. Different emotional states may trigger different activations in the neural network, thus affecting the decision-making process.
[0004] Most of the existing decision-making models under the influence of emotions based on electroencephalogram (EEG) adopt some simple machine learning methods, such as support vector machines, logistic regression, hidden Markov models, etc. Although good results have been achieved, their generalization and accuracy still need to be improved. Summary of the Invention
[0005] The present invention provides a method, a prediction method, and a device for constructing an electroencephalogram decision result prediction model to solve the defects of poor generalization and accuracy of the decision result prediction model in the prior art.
[0006] The present invention provides a method for constructing an electroencephalogram decision result prediction model, including:
[0007] Obtaining sample electroencephalogram signals and their corresponding emotion category labels and decision category labels, as well as an initial model;
[0008] Respectively extracting the decision features and emotion features of the sample electroencephalogram signals, and respectively performing decision result prediction and emotion category prediction based on the decision features and emotion features to obtain decision prediction results and emotion prediction results;
[0009] Based on the difference between the decision prediction result and the decision category label, determining the decision task loss; based on the difference between the emotion prediction result and the emotion category label, determining the emotion task loss; based on the similarity between the decision category labels corresponding to every two sample electroencephalogram signals, and the similarity between the decision features of the every two sample electroencephalogram signals, determining the contrast task loss;
[0010] Based on the decision task loss, the emotion task loss, and the contrast task loss, perform parameter iteration on the initial model to obtain the EEG decision result prediction model.
[0011] According to the method for constructing an EEG decision result prediction model proposed by the present invention, the initial model includes a time learning module, a graph learning module, and a multi-task learning module. The steps of respectively extracting the decision features and emotion features of the sample EEG signals include:
[0012] Input the sample EEG signal into the time learning module to obtain the time features output by the time learning module;
[0013] Input the time features into the graph learning module to obtain the graph structure features output by the graph learning module;
[0014] Input the graph structure features into the perception layer in the multi-task learning module respectively to obtain the decision features and emotion features output by the perception layer respectively.
[0015] According to the method for constructing an EEG decision result prediction model proposed by the present invention, the time learning module includes a meiosis layer, a feature extraction layer, and a feature fusion layer. The steps of inputting the sample EEG signal into the time learning module to obtain the time features output by the time learning module include:
[0016] Input the sample EEG signal into the meiosis layer to perform data augmentation on the sample EEG signal to obtain the enhanced EEG signal output by the meiosis layer;
[0017] Input the enhanced EEG signal into the feature extraction layer to perform multi-scale power feature extraction and splicing on the enhanced EEG signal to obtain the first spliced feature output by the feature extraction layer;
[0018] Input the first spliced feature into the feature fusion layer to perform attention feature fusion on the first spliced feature to obtain the time features output by the feature fusion layer.
[0019] According to the method for constructing an EEG decision result prediction model proposed by the present invention, the graph learning module includes a graph structure representation layer, a local graph filtering layer, and a global graph filtering layer. The steps of inputting the time features into the graph learning module to obtain the graph structure features output by the graph learning module include:
[0020] Input the time features into the graph structure representation layer to extract the initial graph structure features of the time features to obtain the initial graph structure features output by the graph structure representation layer;
[0021] Input the initial graph structure features into the local graph filtering layer, aggregate the initial graph structure features, and obtain the local graph structure features output by the local graph filtering layer;
[0022] Input the local graph structure features into the global graph filtering layer, learn the relationships between the local graph structure features, and obtain the graph structure features output by the global graph filtering layer.
[0023] According to the method for constructing an electroencephalogram decision result prediction model proposed by the present invention, the obtaining of the sample electroencephalogram signals includes:
[0024] Obtain the original electroencephalogram signals;
[0025] Preprocess the original electroencephalogram signals to obtain the sample electroencephalogram signals, and the preprocessing includes at least one of downsampling, band-pass filtering, rereferencing, and artifact removal.
[0026] The present invention also proposes a decision result prediction method, including:
[0027] Obtain the electroencephalogram signals under the target decision task;
[0028] Input the electroencephalogram signals into the electroencephalogram decision result prediction model, predict whether the decision result under the target decision task is accurate, and obtain the prediction result output by the electroencephalogram decision result prediction model, where the electroencephalogram decision result prediction model is obtained based on the method for constructing an electroencephalogram decision result prediction model described above.
[0029] The present invention also proposes an electroencephalogram decision result prediction model construction device, including:
[0030] A data acquisition unit for acquiring sample electroencephalogram signals and their corresponding emotion category labels and decision category labels, as well as an initial model;
[0031] A feature extraction unit for respectively extracting the decision features and emotion features of the sample electroencephalogram signals, and respectively performing decision result prediction and emotion category prediction based on the decision features and emotion features to obtain a decision prediction result and an emotion prediction result;
[0032] A loss determination unit for determining a decision task loss based on the difference between the decision prediction result and the decision category label; determining an emotion task loss based on the difference between the emotion prediction result and the emotion category label; and determining a contrast task loss based on the similarity between the decision category labels corresponding to every two sample electroencephalogram signals and the similarity between the decision features of every two sample electroencephalogram signals;
[0033] A parameter iteration unit, configured to perform parameter iteration on the initial model based on the decision task loss, the emotion task loss, and the contrast task loss, so as to obtain the EEG decision result prediction model.
[0034] The present invention also provides a decision result prediction device, including:
[0035] An EEG signal acquisition unit, configured to acquire EEG signals under a target decision task;
[0036] A decision result prediction unit, configured to input the EEG signals into the EEG decision result prediction model, and predict whether the decision result under the target decision task is accurate, so as to obtain a prediction result output by the EEG decision result prediction model, where the EEG decision result prediction model is obtained based on the above-mentioned EEG decision result prediction model construction method.
[0037] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the EEG decision result prediction model construction method or the decision result prediction method as described in any one of the above.
[0038] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the EEG decision result prediction model construction method or the decision result prediction method as described in any one of the above.
[0039] The present invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the EEG decision result prediction model construction method or the decision result prediction method as described in any one of the above.
[0040] The EEG decision result prediction model construction method, prediction method, and device provided by the present invention construct a decision result prediction model incorporating emotion information, and perform joint training through three tasks of emotion, decision, and supervised contrast, and realize single-trial prediction of the decision result by means of multi-task learning technology. The EEG decision result prediction model obtained by using the construction method provided by the embodiments of the present invention has good generalization performance and can improve the accuracy of decision result prediction. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.
[0042] Figure 1 is one of the schematic flowcharts of the method for constructing an electroencephalogram decision result prediction model proposed by the present invention;
[0043] Figure 2 is the second of the schematic flowcharts of the method for constructing an electroencephalogram decision result prediction model proposed by the present invention;
[0044] Figure 3 is the schematic flowchart of the decision result prediction method proposed by the present invention;
[0045] Figure 4 is the schematic structural diagram of the electroencephalogram decision result prediction model construction device proposed by the present invention;
[0046] Figure 5 is the schematic structural diagram of the decision result prediction device proposed by the present invention;
[0047] Figure 6 is the schematic structural diagram of the electronic device proposed by the present invention. Detailed implementation manners
[0048] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without creative efforts shall fall within the protection scope of the present invention.
[0049] Most of the existing decision-making models based on EEG under the influence of emotions adopt some simple machine learning, such as support vector machines, logistic regression, hidden Markov models, etc. Although good results have been achieved, their generalization and accuracy still need to be improved, and the emotion information has not been well integrated into the decision-making model.
[0050] Aiming at the problem of poor generalization and accuracy of the decision-making model in the prior art, the inventive concept of the present invention lies in: using a deep learning model to construct a decision-making model integrating emotion information, jointly training through three tasks of emotion, decision-making, and supervised comparison, and realizing single-trial prediction of decision results by means of multi-task learning technology. The electroencephalogram decision result prediction model obtained by the construction method proposed in the embodiments of the present invention has good generalization performance and can improve the accuracy of decision result prediction.
[0051] Based on the above inventive concept, the present invention proposes a method, a prediction method and a device for constructing an electroencephalogram decision result prediction model, which can be applied to decision result prediction scenarios considering the emotional factors of the subjects, such as tasks of spatial decision-making, perceptual decision-making, consumption decision-making, etc., so as to improve the accuracy of decision prediction.
[0052] The technical solution of the present invention will be described in detail below in conjunction with the accompanying drawings. Figure 1 It is one of the schematic flowcharts of the method for constructing an electroencephalogram decision result prediction model proposed by the present invention. The execution subject of each step in this method can be an electroencephalogram decision result prediction model construction device, which can be implemented by software and / or hardware, and can be integrated in an electronic device. The electronic device can be a terminal device (such as a smart phone, a personal computer, etc.), or a server (such as a local server or a cloud server, or can also be a server cluster, etc.), or a processor, or a chip, etc. As Figure 1 shown, the method includes the following steps:
[0053] Step 110, obtaining sample electroencephalogram signals and their corresponding emotion category labels and decision category labels, as well as an initial model;
[0054] Step 120, respectively extracting the decision features and emotion features of the sample electroencephalogram signals, and respectively performing decision result prediction and emotion category prediction based on the decision features and emotion features to obtain decision prediction results and emotion prediction results;
[0055] Step 130, determining the decision task loss based on the difference between the decision prediction result and the decision category label; determining the emotion task loss based on the difference between the emotion prediction result and the emotion category label; determining the contrast task loss based on the similarity between the decision category labels corresponding to every two sample electroencephalogram signals and the similarity between the decision features of every two sample electroencephalogram signals;
[0056] Step 140, performing parameter iteration on the initial model based on the decision task loss, the emotion task loss, and the contrast task loss to obtain an electroencephalogram decision result prediction model.
[0057] Specifically, the electroencephalogram decision result prediction model is used to predict whether the decision made by the subject under the target decision task is accurate. Considering that the emotional center of the brain is closely connected to the decision-making network, and there is a complex and delicate interaction between the two. Different emotional states may trigger different activations in the neural network, thereby affecting the decision-making process. Therefore, through the analysis of electroencephalogram signals, an emotion-influenced decision result prediction model can be effectively constructed.
[0058] The sample electroencephalogram signals and their corresponding emotion category labels and decision category labels constitute a training data set, and the training data set can be divided into a training set and a test set. The training set is used for model training, and the test set is used for model testing. By designing a task of emotion influencing decision-making, the original electroencephalogram signal S1 of the subject when making a decision and its corresponding emotion category label L1 and decision category label L2 can be obtained.
[0059] The task of emotion influencing decision-making consists of two parts: emotion induction and decision-making tasks. In the emotion induction stage, stimuli such as films, pictures, music, etc. are used to induce emotions such as happiness, neutrality, sadness, fear, etc. in the subjects. The decision-making task stage includes tasks such as multi-trial spatial decision-making, perceptual decision-making, and consumption decision-making. During this process, the original electroencephalogram (EEG) signals of the subjects performing multi-trial decision-making tasks under different emotions are collected to obtain S1. The emotion category label is the induced emotion category, and the decision category label is the information on whether each single trial decision is accurate under each emotion, which can include accurate or inaccurate, for example. The initial model can be a deep learning model.
[0060] Here, the original EEG signal can be directly used as the sample EEG signal. Or, in some embodiments, in order to further improve the accuracy of signal processing, obtaining the sample EEG signal in step 110 includes:
[0061] Step 111, obtaining the original EEG signal;
[0062] Step 112, preprocessing the original EEG signal to obtain the sample EEG signal. The preprocessing includes at least one of downsampling, band-pass filtering, rereferencing, and artifact removal.
[0063] Specifically, in this embodiment, after obtaining the original EEG signal S1, the original EEG signal can be preprocessed to obtain the sample EEG signal S2. The preprocessing can specifically include downsampling the original EEG signal to 250 Hz, band-pass filtering at 0.1 - 40 Hz, rereferencing using M1 and M2 as reference electrodes, and removing artifacts such as electrooculogram by independent component analysis.
[0064] Then, step 120 is executed to respectively extract the decision-making features and emotion features of the sample EEG signal, and based on the decision-making features and emotion features, decision result prediction and emotion category prediction are respectively performed to obtain the decision prediction result and the emotion prediction result. In the embodiment of the present invention, emotion factors are considered when predicting the decision result, so the decision-making features and emotion features of the sample EEG signal can be extracted first. Here, the decision-making features can represent the features related to whether the decision is accurate in the sample EEG signal, and the emotion features can represent the features related to the subject's emotion in the sample EEG signal. On this basis, the decision prediction result and the emotion prediction result are respectively predicted based on the decision-making features and emotion features. The decision prediction result represents the accuracy of the decision made by the subject during the decision-making task, and the emotion prediction result represents the emotion category of the subject when making this decision.
[0065] In some embodiments, the initial model includes a temporal learning module, a graph learning module, and a multi-task learning module. Specifically, extracting the decision-making features and emotion features of the sample EEG signal in step 120 includes:
[0066] Step 121: Input the sample EEG signal into the time learning module to obtain the time features output by the time learning module.
[0067] Step 122: Input the time features into the graph learning module to obtain the graph structure features output by the graph learning module.
[0068] Step 123: Input the graph structure features into the perception layer in the multi-task learning module respectively to obtain the decision-making features and emotion features output by the perception layer respectively.
[0069] Specifically, the extraction of time features can be obtained through the time learning module. The time learning module includes a meiosis layer, a feature extraction layer, and a feature fusion layer. Input the sample EEG signal into the time learning module to obtain the time features output by the time learning module, including:
[0070] Input the sample EEG signal into the meiosis layer to perform data augmentation on the sample EEG signal to obtain the enhanced EEG signal output by the meiosis layer.
[0071] Input the enhanced EEG signal into the feature extraction layer to perform multi-scale power feature extraction and splicing on the enhanced EEG signal to obtain the first spliced feature output by the feature extraction layer.
[0072] Input the first spliced feature into the feature fusion layer to perform attention feature fusion on the first spliced feature to obtain the time features output by the feature fusion layer.
[0073] Specifically, in the time learning module, first, based on the meiosis layer, perform meiosis augmentation on the sample EEG signal S2 to obtain the enhanced EEG signal A. Subsequently, perform multi-scale power feature extraction on the enhanced EEG signal A, and splice the extracted multi-scale power features to obtain the first spliced feature F1.
[0074] The first spliced feature F1 passes through the kernel-based attention feature fusion layer to obtain the second spliced feature F2. The second spliced feature F2 can be directly used as the time features. Further, the second spliced feature F2 can also be rearranged to obtain the time features F T 。
[0075] Here, the steps included in the kernel-based attention feature fusion layer are batch normalization, 1×1 convolution, LeakyReLU activation function, average pooling, and batch normalization in sequence.
[0076] In step 122, the graph learning module includes a graph structure representation layer, a local graph filtering layer, and a global graph filtering layer. Input the time features into the graph learning module to obtain the graph structure features output by the graph learning module, including:
[0077] Input the time features into the graph structure representation layer to extract the initial graph structure features of the time features, and obtain the initial graph structure features output by the graph structure representation layer;
[0078] Input the initial graph structure features into the local graph filtering layer to aggregate the initial graph structure features, and obtain the local graph structure features output by the local graph filtering layer;
[0079] Input the local graph structure features into the global graph filtering layer to learn the relationships between the local graph structure features, and obtain the graph structure features output by the global graph filtering layer.
[0080] Specifically, in the graph learning module, first represent the time features F T obtained by the time learning module in the form of a graph structure to get the initial graph structure features G1, obtain the features G2 through the local graph filtering layer, and finally obtain the graph structure features F G .
[0081] Specifically, the local graph filtering layer is used to aggregate the representations learned in each local graph, and its specific calculation method is as follows:
[0082]
[0083]
[0084] where, W local and b local are respectively the trainable local graph filtering matrix and the local graph filtering bias vector, is the Hadamard product, is the ReLU activation function, F aggregete is the aggregation function used to obtain the hidden embedding of the local graph, G1 filtered (i) is the graph representation of local filtering, Z1,..., Z R are the local graph representations, R is the total number of local graphs, h local is the latent representation of the local graph.
[0085] The global graph filtering layer aims to learn the complex relationships between local graphs, and its calculation formula is as follows:
[0086]
[0087]
[0088]
[0089]
[0090]
[0091] Among them, A global-bas is the basic adjacency matrix of the global graph, M is the trainable attention mask, and A global is the final global adjacency matrix. is the normalized adjacency matrix, where is the degree of A global . Finally, the result obtained by global graph filtering is
[0092] to obtain the graph structure feature F G . After obtaining the graph structure feature F G , it is possible to execute step 123. Pass the graph structure feature F dec through two Multilayer Perceptron (MLP) layers respectively to extract decision features and emotion features, and obtain the decision feature f emo output by the two MLP layers respectively.
[0093] After obtaining the decision feature f dec and the emotion feature f emo respectively, it is possible to perform decision result prediction and emotion category prediction through a softmax function respectively, and obtain the decision prediction result and the emotion prediction result respectively. Determine the model loss function according to the decision prediction result and the emotion prediction result, and perform parameter iteration on the initial model to obtain the EEG decision result prediction model.
[0094] In step 130, based on the difference between the decision prediction result and the decision class label, determine the decision task loss, and based on the difference between the emotion prediction result and the emotion class label, determine the emotion task loss. The decision task loss can specifically be the decision cross-entropy loss L dec , and the emotion task loss can specifically be the emotion cross-entropy loss L emo .
[0095] In addition, it is also possible to determine the contrast task loss based on the similarity between the decision class labels corresponding to every two sample EEG signals, and the similarity between the decision features of every two sample EEG signals. Adopt a supervised contrast strategy to maximize the mutual information of the EEG decision features with the same decision label and minimize the mutual information of the EEG decision features with different decision labels, and calculate its supervised contrast loss L ct . In some embodiments, the calculation formula of L ct can be expressed as follows:
[0096]
[0097] In the formula, N is the total number of sample EEG signals; sim(f(x i ), f(x j )) represents the sample xi and the sample x j The similarity score (in this example, their dot product) between the corresponding decision features; τ is the temperature parameter used to adjust the scale of similarity; s ij is the similarity label, representing the similarity between the decision class labels corresponding to the EEG signals of each pair of samples. If the sample x i and the sample x i belong to the same category, then s ij is 1, otherwise s ij is 0. Where f(x) is the normalized result of the decision feature f dec .
[0098] Based on this, the initial model can be iteratively parameterized based on the decision task loss, emotion task loss, and contrast task loss to obtain an EEG decision result prediction model.
[0099] In some embodiments, the total model loss can be expressed as:
[0100] Total loss L = αL dec + βL emo + γL ct (9)
[0101] where α, β, and γ are adjustable hyperparameters, L dec is the decision task loss, L emo is the emotion task loss, L ct is the contrast task loss. The initial model is iteratively parameterized by minimizing the total loss function, and the resulting EEG decision result prediction model can be used to predict the decision result.
[0102] Figure 2 is the second flowchart of the method for constructing an EEG decision result prediction model proposed by the present invention. For the implementation manners of the steps in the figure, reference can be made to the descriptions of the above embodiments and will not be elaborated herein.
[0103] The method proposed in the embodiments of the present invention effectively combines the advantages of time feature extraction and graph feature extraction to ensure that the spatio-temporal frequency features of EEG signals are effectively extracted. Subsequently, a multi-task learning method is combined to effectively integrate emotion information, and a supervised contrast strategy is adopted to maximize the mutual information of EEG features with the same decision label and minimize the mutual information of EEG features with different decision labels, further optimizing the model.
[0104] Based on any of the above embodiments, Figure 3 is the flowchart of the decision result prediction method proposed by the present invention. As Figure 3 shown, a decision result prediction method is proposed, including the following steps:
[0105] Step 310: Obtain the electroencephalogram (EEG) signals under the target decision-making task;
[0106] Step 320: Input the EEG signals into the EEG decision result prediction model to predict whether the decision under the target decision-making task is accurate, and obtain the prediction result output by the EEG decision result prediction model. The EEG decision result prediction model is obtained based on the above-mentioned EEG decision result prediction model construction method.
[0107] Specifically, after obtaining the emotion-incorporated EEG decision result prediction model through the above construction method, the EEG decision result prediction model can be used to predict whether the decision made by the subject under the target decision-making task is accurate.
[0108] First, obtain the EEG signals when the subject makes a decision under the target decision-making task, input the EEG signals into the EEG decision result prediction model, and the prediction model predicts whether the decision made by the subject under the target decision-making task is accurate to obtain the prediction result output by the prediction model. The prediction result may specifically include that the EEG decision result corresponding to the EEG signals is accurate or inaccurate.
[0109] It can be understood that the EEG decision result prediction model can also output the emotion category corresponding to the EEG signals, that is, the emotion category corresponding when the subject makes a decision, such as emotions including happiness, neutrality, sadness, fear, etc.
[0110] The method proposed in the embodiments of the present invention can improve the accuracy of decision prediction by performing decision prediction through the emotion-incorporated EEG decision result prediction model.
[0111] Next, a description is given of the EEG decision result prediction model construction device proposed by the present invention. The EEG decision result prediction model construction device described below can be correspondingly referred to with the EEG decision result prediction model construction method described above.
[0112] Based on any of the above embodiments, Figure 4 is a schematic structural diagram of the EEG decision result prediction model construction device proposed by the present invention. As Figure 4 shown, a kind of EEG decision result prediction model construction device is proposed, including:
[0113] A data acquisition unit 410, configured to acquire sample EEG signals, their corresponding emotion category labels and decision category labels, and an initial model;
[0114] A feature extraction unit 420, configured to respectively extract the decision features and emotion features of the sample EEG signals, and respectively perform decision result prediction and emotion category prediction based on the decision features and emotion features to obtain a decision prediction result and an emotion prediction result;
[0115] A loss determination unit 430 is configured to determine a decision task loss based on the difference between the decision prediction result and the decision class label; determine an emotion task loss based on the difference between the emotion prediction result and the emotion class label; and determine a contrast task loss based on the similarity between the decision class labels corresponding to every two sample EEG signals and the similarity between the decision features of the every two sample EEG signals.
[0116] A parameter iteration unit 440 is configured to perform parameter iteration on the initial model based on the decision task loss, the emotion task loss, and the contrast task loss, to obtain the EEG decision result prediction model.
[0117] The EEG decision result prediction model construction device proposed in the embodiments of the present invention constructs a decision model incorporating emotion information, and through joint training of three tasks: emotion, decision, and supervised contrast, realizes single-trial prediction of decision results by means of multi-task learning technology. The obtained EEG decision result prediction model has good generalization performance and can improve the accuracy of decision prediction.
[0118] Based on any of the above embodiments, the initial model includes a temporal learning module, a graph learning module, and a multi-task learning module. The feature extraction unit is specifically configured to:
[0119] Input the sample EEG signal into the temporal learning module to obtain the temporal features output by the temporal learning module;
[0120] Input the temporal features into the graph learning module to obtain the graph structure features output by the graph learning module;
[0121] Input the graph structure features into the perception layer in the multi-task learning module respectively to obtain the decision features and emotion features output by the perception layer respectively.
[0122] Based on any of the above embodiments, the temporal learning module includes a meiosis layer, a feature extraction layer, and a feature fusion layer. The feature extraction unit is further specifically configured to:
[0123] Input the sample EEG signal into the meiosis layer to perform data augmentation on the sample EEG signal to obtain the enhanced EEG signal output by the meiosis layer;
[0124] Input the enhanced EEG signal into the feature extraction layer to perform multi-scale power feature extraction and splicing on the enhanced EEG signal to obtain the first spliced feature output by the feature extraction layer;
[0125] Input the first spliced feature into the feature fusion layer to perform attention feature fusion on the first spliced feature to obtain the temporal features output by the feature fusion layer.
[0126] Based on any of the above embodiments, the graph learning module includes a graph structure representation layer, a local graph filtering layer, and a global graph filtering layer. The feature extraction unit is further specifically configured to:
[0127] Input the time feature into the graph structure representation layer, extract the initial graph structure feature of the time feature, and obtain the initial graph structure feature output by the graph structure representation layer;
[0128] Input the initial graph structure feature into the local graph filtering layer, aggregate the initial graph structure feature, and obtain the local graph structure feature output by the local graph filtering layer;
[0129] Input the local graph structure feature into the global graph filtering layer, learn the relationship between the local graph structure features, and obtain the graph structure feature output by the global graph filtering layer.
[0130] Based on any of the above embodiments, the data acquisition unit is specifically configured to:
[0131] Acquire the original electroencephalogram (EEG) signal;
[0132] Preprocess the original EEG signal to obtain the sample EEG signal, where the preprocessing includes at least one of downsampling, band-pass filtering, rereferencing, and artifact removal.
[0133] Based on any of the above embodiments, Figure 5 is a schematic structural diagram of the decision result prediction device proposed by the present invention. As Figure 5 shown, a decision result prediction device is proposed, including:
[0134] An EEG signal acquisition unit 510, configured to acquire an EEG signal under a target decision task;
[0135] A decision result prediction unit 520, configured to input the EEG signal into an EEG decision result prediction model, predict whether the decision under the target decision task is accurate, and obtain a prediction result output by the EEG decision result prediction model, where the EEG decision result prediction model is obtained based on the EEG decision result prediction model construction method described above.
[0136] The decision result prediction device proposed in the embodiments of the present invention can improve the accuracy of decision prediction by performing decision prediction through an EEG decision result prediction model that incorporates emotions.
[0137] Figure 6 Illustrates a schematic physical structure diagram of an electronic device. As Figure 6As shown in the figure, the electronic device may include: a processor 610, a communications interface 620, a memory 630, and a communication bus 640. Among them, the processor 610, the communications interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 may call the logical instructions in the memory 630 to execute the method for constructing an electroencephalogram decision result prediction model. The method includes:
[0138] Obtain sample electroencephalogram signals and their corresponding emotion category labels and decision category labels, as well as an initial model;
[0139] Extract the decision features and emotion features of the sample electroencephalogram signals respectively, and perform decision result prediction and emotion category prediction based on the decision features and emotion features respectively to obtain decision prediction results and emotion prediction results;
[0140] Based on the difference between the decision prediction result and the decision category label, determine the decision task loss; based on the difference between the emotion prediction result and the emotion category label, determine the emotion task loss; based on the similarity between the decision category labels corresponding to every two sample electroencephalogram signals and the similarity between the decision features of the every two sample electroencephalogram signals, determine the contrast task loss;
[0141] Based on the decision task loss, the emotion task loss, and the contrast task loss, perform parameter iteration on the initial model to obtain the electroencephalogram decision result prediction model.
[0142] The processor may call the logical instructions in the memory to execute the decision result prediction method. The method includes:
[0143] Obtain the electroencephalogram signal under the target decision task;
[0144] Input the electroencephalogram signal into the electroencephalogram decision result prediction model to predict whether the decision result under the target decision task is accurate, and obtain the prediction result output by the electroencephalogram decision result prediction model. The electroencephalogram decision result prediction model is obtained based on the above-mentioned method for constructing an electroencephalogram decision result prediction model.
[0145] In addition, when the logical instructions in the above-mentioned memory 630 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.
[0146] On the other hand, the present invention also proposes a computer program product. The computer program product includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the method for constructing an electroencephalogram decision result prediction model proposed in the above-mentioned various methods. The method includes:
[0147] Obtain sample electroencephalogram signals and their corresponding emotion category labels and decision category labels, as well as an initial model;
[0148] Extract the decision features and emotion features of the sample electroencephalogram signals respectively, and perform decision result prediction and emotion category prediction based on the decision features and emotion features respectively to obtain decision prediction results and emotion prediction results;
[0149] Based on the difference between the decision prediction result and the decision category label, determine the decision task loss; based on the difference between the emotion prediction result and the emotion category label, determine the emotion task loss; based on the similarity between the decision category labels corresponding to every two sample electroencephalogram signals and the similarity between the decision features of every two sample electroencephalogram signals, determine the contrast task loss;
[0150] Based on the decision task loss, the emotion task loss, and the contrast task loss, perform parameter iteration on the initial model to obtain the electroencephalogram decision result prediction model.
[0151] When the computer program is executed by a processor, the computer can execute the decision prediction method proposed in the above-mentioned various methods. The method includes:
[0152] Obtain the electroencephalogram signal under the target decision task;
[0153] Input the electroencephalogram (EEG) signal into the EEG decision result prediction model to predict whether the decision result under the target decision task is accurate, and obtain the prediction result output by the EEG decision result prediction model. The EEG decision result prediction model is obtained based on the EEG decision result prediction model construction method described above.
[0154] In another aspect, the present invention also proposes a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is configured to execute the EEG decision result prediction model construction method proposed in the above methods. The method includes:
[0155] Obtain sample EEG signals and their corresponding emotion category labels and decision category labels, as well as an initial model.
[0156] Extract the decision features and emotion features of the sample EEG signals respectively, and perform decision result prediction and emotion category prediction based on the decision features and emotion features respectively to obtain decision prediction results and emotion prediction results.
[0157] Based on the difference between the decision prediction result and the decision category label, determine the decision task loss; based on the difference between the emotion prediction result and the emotion category label, determine the emotion task loss; based on the similarity between the decision category labels corresponding to every two sample EEG signals and the similarity between the decision features of the every two sample EEG signals, determine the contrast task loss.
[0158] Based on the decision task loss, the emotion task loss, and the contrast task loss, perform parameter iteration on the initial model to obtain the EEG decision result prediction model.
[0159] When the computer program is executed by a processor, the computer can execute the decision prediction method proposed in the above methods. The method includes:
[0160] Obtain the EEG signal under the target decision task.
[0161] Input the EEG signal into the EEG decision result prediction model to predict whether the decision result under the target decision task is accurate, and obtain the prediction result output by the EEG decision result prediction model. The EEG decision result prediction model is obtained based on the EEG decision result prediction model construction method described above.
[0162] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without creative labor.
[0163] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0164] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments or equivalently replace some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for constructing an electroencephalogram decision result prediction model, characterized in that, Including: Obtaining a sample electroencephalogram (EEG) signal, its corresponding emotion category label and decision category label, and an initial model; Respectively extracting the decision-making features and emotion features of the sample EEG signal, and respectively predicting the decision-making result and emotion category prediction based on the decision-making features and emotion features to obtain a decision-making prediction result and an emotion prediction result; Determining a decision-making task loss based on the difference between the decision-making prediction result and the decision category label; determining an emotion task loss based on the difference between the emotion prediction result and the emotion category label; Determining a contrast task loss based on the similarity between the decision category labels corresponding to every two sample EEG signals and the similarity between the decision-making features of the every two sample EEG signals; Based on the decision-making task loss, the emotion task loss and the contrast task loss, performing parameter iteration on the initial model to obtain the EEG decision-making result prediction model.
2. The method for constructing an electroencephalogram decision result prediction model according to claim 1, wherein The initial model includes a time learning module, a graph learning module and a multi-task learning module. The respectively extracting the decision-making features and emotion features of the sample EEG signal includes: Inputting the sample EEG signal into the time learning module to obtain the time features output by the time learning module; Inputting the time features into the graph learning module to obtain the graph structure features output by the graph learning module; Inputting the graph structure features into the perception layer in the multi-task learning module respectively to obtain the decision-making features and emotion features respectively output by the perception layer.
3. The method for constructing an electroencephalogram decision result prediction model according to claim 2, wherein The time learning module includes a meiosis layer, a feature extraction layer and a feature fusion layer. The inputting the sample EEG signal into the time learning module to obtain the time features output by the time learning module includes: Inputting the sample EEG signal into the meiosis layer to perform data augmentation on the sample EEG signal to obtain the enhanced EEG signal output by the meiosis layer; Inputting the enhanced EEG signal into the feature extraction layer to perform multi-scale power feature extraction and splicing on the enhanced EEG signal to obtain the first spliced feature output by the feature extraction layer; Inputting the first spliced feature into the feature fusion layer to perform attention feature fusion on the first spliced feature to obtain the time features output by the feature fusion layer.
4. The method for constructing an electroencephalogram decision result prediction model according to claim 2, wherein, The graph learning module includes a graph structure representation layer, a local graph filtering layer and a global graph filtering layer. The inputting the time features into the graph learning module to obtain the graph structure features output by the graph learning module includes: Inputting the time features into the graph structure representation layer to extract the initial graph structure features of the time features to obtain the initial graph structure features output by the graph structure representation layer; Inputting the initial graph structure features into the local graph filtering layer to aggregate the initial graph structure features to obtain the local graph structure features output by the local graph filtering layer; Inputting the local graph structure features into the global graph filtering layer to learn the relationship between the local graph structure features to obtain the graph structure features output by the global graph filtering layer.
5. The method for constructing an electroencephalogram decision result prediction model according to any one of claims 1 to 4, characterized in that, The obtaining the sample EEG signal includes: Obtaining the original EEG signal; Preprocess the original EEG signal to obtain the sample EEG signal, and the preprocessing includes at least one of downsampling, band-pass filtering, rereferencing, and artifact removal.
6. A decision result prediction method, characterized in that, It includes: Obtain the EEG signal under the target decision task; Input the EEG signal into the EEG decision result prediction model to predict whether the decision result under the target decision task is accurate, and obtain the prediction result output by the EEG decision result prediction model. The EEG decision result prediction model is obtained based on the EEG decision result prediction model construction method described in any one of claims 1 to 5.
7. A device for constructing an electroencephalogram decision result prediction model, characterized in that, It includes: A data acquisition unit for acquiring the sample EEG signal, its corresponding emotion category label and decision category label, and the initial model; A feature extraction unit for respectively extracting the decision feature and emotion feature of the sample EEG signal, and respectively performing decision result prediction and emotion category prediction based on the decision feature and emotion feature to obtain a decision prediction result and an emotion prediction result; A loss determination unit for determining the decision task loss based on the difference between the decision prediction result and the decision category label; Determine the emotion task loss based on the difference between the emotion prediction result and the emotion category label; Determine the contrast task loss based on the similarity between the decision category labels corresponding to every two sample EEG signals and the similarity between the decision features of the every two sample EEG signals; A parameter iteration unit for performing parameter iteration on the initial model based on the decision task loss, the emotion task loss, and the contrast task loss to obtain the EEG decision result prediction model.
8. A decision result prediction device, characterized in that It includes: An EEG signal acquisition unit for acquiring the EEG signal under the target decision task; A decision result prediction unit for inputting the EEG signal into the EEG decision result prediction model to predict whether the decision result under the target decision task is accurate, and obtaining the prediction result output by the EEG decision result prediction model. The EEG decision result prediction model is obtained based on the EEG decision result prediction model construction method described in any one of claims 1 to 5.
9. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the EEG decision result prediction model construction method described in any one of claims 1 to 5, or the decision result prediction method described in claim 6.
10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the EEG decision result prediction model construction method described in any one of claims 1 to 5, or the decision result prediction method described in claim 6.