Modeling method, device and equipment of decision performance prediction model and storage medium

Abstract

The embodiment of the application provides a modeling method and device of a decision performance prediction model, equipment and a storage medium, the method comprises the following steps: acquiring an electroencephalogram signal marked with an emotion label and / or a decision performance label; based on the marked electroencephalogram signal, extracting an emotion feature signal and a decision feature signal corresponding to the electroencephalogram signal, and determining a first loss function, a second loss function and a third target function; determining a cross-entropy loss function corresponding to a first predicted value and a first true value as a third loss function; and determining a total target function to satisfy a preset condition to obtain the decision performance prediction model. The method provided by the application can realize prediction of decision performance according to electroencephalogram signals with different emotion characteristics, and the decision performance predicted by using the decision performance prediction model provided by the application has higher accuracy.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a modeling method, apparatus, device, and storage medium for a decision performance prediction model. Background Technology

[0002] Decision-making is a complex cognitive process, central to human information processing, and plays a vital role in people's daily lives. Many factors influence decision-making, such as emotions, individual differences, and individual cognitive biases.

[0003] Current evidence suggests that neurophysiological signals, such as electroencephalograms (EEGs) and functional magnetic resonance imaging (fMRI), can be used to assess information related to decision-making or emotion. Furthermore, the objectivity and resistance to falsification of physiological signals ensure accurate predictions. Among numerous neurophysiological signals, EEG has attracted significant research attention due to its high temporal resolution and ease of acquisition, making it a promising tool for building practical real-time decision prediction systems.

[0004] While some research has explored emotion recognition using EEG signals, there is still a lack of research on the rules governing the transformation of emotions into decisions, and the rationality or correctness of how emotions influence decisions. Therefore, this issue is considered a pressing technical problem that the industry urgently needs to address. Summary of the Invention

[0005] In view of the technical problems existing in the prior art, the present invention provides a modeling method, apparatus, device and storage medium for a decision performance prediction model.

[0006] In a first aspect, the present invention provides a modeling method for a decision performance prediction model, comprising:

[0007] Acquire electroencephalogram (EEG) signals labeled with emotion tags and / or decision performance tags;

[0008] Based on the labeled EEG signals, emotional feature signals and decision feature signals corresponding to the EEG signals are extracted, and a first loss function, a second loss function, and a third objective function are determined; the first loss function is the loss function corresponding to the emotional feature signals; the second loss function is the loss function corresponding to the decision feature signals; and the third objective function is the mutual information objective function corresponding to the emotional feature signals and the decision feature signals.

[0009] The cross-entropy loss function corresponding to the first predicted value and the first true value is determined as the third loss function; the first predicted value is the predicted value of the decision performance output by the decision performance prediction model through the EEG signal; the first true value is the true value of the decision performance corresponding to the EEG signal.

[0010] The decision performance prediction model is obtained when the overall objective function satisfies preset conditions; the overall objective function includes the first loss function, the second loss function, the third loss function, and the objective function.

[0011] The decision performance prediction model includes:

[0012] The feature decomposition layer is used to extract emotional feature signals and decision feature signals corresponding to the labeled EEG signals;

[0013] A feature fusion layer is used to determine the maximum mutual information between the emotion feature signal and the decision feature signal;

[0014] The prediction output layer is used to output predicted values ​​of decision performance based on the decision feature signals.

[0015] Optionally, acquiring the EEG signals labeled with emotion tags and / or decision performance tags includes:

[0016] Based on the first dataset, the electroencephalogram (EEG) signals generated by different subjects were obtained; the first dataset is a dataset used for emotion recognition.

[0017] The emotional state value of the subject represented by the electroencephalogram (EEG) signal is quantified, and the corresponding emotional label is marked on the EEG signal according to the emotional state value.

[0018] The accuracy rate of the subjects' decisions based on the first dataset, as represented by the electroencephalogram (EEG) signals, is statistically analyzed.

[0019] Based on the decision accuracy represented by the EEG signals, the EEG signals are labeled with corresponding decision performance tags.

[0020] Optionally, the step of extracting emotional feature signals and decision feature signals corresponding to the labeled EEG signals, and determining a first loss function, a second loss function, and a third objective function, based on the labeled EEG signals, includes:

[0021] Based on supervised contrastive learning, with the goal of maximizing the similarity between different EEG signals with the same emotion label or the same decision label, and minimizing the similarity between different EEG signals with different emotion labels or different decision labels, the labeling EEG signals are used to extract the emotion feature signals and decision feature signals corresponding to the labeled EEG signals.

[0022] Optionally, the step of extracting emotional feature signals and decision feature signals corresponding to the labeled EEG signals, and determining a first loss function, a second loss function, and a third objective function, based on the labeled EEG signals, includes:

[0023] Determine the first loss function corresponding to the emotion feature signal and the second loss function corresponding to the decision feature signal;

[0024] Based on the deep information maximization DIM model and Jason Shannon complexity, the mutual information between the emotion feature signal and the decision feature signal is determined;

[0025] Based on the contrastive loss function, the third objective function corresponding to the mutual information is determined.

[0026] Optionally, determining the cross-entropy loss function corresponding to the first predicted value and the first true value, as the third loss function, includes:

[0027] The labeled EEG signal is input into the decision performance prediction model, and the predicted value of the decision performance corresponding to the labeled EEG signal is output as the first prediction value.

[0028] The decision performance label corresponding to the labeled EEG signal is determined as the first true value;

[0029] Based on the cross-entropy loss function, a third loss function is determined to correspond to the first predicted value and the first true value.

[0030] Optionally, the step of obtaining the decision performance prediction model when the overall objective function satisfies preset conditions includes:

[0031] Based on the first loss function, the second loss function, the third loss function, the objective function, and the relevant hyperparameters, the overall objective function is determined;

[0032] The decision performance prediction model is obtained when the overall objective function satisfies the condition that the first loss function, the second loss function, and the third loss function are minimized, and the objective function is maximized.

[0033] Secondly, the present invention also provides a method for predicting decision performance, comprising:

[0034] Acquire electroencephalogram (EEG) signals to characterize different decisions made under different emotions;

[0035] The EEG signal is input into the decision performance prediction model, and the decision feature signal corresponding to the EEG signal and the predicted value of the decision performance corresponding to the decision feature signal are output.

[0036] The decision performance prediction model is obtained based on the modeling method of the decision performance prediction model described in the first aspect.

[0037] Thirdly, the present invention also provides an apparatus for predicting decision performance, comprising:

[0038] The acquisition module is used to acquire EEG signals that represent different decisions made under different emotions;

[0039] The prediction output module is used to input the EEG signal into the decision performance prediction model and output the decision feature signal corresponding to the EEG signal, as well as the predicted value of the decision performance corresponding to the decision feature signal.

[0040] The decision performance prediction model is obtained based on the modeling method of the decision performance prediction model as described in the first aspect.

[0041] Fourthly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the modeling method for the decision performance prediction model as described in the first aspect above, and the method for predicting decision performance as described in the second aspect above.

[0042] Fifthly, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the modeling method for the decision performance prediction model as described in the first aspect above, or implements the decision performance prediction method as described in the second aspect above.

[0043] In a sixth aspect, the present invention also provides a computer program product, comprising a computer program that, when executed by a processor, implements a modeling method for a decision performance prediction model as described in the first aspect above, or implements a method for decision performance prediction as described in the second aspect above.

[0044] The decision performance prediction model provided by this invention includes a modeling method, apparatus, device, and storage medium. It acquires and labels electroencephalogram (EEG) signals of different decisions made under different emotional stimuli, using these as training samples. The model extracts emotional and decision features, as well as the mutual information between these features, from these labeled training samples. This allows for the integration of emotional features into the final decision performance prediction. This enables the prediction of decision performance based on EEG signals with different emotional characteristics, and the decision performance prediction model provided by this invention has a higher accuracy rate. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0046] Figure 1 This is a flowchart illustrating the modeling method of the decision performance prediction model provided in this embodiment of the invention;

[0047] Figure 2 This is a schematic diagram of the structure of each convolutional block provided in the embodiments of the present invention;

[0048] Figure 3 This is a flowchart illustrating the decision performance prediction method provided in an embodiment of the present invention;

[0049] Figure 4 This is a schematic diagram illustrating the implementation of the decision performance prediction model provided in this embodiment of the invention;

[0050] Figure 5 This is a schematic diagram of the structure of the decision performance prediction device provided in an embodiment of the present invention;

[0051] Figure 6 This is a schematic diagram of the physical structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0052] In this embodiment of the invention, the term "and / or" describes the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the preceding and following associated objects have an "or" relationship.

[0053] In this embodiment of the invention, the term "multiple" refers to two or more, and other quantifiers are similar.

[0054] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0055] Figure 1 This is a flowchart illustrating the modeling method for the decision performance prediction model provided in this embodiment of the invention, as shown below. Figure 1 As shown, the method includes:

[0056] Step 101: Obtain EEG signals labeled with emotion tags and / or decision performance tags;

[0057] Specifically, EEG signals, as a neurophysiological signal, reflect a user's emotional changes and possess high temporal resolution and ease of acquisition. To leverage the advantages of EEG signals, this invention uses different experimental paradigms and data collection methods to label EEG signals with emotion tags and / or decision-making performance tags. The EMOBACK experimental paradigm is a widely used method for studying the influence of emotion on cognitive and neurophysiological processes. In the EMOBACK paradigm, "EMO" stands for emotion, and "BACK" stands for background. This paradigm typically involves presenting emotional stimuli to subjects while they view documents with different visual stimuli. By manipulating the subjects' emotional states through different visual stimuli, subjects perform tasks based on the attributes of the visual stimuli. By comparing task performance under emotional and non-emotional stimuli conditions, it is possible to understand how emotion affects cognitive and neurophysiological processes. This invention collects EEG signals (data) from multiple subjects making decision-making tasks under different emotional arousal states. EEG signals generated under different emotions are labeled with corresponding emotion tags. These emotion tags can be joy, anger, sorrow, or happiness, or positive, neutral, or negative emotions, depending on different emotion classification criteria. Furthermore, when subjects perform decisions under different emotions, the correlation between emotion and decision-making can be determined in different ways. For example, when subjects perform an n-back task, a classification judgment task, or a mathematical addition task, the accuracy rate of their decision-making can be determined, i.e., the corresponding decision accuracy rate under a certain emotion can be used as the correlation between that emotion and decision-making. A quantifiable method is then used to evaluate this correlation between emotion and decision-making, determining the corresponding decision performance, and labeling the EEG signals generated when making the above decisions under that emotion with corresponding decision performance tags.

[0058] Step 102: Based on the labeled EEG signals, extract the emotion feature signals and decision feature signals corresponding to the EEG signals, and determine the first loss function, the second loss function, and the third objective function; the first loss function is the loss function corresponding to the emotion feature signals; the second loss function is the loss function corresponding to the decision feature signals; the third objective function is the mutual information objective function corresponding to the emotion feature signals and the decision feature signals.

[0059] EEG signals are dynamically changing over time, and brain channels can receive neural activity from multiple sources. After obtaining EEG signals labeled with emotion tags and / or decision performance tags through the above-mentioned emotional background experimental paradigm, in order to make full use of this feature, the labeled EEG signals can be processed through the spatiotemporal convolutional layer in the decision performance prediction model, i.e., the convolution block E1 composed of temporal and spatial convolutions, to obtain EEG signals with spatiotemporal characteristics, thus making the representation of EEG signals richer.

[0060] After the above processing, it is also necessary to extract the emotional feature signals and decision feature signals from the EEG signals with spatiotemporal characteristics. Specifically, based on the emotion labels and / or decision performance labels marked on the EEG signals, the emotional feature signals f in the EEG signals can be extracted from the convolutional blocks E2 and E3 included in the feature decomposition layer of the decision performance prediction model. e =E2(E1(x)) and / or decision feature signal f t =E3(E1(x)). For example, under the stimulation of a certain emotion at a certain starting moment, the subject makes some decisions and records the ending time. The EEG signal within the time period determined by the starting and ending times can be used as the decision feature signal corresponding to the EEG signal, and the EEG signal within a time period centered on the starting time can be used as the emotion feature signal corresponding to the EEG signal. Here, the time period can be 2 seconds before and after or 0.5 seconds. Of course, the above is only an illustrative description and does not limit how to extract the emotion feature signal and decision feature of the EEG signal in this invention. Schematic diagram of the structure of convolutional blocks E1, E2, E3, as shown below. Figure 2 As shown.

[0061] After extracting the emotional and / or decision-making feature signals from the aforementioned EEG signals, based on the set of emotional feature signals, a first loss function corresponding to the emotional feature signal and a second loss function corresponding to the decision-making feature signal of each EEG signal are determined when the EEG signal is used as a training sample. Then, based on the emotional and decision-making feature signals corresponding to the EEG signals, the feature fusion layer in the decision performance prediction model determines the maximum mutual information between the emotional and decision-making feature signals, and subsequently determines the objective function corresponding to this mutual information as the third objective function.

[0062] Step 103: Determine the cross-entropy loss function corresponding to the first predicted value and the first true value, as the third loss function; the first predicted value is the predicted value of the decision performance output by the decision performance prediction model through the EEG signal; the first true value is the true value of the decision performance corresponding to the EEG signal.

[0063] After inputting the aforementioned decision feature signal into the prediction output layer of the decision performance prediction model proposed in this invention, the prediction result of the decision performance corresponding to the EEG signal is output as the first predicted value. Combined with the first true value determined by the decision performance label corresponding to the EEG signal, the cross-entropy loss function corresponding to the first predicted value and the first true value is determined as the third loss function.

[0064] Step 104: Under the condition that the overall objective function meets the preset conditions, the decision performance prediction model is obtained; the overall objective function includes the first loss function, the second loss function, the third loss function and the third objective function.

[0065] Based on the first, second, third, and third loss functions and the third objective function mentioned above, the overall objective function is determined. This involves end-to-end training, jointly optimizing all loss functions and the objective function to obtain the set of parameters that best optimizes the prediction performance of the decision performance prediction model. In other words, the overall objective function must satisfy preset conditions. These preset conditions are pre-defined conditions used to maximize the prediction performance of the decision performance prediction model. These conditions could be minimizing all loss functions while maximizing the objective function; alternatively, depending on the specific scenario, certain loss functions or objective functions could be minimized while the rest are maximized; or vice versa. The decision performance prediction model is trained and obtained when the overall objective function satisfies the preset conditions.

[0066] The modeling method for the decision performance prediction model provided by this invention acquires and labels electroencephalogram (EEG) signals of different decisions made under different emotional stimuli, using these signals as training samples. It then extracts emotional and decision features, as well as the mutual information between these emotional and decision features, from these labeled training samples. This allows the final decision performance prediction to incorporate emotional features. This enables the prediction of decision performance based on EEG signals with different emotional characteristics, and the decision performance prediction model provided by this invention has a higher accuracy rate.

[0067] Optionally, acquiring the EEG signals labeled with emotion tags and / or decision performance tags includes:

[0068] Based on the first dataset, the electroencephalogram (EEG) signals generated by different subjects were obtained; the first dataset is a dataset used for emotion recognition.

[0069] The emotional state value of the subject represented by the electroencephalogram (EEG) signal is quantified, and the corresponding emotional label is marked on the EEG signal according to the emotional state value.

[0070] The accuracy rate of the subjects' decisions based on the first dataset, as represented by the electroencephalogram (EEG) signals, is statistically analyzed.

[0071] Based on the decision accuracy represented by the EEG signals, the EEG signals are labeled with corresponding decision performance tags.

[0072] Specifically, the methods for labeling EEG signals with emotion tags and / or decision-making performance tags are as follows:

[0073] Based on the first dataset, EEG signals generated by different subjects under the stimulation of the first dataset were obtained. This first dataset is used to evoke specific emotions in the subjects and can employ various datasets used for emotion research. Different emotion labels can be determined based on different categorization criteria within the selected dataset. For example, the Geneva Affective Picture Database (GAPED) from the EMOBACK experimental paradigm can be used as the first dataset. This dataset contains 730 images, categorized into four specific negative content types: spiders, snakes, and scenes that evoke negative emotions. Positive and neutral images are also included: positive images mainly represent humans, baby animals, and natural landscapes, while neutral images mainly depict inanimate objects. These images are used as emotional stimuli, and EEG signals with corresponding emotions generated by different subjects when viewing these images are obtained. Alternatively, the International Affective Picture System (IAPS), containing over 10,000 images, can be used as the first dataset, widely used in emotion research. This dataset has detailed categorization, including five subcategories: highly positive, low-positive, neutral, low-negative, and highly negative. Alternatively, the Nencki Affective Picture System (NAPS), comprising 1,356 images, can be used as the first dataset, categorized into positive, negative, and neutral emotions based on the evoked emotional responses. Alternatively, the Affective Image Database (AID), comprising 1,000 images, can be used as the first dataset, categorized according to subjective emotional responses into five emotion categories: happiness, tranquility, surprise, sadness, and anger. In other words, the visual stimuli generated by the participants viewing the information in the first dataset will evoke specific emotions in the participants.

[0074] To quantify participants' emotions, different quantification standards can be used depending on the initial dataset, or different standards can be directly selected. Specifically, the Self-Assessment Manikin Scale (SAM), also known as the Self-Assessment Manikin scale, can be used to collect and assess participants' valence and arousal. The SAM is a commonly used subjective emotion assessment tool that can quickly and intuitively measure an individual's response to different emotional states. Based on the participants' SAM scores after emotional arousal, emotions can be categorized into different levels, and based on these levels, they can be determined as positive, neutral, or negative emotions. For example, emotions with a SAM score higher than 5 are defined as positive emotions, those with a SAM score of 5 are defined as neutral emotions, and those with a SAM score lower than 5 are defined as negative emotions, thus obtaining the corresponding emotion labels for the participants. These are represented by positive, neutral, and negative emotions, respectively. Alternatively, the Positive and Negative Affect Schedule (PANAS) can be used, which assesses positive and negative emotions separately. It includes N items (N ≥ 20), each consisting of a descriptive word, such as happy, depressed, etc. A 5-point scale is used to measure the degree of the participant's feeling about each item. This yields the corresponding emotion label for the participant. The value of i is determined based on the total number of items included in the positive and negative emotion scales.

[0075] In addition, participants performed decision-making tasks while viewing these images, specifically using the N-back visual sequential stimulus memory task, which includes two different tasks: 1-Back and 2-Back. In the 1-Back task, participants need to judge whether the current visual stimulus is consistent with the stimulus one step back; in the 2-Back task, participants need to judge whether the current visual stimulus is consistent with the stimulus two steps back. Simply put, in the 1-Back task, participants need to report whether the current number matches the previous number each time; if they match, they press "Yes"; if they don't, they press "No." In the 2-Back task, participants need to report whether the current number matches the previous two numbers each time; if they match, they press "Yes"; if they don't match, they press "No." The accuracy rate of the participants performing the task multiple times is calculated. The number of times can be set according to needs; naturally, the more times, the higher the accuracy of the decision performance prediction model, but the more time-consuming it is. If the sample size is large, the sampling time may be immeasurable. Therefore, to balance accuracy and timeliness, it is usually set to around 100.

[0076] The K-means algorithm was used to cluster participants based on their accuracy in performing 1-back tasks and 2-back tasks, with each cluster consisting of 2^n clusters. This yielded decision performance labels representing the participants' decision-making performance levels. These represent low and high decision-making performance, respectively. The decision-making performance labels corresponding to EEG signals are determined using the above method. Furthermore, tasks requiring participants to make decisions, such as mathematical addition or classification tasks, can also be used, as long as a reasonable experimental paradigm and data collection method are established.

[0077] EEG signals labeled with emotion tags and / or decision performance tags can be represented as follows: Where s∈{1,...,S} represents the s-th subject, and n s This represents the total number of labeled EEG signals for the s-th subject. This represents the i-th labeled EEG signal of the s-th subject. This represents the emotion label of the i-th EEG signal in the s-th subject. The decision label represents the EEG signal marker of the s-th subject.

[0078] Optionally, the step of extracting emotional feature signals and decision feature signals corresponding to the labeled EEG signals, and determining a first loss function, a second loss function, and a third objective function, based on the labeled EEG signals, includes:

[0079] Based on supervised contrastive learning, with the goal of maximizing the similarity between different EEG signals with the same emotion label or the same decision label, and minimizing the similarity between different EEG signals with different emotion labels or different decision labels, the labeling EEG signals are used to extract the emotion feature signals and decision feature signals corresponding to the labeled EEG signals.

[0080] Specifically, based on the emotion labels and / or decision labels corresponding to the aforementioned EEG signals, two convolutional neural network (CNN) blocks are used to obtain the emotion feature signals and decision feature signals corresponding to the EEG signals. Then, a supervised contrastive learning method is used to maximize the similarity between two EEG signals with the same emotion label or the same decision performance label and minimize the similarity between two EEG signals with different emotion labels or different decision performance labels, so as to extract the emotion-related features and / or decision-related features from the EEG signals, which are respectively used as emotion feature signals and decision feature signals.

[0081] Optionally, the step of extracting emotional feature signals and decision feature signals corresponding to the labeled EEG signals, and determining a first loss function, a second loss function, and a third objective function, based on the labeled EEG signals, includes:

[0082] Determine the first loss function corresponding to the emotion feature signal and the second loss function corresponding to the decision feature signal;

[0083] Based on the deep information maximization DIM model and Jason Shannon complexity, the mutual information between the emotion feature signal and the decision feature signal is determined;

[0084] Based on the contrastive loss function, the third objective function corresponding to the mutual information is determined.

[0085] Specifically, the emotional and decision-making feature signals corresponding to the EEG signals are extracted and can be represented as f. e and f t Each labeled EEG signal serves as a training sample i. All labeled EEG signals constitute the complete sample set A(i). Emotional feature signals are EEG signals corresponding to emotion labels. Positive and negative samples of emotional feature signals are distinguished: samples with the same label are positive samples of each other, and samples with different labels are negative samples of each other. Decision feature signals correspond to EEG signals labeled with decision performance labels. Positive and negative samples of decision feature signals are distinguished: samples with the same label are positive samples of each other, and samples with different labels are negative samples of each other.

[0086] The first loss function corresponding to the emotion feature signal can be expressed as:

[0087]

[0088] in, Let P(i) represent the loss function (first loss function) corresponding to the emotional feature signal, P(i) be the set of positive samples corresponding to the emotional feature signal, |P(i)| be the number of elements in the set of positive samples, and A(i) be the entire sample set. This represents the emotional feature signal corresponding to the i-th EEG signal. This represents the emotional feature signal corresponding to the p-th positive sample. Let represent the emotional feature signal corresponding to the a-th sample, p∈P(i) represent any positive sample p in the set of positive samples P(i), a∈A(i) represent any sample a in the set of all samples A(i), exp() represents an exponential function with the natural constant e as the base, and τ represents the temperature coefficient, which is used to prevent the log function from tending to infinity. Here, the value τ is generally taken as 1.

[0089] The second loss function corresponding to the decision feature signal can be expressed as:

[0090]

[0091] in, The loss function (second loss function) represents the decision feature signal, and the meanings of other related parameters are the same as those in the first loss function.

[0092] Based on the Deep Infomax (DIM) model and the Jensen-Shannon (JS) divergence, the mutual information between the emotion feature signal and the decision feature signal is determined; that is, a mutual information estimator M is introduced to maximize the estimated emotion feature signal f. e and decision characteristic signal f t The mutual information between two random variables is crucial. The DIM model is a neural network model used for unsupervised learning, its main function being to maximize the mutual information in the input data. Mutual information is a metric that measures the correlation between two random variables. In the DIM model, the input data is first transformed into a latent vector representation through an encoder network, and then a contrastive loss function is used to maximize the mutual information in the latent vector representation. The contrastive loss function is a commonly used loss function in unsupervised learning; it is based on the similarity and difference between different parts of the input data and is used to force the model to learn useful information from the data. JS divergence is a metric used to measure the similarity between two probability distributions; it can be used to measure the similarity between the features learned in the latent space encoder and the distribution of the original data. In the DIM model, emotion feature signals and decision feature signals are transformed into vector representations in the latent space, and then the mutual information of the above vector representations is more accurately represented by maximizing the JS divergence. Furthermore, the neural network parameters can be optimized through backpropagation to learn a representation with high representational power. Assuming the input data are x, y, z, the mutual information between x and y can be expressed as: sp(z) = log(1 + e) z );

[0093] Among them, M Θ (X; Y) represents the mutual information between all elements in set X and set Y. This indicates -sp(-T) θ The mathematical expectation of the joint distribution of (x,y) is -sp(-T). θ (x,y) represents -T θ The softplus function value of (x,y), T θ (x,y) represents the output value of a fully connected layer with x and y as input vectors and θ as input parameters. sp(T) θ The mathematical expectation of the marginal distribution function of (x,y) is sp(T).θ (x,y) represents T θ The softplus function value of (x,y), where x represents any element in set X and y represents any element in set Y.

[0094] The mutual information between the emotional feature signal and the decision feature signal, corresponding to the third objective function, can be expressed as:

[0095] in, Let represent the mutual information objective function (i.e., the third objective function). This represents the mutual information between the emotional feature signal corresponding to the i-th EEG signal and the decision feature signal corresponding to the i-th EEG signal, and N represents the number of samples in each batch of all EEG signals. The key point is that during training, the sample data is not input all at once, but in batches, where N represents the number of samples in each batch.

[0096] Optionally, determining the cross-entropy loss function corresponding to the first predicted value and the first true value, as the third loss function, includes:

[0097] The labeled EEG signal is input into the decision performance prediction model, and the predicted value of the decision performance corresponding to the labeled EEG signal is output as the first prediction value.

[0098] The decision performance label corresponding to the labeled EEG signal is determined as the first true value;

[0099] Based on the cross-entropy loss function, a third loss function is determined to correspond to the first predicted value and the first true value.

[0100] Specifically, after the labeled EEG signal is input into the decision performance prediction model, the model outputs the predicted value of the decision performance corresponding to the labeled EEG signal, which can be represented as y. pred This is achieved through a fully connected layer classifier C, which processes the decision feature signal f. t Input classifier C, output the prediction result of decision performance. The classifier C is trained by minimizing the cross-entropy loss. The goal of the cross-entropy loss function is to minimize the difference between the model's predicted result and the true label. During training, the backpropagation algorithm is used to adjust the model parameters to minimize the value of the cross-entropy loss function, thereby improving the accuracy of the decision performance prediction model. The third loss function corresponding to the first predicted value and the first true value can be expressed as:

[0101]

[0102] in, Let y represent the cross-entropy loss function (i.e., the third loss function) of the decision feature signal corresponding to the i-th EEG signal. (i) This represents the decision feature label corresponding to the i-th EEG signal; This represents the predicted decision performance value of the decision feature signal corresponding to the i-th EEG signal.

[0103] Optionally, the step of obtaining the decision performance prediction model when the overall objective function satisfies preset conditions includes:

[0104] Based on the first loss function, the second loss function, the third loss function, the objective function, and the relevant hyperparameters, the overall objective function is determined;

[0105] If the overall objective function satisfies the condition that the first loss function, the second loss function, and the third loss function are minimized, and the objective function is maximized, then the training of the decision performance prediction model is completed, and the decision performance prediction model is obtained.

[0106] Specifically, the decision performance prediction model is trained in an end-to-end manner, jointly optimizing all loss functions and objective functions to obtain a set of parameters that best enables the decision performance prediction model to make predictions.

[0107] Based on the first loss function, the second loss function, the third loss function, the objective function, and the relevant hyperparameters, the overall objective function is determined, which can be expressed as:

[0108]

[0109] Where α, β, γ, and κ are hyperparameters controlling the three loss functions and the objective function, E1 represents a convolutional block consisting of temporal and spatial convolutions, E2 represents a convolutional block used to extract sentiment feature signals, E3 represents a convolutional block used to extract decision feature signals, and M represents the mutual information estimator. Let represent the cross-entropy loss function (i.e., the third loss function) of the decision feature signal corresponding to the i-th EEG signal. The loss function (first loss function) represents the loss function corresponding to the emotional feature signal. The loss function (second loss function) corresponding to the decision feature signal is represented. This represents the mutual information objective function (i.e., the third objective function). The hyperparameters α, β, γ, and κ can be set in advance and adjusted according to the actual application scenario or the results of the operation.

[0110] The modeling method for the decision performance prediction model provided by this invention acquires and labels electroencephalogram (EEG) signals of different decisions made under different emotional stimuli, using these signals as training samples. It then extracts emotional and decision features, as well as the mutual information between these emotional and decision features, from these labeled training samples. This allows the final decision performance prediction to incorporate emotional features. This enables the prediction of decision performance based on EEG signals with different emotional characteristics, and the decision performance prediction model provided by this invention has a higher accuracy rate.

[0111] Figure 3 This is a flowchart illustrating the decision performance prediction method provided in an embodiment of the present invention, as shown below. Figure 3 As shown, the method includes:

[0112] Step 301: Obtain EEG signals to represent different decisions made under different emotions;

[0113] Step 302: Input the EEG signal into the decision performance prediction model, and output the decision feature signal corresponding to the EEG signal and the predicted value of the decision performance corresponding to the decision feature signal;

[0114] The decision performance prediction model is obtained based on the modeling method of the decision performance prediction model in any of the embodiments.

[0115] Specifically, various electroencephalogram (EEG) signals are acquired through instruments, which represent different decisions made by different users under different emotional states.

[0116] By inputting these EEG signals into a trained decision performance prediction model, the model can output the corresponding decision characteristic signal and the predicted decision performance value. This can be understood as directly predicting decision performance based on EEG signals. It is one approach to studying the relationship between emotion and decision-making, and it quantitatively assesses the impact of emotion on decision performance to determine the level of decision performance.

[0117] For example, the decision performance prediction model includes a spatiotemporal signal convolution module, a feature decomposition module, and a prediction output module. The model inputs an electroencephalogram (EEG) signal to obtain a decision feature signal corresponding to the EEG signal, and a predicted value of the decision performance corresponding to the decision feature signal, including:

[0118] The EEG signal is input into the spatiotemporal signal convolution module to obtain an EEG signal with spatiotemporal characteristics, which is used as the preprocessed EEG signal.

[0119] The preprocessed EEG signal is input into the feature decomposition module to extract the emotional feature signal and decision feature signal corresponding to the preprocessed EEG signal;

[0120] The decision feature signal is input into the prediction output module, which outputs the predicted value of the decision performance corresponding to the decision feature signal.

[0121] It should be noted that the specific implementation of the decision performance prediction method in this embodiment of the invention is the same as described above. Figure 1 The specific implementation of the modeling method for the decision performance prediction model in the illustrated embodiments is similar, and can be found in the above. Figure 1 The specific implementation of the modeling method for the decision performance prediction model in the illustrated embodiment will not be described again here.

[0122] To more clearly illustrate the modeling method of the decision performance prediction model provided in the embodiments of this application, specific examples are given below.

[0123] Figure 4 This is a schematic diagram illustrating the implementation of the decision performance prediction model provided in this embodiment of the invention, such as... Figure 4 As shown, it includes:

[0124] The spatiotemporal convolution module is used to take the input labeled EEG signal and use a convolution block E1 composed of temporal and spatial convolution to obtain an EEG signal with spatiotemporal characteristics; it can enrich the temporal and spatial representation of the input EEG signal.

[0125] The feature decomposition module is used to extract emotional feature signals and / or decision feature signals corresponding to the EEG signal through convolutional blocks E2 and E3 composed of two convolutional neural networks (CNNs), respectively, and to determine the first loss function corresponding to the emotional feature signal. The second loss function corresponding to the decision feature signal

[0126] The feature fusion module is used to maximize the mutual information between the emotion feature signal and the decision feature signal through the mutual information estimator M, and to determine the objective function corresponding to this mutual information, i.e., the third objective function.

[0127] The prediction output module is used to input the decision feature signal into the classifier C of the fully connected layer, output the prediction result of the decision performance, and determine the corresponding cross-entropy loss function based on the prediction result and the decision label corresponding to the decision feature signal. The classifier is trained by minimizing the cross-entropy loss;

[0128] When the overall objective function, composed of the first loss function, the second loss function, the third loss function, and the third objective function, satisfies preset conditions, the training of the decision performance prediction model is completed, and the decision performance prediction model is obtained.

[0129] Figure 5 This is a schematic diagram of the structure of the decision performance prediction device provided in an embodiment of the present invention, as shown below. Figure 5 As shown, the device includes:

[0130] The acquisition module 501 is used to acquire electroencephalogram (EEG) signals that represent different decisions made under different emotions;

[0131] The prediction output module 502 is used to input the electroencephalogram (EEG) signal into the decision performance prediction model, and output the decision feature signal corresponding to the EEG signal, as well as the predicted value of the decision performance corresponding to the decision feature signal.

[0132] The decision performance prediction model is obtained based on the modeling method of any of the decision performance prediction models described above.

[0133] The decision performance prediction apparatus provided in this embodiment of the invention can execute the technical solution of the decision performance prediction method in any of the above embodiments. Its implementation principle and beneficial effects are similar to those of the decision performance prediction method. Please refer to the implementation principle and beneficial effects of the decision performance prediction method. It will not be repeated here.

[0134] Figure 6 This is a schematic diagram of the physical structure of the electronic device provided in the embodiments of the present invention, such as... Figure 6 As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute a modeling method for a decision performance prediction model, the method including:

[0135] Acquire electroencephalogram (EEG) signals labeled with emotion tags and / or decision performance tags; based on the labeled EEG signals, extract the corresponding emotion feature signals and decision feature signals, and determine a first loss function, a second loss function, and a third objective function; the first loss function is the loss function corresponding to the emotion feature signals; the second loss function is the loss function corresponding to the decision feature signals; the third objective function is the mutual information objective function corresponding to the emotion feature signals and the decision feature signals; determine the cross-entropy loss function corresponding to the first predicted value and the first true value as the third loss function; the first predicted value is the decision value output by the decision performance prediction model from the EEG signals. The predicted value of performance; the first true value is the true value of decision performance corresponding to the EEG signal; the decision performance prediction model is obtained when the overall objective function satisfies preset conditions; the overall objective function includes the first loss function, the second loss function, the third loss function, and the third objective function; wherein, the decision performance prediction model includes: a feature decomposition layer, used to extract emotional feature signals and decision feature signals corresponding to the labeled EEG signal; a feature fusion layer, used to determine the maximum mutual information between the emotional feature signals and the decision feature signals; and a prediction output layer, used to output the predicted value of decision performance based on the decision feature signals.

[0136] Alternatively, a method for predicting decision performance provided by the above methods may be implemented, the method comprising: acquiring electroencephalogram (EEG) signals to characterize different decisions made under different emotions; inputting the EEG signals into a decision performance prediction model, outputting a decision feature signal corresponding to the EEG signals, and a predicted value of decision performance corresponding to the decision feature signals; wherein the decision performance prediction model is obtained based on the modeling method of the above decision performance prediction model.

[0137] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0138] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the modeling method of the decision performance prediction model provided by the above methods. The method includes: acquiring electroencephalogram (EEG) signals labeled with emotion tags and / or decision performance tags; extracting emotion feature signals and decision feature signals corresponding to the labeled EEG signals based on the labeled EEG signals, and determining a first loss function, a second loss function, and a third objective function; the first loss function is the loss function corresponding to the emotion feature signals; the second loss function is the loss function corresponding to the decision feature signals; the third objective function is the mutual information objective function corresponding to the emotion feature signals and the decision feature signals; determining a first predicted value and a second predicted value. A cross-entropy loss function corresponding to a true value is used as the third loss function; the first predicted value is the predicted value of the decision performance output by the decision performance prediction model from the EEG signal; the first true value is the true value of the decision performance corresponding to the EEG signal; the decision performance prediction model is obtained when the overall objective function satisfies preset conditions; the overall objective function includes the first loss function, the second loss function, the third loss function, and the third objective function; wherein, the decision performance prediction model includes: a feature decomposition layer, used to extract emotional feature signals and decision feature signals corresponding to the labeled EEG signal; a feature fusion layer, used to determine the maximum mutual information between the emotional feature signals and the decision feature signals; and a prediction output layer, used to output the predicted value of the decision performance based on the decision feature signals.

[0139] Alternatively, a method for predicting decision performance provided by the above methods may be implemented, the method comprising: acquiring electroencephalogram (EEG) signals to characterize different decisions made under different emotions; inputting the EEG signals into a decision performance prediction model, outputting a decision feature signal corresponding to the EEG signals, and a predicted value of decision performance corresponding to the decision feature signals; wherein the decision performance prediction model is obtained based on the above-described modeling method for decision performance prediction.

[0140] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a modeling method for the decision performance prediction model provided by the methods described above. This method includes: acquiring electroencephalogram (EEG) signals labeled with emotion tags and / or decision performance tags; extracting emotion feature signals and decision feature signals corresponding to the labeled EEG signals based on the labeled EEG signals, and determining a first loss function, a second loss function, and a third objective function; wherein the first loss function is the loss function corresponding to the emotion feature signals; the second loss function is the loss function corresponding to the decision feature signals; the third objective function is the mutual information objective function corresponding to the emotion feature signals and the decision feature signals; and determining a cross-entropy loss function corresponding to a first predicted value and a first true value. The first predicted value is the predicted value of the decision performance output by the decision performance prediction model from the EEG signal; the first true value is the true value of the decision performance corresponding to the EEG signal; the decision performance prediction model is obtained when the overall objective function satisfies the preset conditions; the overall objective function includes the first loss function, the second loss function, the third loss function, and the third objective function; wherein, the decision performance prediction model includes: a feature decomposition layer, used to extract the emotional feature signal and the decision feature signal corresponding to the labeled EEG signal; a feature fusion layer, used to determine the maximum mutual information between the emotional feature signal and the decision feature signal; and a prediction output layer, used to output the predicted value of the decision performance based on the decision feature signal.

[0141] Alternatively, a method for predicting decision performance provided by the above methods may be implemented, the method comprising: acquiring electroencephalogram (EEG) signals to characterize different decisions made under different emotions; inputting the EEG signals into a decision performance prediction model, outputting a decision feature signal corresponding to the EEG signals, and a predicted value of decision performance corresponding to the decision feature signals; wherein the decision performance prediction model is obtained based on the above-described modeling method for decision performance prediction.

[0142] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0143] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0144] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A modeling method for a decision performance prediction model, characterized in that, include: Acquire electroencephalogram (EEG) signals labeled with emotion tags and / or decision performance tags; Based on the labeled EEG signals, emotional feature signals and decision feature signals corresponding to the EEG signals are extracted, and a first loss function, a second loss function, and a third objective function are determined; the first loss function is the loss function corresponding to the emotional feature signals; the second loss function is the loss function corresponding to the decision feature signals; and the third objective function is the mutual information objective function corresponding to the emotional feature signals and the decision feature signals. The cross-entropy loss function corresponding to the first predicted value and the first true value is determined as the third loss function; the first predicted value is the predicted value of the decision performance output by the decision performance prediction model of the electroencephalogram signal. The first true value is the true value of the decision performance corresponding to the electroencephalogram signal; The decision performance prediction model includes: a feature decomposition layer, used to extract emotional feature signals and decision feature signals corresponding to the labeled EEG signals; a feature fusion layer, used to determine the maximum mutual information between the emotional feature signals and the decision feature signals; and a prediction output layer, used to output a predicted value of decision performance based on the decision feature signals. Under the condition that the overall objective function satisfies the preset conditions, a well-trained decision performance prediction model is obtained; the overall objective function includes the first loss function, the second loss function, the third loss function, and the third objective function.

2. The modeling method for the decision performance prediction model according to claim 1, characterized in that, The acquisition of EEG signals labeled with emotion tags and / or decision performance tags includes: Based on the first dataset, the electroencephalogram (EEG) signals generated by different subjects were obtained; the first dataset is a dataset used for emotion recognition. The emotional state value of the subject represented by the electroencephalogram (EEG) signal is quantified, and the corresponding emotional label is marked on the EEG signal according to the emotional state value. The accuracy rate of the subjects' decisions based on the first dataset, as represented by the electroencephalogram (EEG) signals, is statistically analyzed. Based on the decision accuracy represented by the EEG signals, the EEG signals are labeled with corresponding decision performance tags.

3. The modeling method for the decision performance prediction model according to claim 2, characterized in that, Based on the labeled EEG signal, the emotional feature signal and decision feature signal corresponding to the EEG signal are extracted, and a first loss function, a second loss function and a third objective function are determined, including: Based on supervised contrastive learning, with the goal of maximizing the similarity between different EEG signals with the same emotion label or the same decision label, and minimizing the similarity between different EEG signals with different emotion labels or different decision labels, the labeling EEG signals are used to extract the emotion feature signals and decision feature signals corresponding to the labeled EEG signals.

4. The modeling method for the decision performance prediction model according to claim 3, characterized in that, Based on the labeled EEG signal, the emotional feature signal and decision feature signal corresponding to the EEG signal are extracted, and a first loss function, a second loss function and a third objective function are determined, including: Determine the first loss function corresponding to the emotion feature signal and the second loss function corresponding to the decision feature signal; Based on the depth information maximization DIM model and Jason Shannon divergence, the mutual information between the emotion feature signal and the decision feature signal is determined; Based on the contrastive loss function, the third objective function corresponding to the mutual information is determined.

5. The modeling method for the decision performance prediction model according to claim 4, characterized in that, The step of determining the cross-entropy loss function corresponding to the first predicted value and the first true value, as the third loss function, includes: The labeled EEG signal is input into the decision performance prediction model, and the predicted value of the decision performance corresponding to the labeled EEG signal is output as the first prediction value. The decision performance label corresponding to the labeled EEG signal is determined as the first true value; Based on the cross-entropy loss function, a third loss function is determined to correspond to the first predicted value and the first true value.

6. The modeling method for the decision performance prediction model according to claim 5, characterized in that, The process of obtaining a trained decision performance prediction model when the overall objective function satisfies preset conditions includes: Based on the first loss function, the second loss function, the third loss function, the third objective function, and the relevant hyperparameters, the overall objective function is determined; If the overall objective function satisfies the condition that the first loss function, the second loss function, and the third loss function are minimized, and the third objective function is maximized, then the training of the decision performance prediction model is completed, and the trained decision performance prediction model is obtained.

7. A method for predicting decision performance, characterized in that, include: Acquire electroencephalogram (EEG) signals to characterize different decisions made under different emotions; The EEG signal is input into the trained decision performance prediction model, and the decision feature signal corresponding to the EEG signal and the predicted value of decision performance corresponding to the decision feature signal are output. The trained decision performance prediction model is obtained based on the modeling method of the decision performance prediction model as described in any one of claims 1 to 6.

8. A device for predicting decision performance, characterized in that, include: The acquisition module is used to acquire EEG signals that represent different decisions made under different emotions; The prediction output module is used to input the EEG signal into the trained decision performance prediction model and output the decision feature signal corresponding to the EEG signal, as well as the predicted value of the decision performance corresponding to the decision feature signal. The trained decision performance prediction model is obtained based on the modeling method of the decision performance prediction model as described in any one of claims 1 to 6.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the program, it implements the modeling method of the decision performance prediction model as described in any one of claims 1 to 6; or, it implements the decision performance prediction method as described in claim 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the modeling method of the decision performance prediction model as described in any one of claims 1 to 6; or, it implements the decision performance prediction method as described in claim 7.

Images