An emotion state detection method and system based on adversarial training

Abstract

The application provides an emotion state detection method and system based on adversarial training, the method comprising collecting electroencephalogram signals of different experimental paradigms and / or different subjects, and extracting differential entropy features of the electroencephalogram signals; dividing the differential entropy features into source domain data and target domain data; generating source domain differential entropy features according to the source domain data and target domain differential entropy features according to the target domain data; adjusting a dynamic adversarial factor according to the source domain differential entropy features and the target domain differential entropy features, and generating an emotion recognition model based on the adjusted dynamic adversarial factor; inputting the target domain differential entropy features into the emotion recognition model to perform detection on the target domain differential entropy features, and generating a detection result of the emotion state. The method can generate an emotion recognition model through adversarial training, and more common information between domains can be obtained through the emotion recognition model, so that the emotion recognition model can more accurately distinguish the emotion state, and the accuracy of emotion state detection is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to an emotion state detection method and system based on adversarial training. BACKGROUND

[0002] Neural cells in the brain communicate with each other through some weak neural signals, and produce some measurable changes, which can be referred to as brain waves. In an implementation, the changes can be recorded by an electroencephalogram (EEG) method.

[0003] Exemplarily, the brain waves can be a reflection of brain activity and state, reflecting physiological signals emitted by humans at different cognitive processing levels in different tasks or different psychological states. Therefore, the brain activity can be recorded by the brain wave changes, and the emotion state can be reflected by the electroencephalogram EEG. In the process of reflecting the emotion state by the electroencephalogram EEG, different brain wave signals are required to have the same feature distribution. However, due to the influence of different collection environments, different experimental paradigms, different collection devices, different subject sources and other factors, there are large distribution differences between different brain wave signals. Therefore, the distribution differences of the brain wave signals lead to a lack of objective diagnostic criteria for the detection results of the emotion state, resulting in inaccurate detection results of the emotion state. In addition, the distribution differences of the brain wave signals will lead to the fact that the emotion state detection method across subjects or paradigms cannot be applied to the emotion state detection method of other subjects or paradigms, and when there is a detection task of a new subject, a large amount of labeled brain wave signals are also required, which will also lead to a large amount of work and low detection efficiency of the emotion state. SUMMARY

[0004] Some embodiments of the present application provide an emotion state detection method and system based on adversarial training to solve the problem of inaccurate detection results of the emotion state caused by the distribution differences of the brain wave signals.

[0005] In one aspect, the present application provides an emotion state detection method based on adversarial training, comprising:

[0006] Collecting brain wave signals, the brain wave signals being brain wave signals of different experimental paradigms and / or different subjects;

[0007] Extracting differential entropy features of the brain wave signals;

[0008] Dividing the differential entropy features into source domain data and target domain data;

[0009] Generating source domain differential entropy features according to the source domain data, and generating target domain differential entropy features according to the target domain data;

[0010] Adjust a dynamic adversarial factor according to the source domain differential entropy feature and the target domain differential entropy feature, and generate an emotion recognition model based on the adjusted dynamic adversarial factor;

[0011] Input the target domain differential entropy feature into the emotion recognition model to perform detection on the target domain differential entropy feature, and generate a detection result of an emotion state.

[0012] In some embodiments, the emotion state detection method based on adversarial training further comprises:

[0013] Extract a differential entropy feature in the brain wave signal according to the lead and the frequency band identifier.

[0014] In some embodiments, the step of dividing the differential entropy feature into source domain data and target domain data comprises:

[0015] Obtain the differential entropy feature;

[0016] Calculate the differential entropy of each lead and frequency band according to the lead and the frequency band identifier.

[0017] According to the differential entropy of each lead and frequency band, divide the differential entropy feature into source domain data and target domain data based on a cross-subject or paradigm manner.

[0018] In some embodiments, the step of performing preprocessing on the brain wave signal comprises:

[0019] Delete baseline drift in the brain wave signal;

[0020] Use a band-limited filter to perform noise removal on the brain wave signal after deleting the baseline drift;

[0021] Use a low-pass filter to perform noise removal again on the brain wave signal after noise removal;

[0022] Delete artifact signals in the brain wave data after secondary noise removal to obtain valid brain wave signals, wherein the artifact signals are brain wave signals that do not meet preset conditions.

[0023] In some embodiments, the steps of generating a source domain differential entropy feature according to the source domain data and generating a target domain differential entropy feature according to the target domain data comprise:

[0024] Input the source domain data and the target domain data into a feature extractor, respectively;

[0025] Perform feature extraction on the source domain data in the feature extractor to obtain a source domain differential entropy feature;

[0026] The feature extractor extracts features from the target domain data to obtain target domain differential entropy features.

[0027] In some embodiments, the emotion state detection method based on adversarial training further comprises:

[0028] An optimal distance is calculated according to the source domain differential entropy features and the target domain differential entropy features, the optimal distance being a minimum distance of transfer between the source domain differential entropy features and the target domain differential entropy features;

[0029] A forward distribution transfer vector and a reverse distribution transfer vector are generated based on the optimal distance;

[0030] The forward distribution transfer vector and the reverse distribution transfer vector are input into a distribution transformation discriminator to obtain a domain prediction result;

[0031] A cross-entropy loss of the optimal distance is calculated according to the domain prediction result;

[0032] The gradient of the cross-entropy loss is back-propagated.

[0033] In some embodiments, the emotion state detection method based on adversarial training further comprises:

[0034] The source domain differential entropy features and the target domain differential entropy features are input into a global domain discriminator to obtain a global domain discrimination result;

[0035] A global domain discrimination loss function is calculated according to the global domain discrimination result;

[0036] The gradient of the global domain discrimination loss function is back-propagated through a gradient reversal layer.

[0037] In some embodiments, the emotion state detection method based on adversarial training further comprises:

[0038] The source domain differential entropy features and the target domain differential entropy features are input into at least one local domain discriminator to obtain a local domain discrimination result;

[0039] A local domain discrimination loss function is calculated according to the local domain discrimination result;

[0040] The gradient of the local domain discrimination loss function is back-propagated through a gradient reversal layer.

[0041] In some embodiments, the step of adjusting a dynamic adversarial factor according to the source domain differential entropy features and the target domain differential entropy features, and generating an emotion recognition model based on the adjusted dynamic adversarial factor, comprises:

[0042] calculate a distance between the source domain differential entropy feature and the target domain differential entropy feature according to the global domain discriminant loss function and the local domain discriminant loss function;

[0043] perform a weighted calculation on the distance to obtain a dynamic adversarial factor of an adversarial model;

[0044] input the dynamic adversarial factor into the adversarial model to train the adversarial model through the dynamic adversarial factor to obtain a model training result;

[0045] adjust the dynamic adversarial factor and the cross-entropy loss according to the model training result until the adversarial model converges;

[0046] output the adversarial model reaching a preset state to obtain an emotion recognition model.

[0047] In another aspect, some embodiments of the present application provide an emotion state detection system based on adversarial training, comprising:

[0048] a collection module configured to collect electroencephalogram signals, the electroencephalogram signals being electroencephalogram signals of different experimental paradigms and / or different subjects;

[0049] an extraction module configured to extract differential entropy features of the electroencephalogram signals;

[0050] a feature division module configured to divide the differential entropy features into source domain data and target domain data;

[0051] a feature generation module configured to generate a source domain differential entropy feature according to the source domain data and a target domain differential entropy feature according to the target domain data;

[0052] a model generation module configured to adjust a dynamic adversarial factor according to the source domain differential entropy feature and the target domain differential entropy feature, and generate an emotion recognition model based on the adjusted dynamic adversarial factor;

[0053] a result generation module configured to input the target domain differential entropy feature into the emotion recognition model to perform detection on the target domain differential entropy feature and generate a detection result of an emotion state.

[0054] According to the technical solutions, some embodiments of the present application provide an emotion state detection method and system based on adversarial training. The method comprises collecting electroencephalogram signals of different experimental paradigms and / or different subjects; extracting differential entropy features of the electroencephalogram signals; dividing the differential entropy features into source domain data and target domain data; generating source domain differential entropy features according to the source domain data, and generating target domain differential entropy features according to the target domain data; adjusting a dynamic adversarial factor according to the source domain differential entropy features and the target domain differential entropy features, and generating an emotion recognition model based on the adjusted dynamic adversarial factor; inputting the target domain differential entropy features into the emotion recognition model to perform detection on the target domain differential entropy features, and generating a detection result of the emotion state. The method can generate an emotion recognition model through adversarial training, and can obtain more common information between domains through the emotion recognition model. Therefore, the emotion recognition model can more accurately distinguish the emotion state, and improve the accuracy of emotion state detection. BRIEF DESCRIPTION OF DRAWINGS

[0055] In order to more clearly illustrate the technical solutions of some embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0056] Figure 1 A flowchart of an emotion state detection method based on adversarial training is provided for some embodiments of the present application.

[0057] Figure 2 A flowchart of performing preprocessing on electroencephalogram signals is provided for some embodiments of the present application.

[0058] Figure 3 A flowchart of performing label classification by a label classifier is provided for some embodiments of the present application.

[0059] Figure 4 A flowchart of generating a global domain discriminative loss function and a local domain discriminative loss function according to source domain differential entropy features and target domain differential entropy features is provided for some embodiments of the present application.

[0060] Figure 5 A flowchart of adjusting a dynamic adversarial factor according to source domain differential entropy features and target domain differential entropy features, and generating an emotion recognition model based on the adjusted dynamic adversarial factor is provided for some embodiments of the present application.

[0061] Figure 6 A flowchart of generating a cross-entropy loss of an optimal distance according to a distribution transformation discriminator is provided for some embodiments of the present application. DETAILED DESCRIPTION

[0062] In order to make the purposes, technical solutions and advantages of some embodiments of the present application clearer, the following will clearly and completely describe the technical solutions of some embodiments of the present application with reference to the specific embodiments and the corresponding drawings of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments of the present application.

[0063] It should be noted that the brief description of the terms in some embodiments of the present application is only for the convenience of understanding the subsequently described embodiments, and is not intended to limit the implementation of some embodiments of the present application. Unless otherwise specified, these terms should be understood according to their ordinary and general meanings.

[0064] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that can perform the functions associated with that element.

[0065] Neural cells in the brain communicate with each other through some weak neural signals, resulting in some measurable changes, which can be referred to as brain waves. In an implementation, these changes can be recorded by electroencephalogram (EEG) method. Illustratively, brain waves can be a reflection of brain activity and state, reflecting the physiological signals emitted by humans at different cognitive processing levels under different tasks or different psychological states. Therefore, the changes in the brain activity can be recorded by brain waves, and the emotional state can be reflected by electroencephalogram EEG.

[0066] In the process of reflecting the emotional state by electroencephalogram EEG, different brain wave signals are required to have the same characteristic distribution. However, due to the influence of different collection environments, different experimental paradigms, different collection devices, different subject sources and other factors, there will be a large distribution difference between different brain wave signals. Therefore, the distribution difference of the brain wave signals leads to a lack of objective diagnostic criteria for the detection results of the emotional state, resulting in inaccurate detection results of the emotional state. In addition, the distribution difference of the brain wave signals will lead to the fact that the emotional state detection method across subjects or paradigms cannot be applied to the emotional state detection method of other subjects or paradigms, and when there is a detection task of a new subject, a large amount of labeled brain wave signals are also required, which will also lead to a large workload and low detection efficiency of the emotional state.

[0067] In order to solve the problem that the detection result of the emotional state is inaccurate due to the distribution difference of the brain wave signal, some embodiments of the present application provide an emotional state detection method based on adversarial training. The method can solve the problem of distribution difference of the brain wave signal in the emotional state detection caused by multiple factors. In order to facilitate the understanding of the technical solutions in some embodiments of the present application, the following will be described in detail in combination with some specific embodiments and the accompanying drawings.

[0068] Figure 1 The flowchart of the emotional state detection method based on adversarial training provided by some embodiments of the present application is shown in FIG. 1. As shown in FIG. 1, the method can include the following steps S1-S6, and the specific content is as follows: Figure 1

[0069] Step S1: collecting a brain wave signal, wherein the brain wave signal is a brain wave signal under different experimental paradigms and / or different subjects.

[0070] In some embodiments, the brain wave signal can be collected under different experimental paradigms and / or different subjects. The specific collection method is not limited in the present application. For example, a brain wave signal can be collected by using a brain wave collection device. After step S1 is executed, the following step S2 can be executed.

[0071] Step S2: extracting differential entropy features of the brain wave signal.

[0072] In order to improve the effectiveness and accuracy of the brain wave signal, after the brain wave signal is collected, the brain wave signal can be preprocessed, and the differential entropy features of the preprocessed brain wave signal can be extracted.

[0073] Figure 2 The flowchart of the preprocessing of the brain wave signal provided by some embodiments of the present application is shown in FIG. 2. As shown in FIG. 2, the method can include the following steps S21-S22, and the specific content is as follows: Figure 2 ​As shown, the electroencephalogram signal can be preprocessed in the following manner. First, the baseline drift in the electroencephalogram signal is removed. In the electroencephalogram signal, due to the influence of baseline drift, the electroencephalogram signal will constantly drift upwards or downwards, which will interfere with the signal and make it difficult to identify the real and effective data. Therefore, in order to avoid interference with the electroencephalogram signal, the baseline drift can be removed. After the baseline drift is removed, a band-limited filter can be used to remove noise from the electroencephalogram signal after removing the baseline drift, and then a low-pass filter can be used to remove noise from the electroencephalogram signal again. That is, in the embodiments of the present application, the electroencephalogram signal is subjected to two denoising processes, one of which can help remove some high-frequency noise and low-frequency noise, making the electroencephalogram signal smoother and increasing the signal-to-noise ratio, which helps to reduce false judgments of the electroencephalogram signal and reduce the false judgment rate; the second denoising can further reduce noise and make the signal smoother, improve the signal-to-noise ratio, and thus improve the accuracy of subsequent analysis. After the second denoising is completed, the artifact signal in the electroencephalogram data after the second denoising can be removed to obtain the effective electroencephalogram signal, wherein the artifact signal is the electroencephalogram signal that does not meet the preset conditions, thereby improving the quality of the electroencephalogram signal, improving the effectiveness and accuracy of the electroencephalogram signal, and reducing interference with the electroencephalogram signal.

[0074] For example, when preprocessing the electroencephalogram signal, first, the baseline drift of the electroencephalogram signal can be removed, then a 50Hz band-limited filter can be used to remove power frequency noise, and then a low-pass filter can be used to perform 45Hz low-pass filtering to further remove noise. Finally, visual inspection can be performed to further remove artifact signals caused by body movement, blinking, eye movement, electromyography, etc., and reduce interference with the electroencephalogram signal data.

[0075] After the electroencephalogram signal preprocessing is completed, the differential entropy features of the preprocessed electroencephalogram signal can be extracted. In some embodiments, the differential entropy features of the preprocessed electroencephalogram signal can be extracted in the following manner. Specifically, the differential entropy features of the frequency band electroencephalogram signal can be obtained for each frequency band according to the leads used when collecting the electroencephalogram signal. The lead refers to the electrode used when collecting the electroencephalogram signal, and the frequency band is obtained by segmenting the frequency characteristics of the collected electroencephalogram data.

[0076] For example, after collecting the electroencephalogram signals of different subjects under different experimental paradigms and performing preprocessing, the preprocessed electroencephalogram signals can be identified, and the leads or frequency bands in the electroencephalogram signals can be pre-identified. Then, the differential entropy features can be extracted according to the identification according to different leads and frequency bands.

[0077] In some embodiments, the differential entropy is a generalization of Shannon entropy, also known as information entropy, to continuous variables. The differential entropy is used to measure the total amount of information of a continuous random signal, thereby performing statistics on the uncertainty of the probability density distribution of the continuous signal. Exemplarily, the differential entropy features of the electroencephalogram signals can be extracted according to the following formula:

[0078]

[0079] wherein a and b are the value intervals of the electroencephalogram signal input, i is the serial number of the electroencephalogram signal, x is a variable representing the electroencephalogram signal, and e is the natural base. After step S2 is executed, the following step S3 can be executed.

[0080] Step S3: dividing the differential entropy features into source domain data and target domain data.

[0081] In the field of transfer learning, the source domain and the target domain respectively refer to different data distributions, which can be understood as different data sets. In some embodiments, the source domain can refer to a data set that has been labeled, and the target domain can refer to a data set that has not been labeled. The source domain and the target domain have different distribution characteristics, so the model of the source domain cannot be directly applied to the target domain. The purpose of the emotion state detection method based on adversarial training in the embodiments of the present application is to apply the features learned by the source domain to the target domain to realize the training of the task in the target domain.

[0082] In some embodiments, the differential entropy features can be divided into source domain data and target domain data according to actual needs and in a cross-subject and / or paradigm manner. Exemplarily, the differential entropy features of the electroencephalogram signals can be divided into source domain data and target domain data according to the cross-subject or paradigm manner based on the differential entropy features of different leads and frequency bands of the electroencephalogram signals. After step S3 is executed, the following step S4 can be executed.

[0083] Step S4: generating source domain differential entropy features according to the source domain data, and generating target domain differential entropy features according to the target domain data.

[0084] After the source domain data and the target domain data are divided, the source domain differential entropy features can be generated according to the source domain data, and the target domain differential entropy features can be generated according to the target domain data. In some embodiments, the source domain differential entropy features and the target domain differential entropy features can be generated in the following manner. First, the source domain data and the target domain data are respectively input into a feature extractor, and then the feature extraction is performed on the source domain data in the feature extractor to obtain the source domain differential entropy features, and the feature extraction is performed on the target domain data in the feature extractor to obtain the target domain differential entropy features.

[0085] Exemplarily, the source domain data and the target domain data can be input into a feature extractor G f The feature extractor G f may be a multi-layer perceptron, and can obtain a source domain differential entropy feature F s by performing feature extraction on the source domain data, and obtain a target domain differential entropy feature F t by performing feature extraction on the target domain data. After step S4 is performed, the following step S5 can be performed.

[0086] Step S5: adjusting a dynamic adversarial factor according to the source domain differential entropy feature and the target domain differential entropy feature, and generating an emotion recognition model based on the adjusted dynamic adversarial factor.

[0087] Figure 3 A flowchart of performing label classification by a label classifier is provided for some embodiments of the present application, as shown in FIG. 5. In some embodiments, the source domain differential entropy feature can be input into the label classifier to obtain a label prediction result, and then a label loss function can be calculated according to the label prediction result, and the gradient of the label loss function can be back propagated. Figure 3

[0088] Exemplarily, the source domain differential entropy feature F s may be input into a label classifier G y to obtain a label prediction result, and a label loss can be calculated according to the label prediction result.

[0089]

[0090] wherein n s is the number of source domain data, and c is the serial number of electroencephalogram signal classification, and there are c classifications from 1 to c, and c is a positive integer. G f may also be understood as a feature extractor, wherein f is an abbreviation of feature.

[0091] In some embodiments, the label loss can be used to measure the emotional state, for example, the label loss can be the error of two states or multiple states. Calculating the label loss can be used to distinguish the emotional state, and thus improve the accuracy of classification.

[0092] ​In some embodiments, after generating the source domain differential entropy features and the target domain differential entropy features, these features can be input into a global domain discriminator to obtain a global domain discrimination result. A global domain discrimination loss function is then calculated based on the global domain discrimination result, and its gradient is backpropagated through a gradient reversal layer. Simultaneously, the source domain differential entropy features and the target domain differential entropy features can also be input into at least one local domain discriminator to obtain a local domain discrimination result. A local domain discrimination loss function is then calculated based on the local domain discrimination result, and its gradient is finally backpropagated through a gradient reversal layer.

[0093] For example, Figure 4 This application provides flowcharts illustrating the generation of global and local discriminant loss functions based on source and target domain differential entropy features, as shown in some embodiments. Figure 4 As shown, the source domain differential entropy feature F can be... s and the differential entropy feature F of the target domain t Input to global discriminator G d The global domain discrimination result is obtained, and the global domain discrimination loss function is calculated based on the global domain discrimination result. Then, the gradient of the global domain discrimination loss function can be backpropagated through a gradient reversal layer (GRL). For example, the gradient of the global domain discrimination loss function can be backpropagated through a gradient reversal layer (GRL). Simultaneously, the source domain differential entropy feature F... s and the differential entropy feature F of the target domain t The input is fed into c local discriminators, such as In the expression, where d and c are both positive integers, c local domain discrimination results are obtained and the loss is calculated. The gradient is then backpropagated through the gradient inversion layer (GRL).

[0094] In some embodiments, the global domain discriminator can be a multilayer perceptron, and the loss function of the global domain discriminator can be:

[0095]

[0096] Here, 's' is an abbreviation for 'source,' 'g' is an abbreviation for 'global,' and 't' is an abbreviation for 'target,' representing the target domain. Local Domain Discriminator It can also be a multilayer perceptron, and in some embodiments, a local domain discriminator. The loss function can be:

[0097]

[0098] Global discriminators and local discriminators have essentially the same structure, with deep features as input. The difference lies in that the deep features input to the local discriminator need to be multiplied by the probability of a class label. In other words, the input to the local discriminator is the product of the deep features and the corresponding class label, focusing on locally distributed features.

[0099] It should be noted that the classification objectives of domain discriminators such as the global domain discriminator and the local domain discriminator are opposite to those of the label classifier. Therefore, to ensure the consistency of the overall objective, a gradient reversal layer can be used to backpropagate the gradient of the label loss function. In some embodiments, to achieve the effect of gradient reversal, the gradient can be multiplied by -λ during backpropagation. Here, the negative sign indicates that a reverse gradient is propagated, and λ is a parameter that can be arbitrarily adjusted.

[0100] Figure 5 This application provides flowcharts illustrating the process of adjusting dynamic adversarial factors based on source domain differential entropy features and target domain differential entropy features, and generating an emotion recognition model based on the adjusted dynamic adversarial factors, as shown in some embodiments. Figure 5 As shown, after generating the source domain differential entropy features and the target domain differential entropy features, the dynamic adversarial factor can be adjusted in the following way, and an emotion recognition model can be generated based on the adjusted dynamic adversarial factor. First, the margin between the source domain differential entropy features and the target domain differential entropy features is calculated according to the global domain discriminant loss function and the local domain discriminant loss function. Then, a weighted calculation is performed on the margin to obtain the dynamic adversarial factor of the adversarial model. The dynamic adversarial factor is then input into the adversarial model to train the adversarial model and obtain the model training results. The dynamic adversarial factor and cross-entropy loss are adjusted according to the model training results until the adversarial model converges. Finally, the adversarial model that has reached the preset state is output, thus obtaining the emotion recognition model.

[0101] In some embodiments, the role of adversarial models is to enhance their learning performance through adversarial learning. Adversarial learning involves inputting data from different data domains into the adversarial model and subjecting them to mutual competition, making it impossible for the model to explicitly distinguish which domain the data originates from. The purpose of adversarial learning is that as the model gradually loses the ability to clearly distinguish the specific source of the data, it learns to comprehensively consider data from different domains, ultimately extracting features independent of the original data domain. Thus, during the learning process, for data with fewer features, the adversarial model can learn common features between different domains, both in data-rich and data-scarce domains, thereby enhancing its performance in a particular domain. For example, common features between the source and target domains can be learned, providing an analytical foundation for the analysis of the target domain.

[0102] To improve the accuracy of emotion state detection results, in some embodiments, the optimal distance can be calculated based on the source domain differential entropy features and the target domain differential entropy features. The optimal distance is the minimum distance between the source domain differential entropy features and the target domain differential entropy features. The optimal distance is a measure of the difference in data domain distribution. Its characteristic is that the calculation of the optimal distance yields a transformation matrix of data from the source domain to the target domain. Therefore, a discriminator can be used to determine whether a transformation matrix is ​​obtained from the source domain to the target domain transformation or from the target domain to the source domain transformation. In this way, by calculating the optimal distance, the distribution transformation information between the source and target domain EEG signals can be learned. During training, the adversarial model learns how the distribution of source domain data fits to the distribution of target domain data through the optimal distance transformation, and it can also learn the inverse distribution transformation from the target domain data distribution to the source domain data distribution. After the optimal distance is calculated, forward and reverse distribution transition vectors are generated based on the optimal distance. These vectors are then input into the distribution transformation discriminator to obtain the domain prediction result. Next, the cross-entropy loss of the optimal distance is calculated based on the domain prediction result, and finally, the gradient of the cross-entropy loss is backpropagated.

[0103] For example, it can first be based on the source domain differential entropy feature F s and the differential entropy feature F of the target domain t The optimal distance can be calculated, for example, using the following formula:

[0104]

[0105] The parameters in the above formula are explained in the foregoing embodiments and will not be repeated here. Based on this calculation formula, the positive distribution transfer vector T can be obtained. s and the reverse distribution transition vector T t . Figure 6 This is a schematic diagram illustrating the process of generating the optimal distance cross-entropy loss based on the distribution transformation discriminator according to some embodiments of this application, such as... Figure 6 As shown, the distribution transfer vector T can be... s and the reverse distribution transition vector T t Input to the distribution transformation discriminator G t In the process of obtaining the domain prediction result, the cross-entropy loss can then be calculated based on this domain prediction result. In some embodiments, the cross-entropy loss can be calculated using the following formula:

[0106]

[0107] To improve the accuracy of the emotion recognition model, in some embodiments, after the cross-entropy loss is calculated, the gradient of the cross-entropy loss can be back propagated. The back propagation algorithm passes the error signal from the output layer to the input layer, calculates the error gradient of each neuron in the brain wave signal, and uses the error gradient to adjust the related parameters in the emotion recognition model. In this way, the emotion recognition model can enhance the learning and optimization ability, improve the accuracy of the emotion recognition model, and also improve the generalization ability of the model.

[0108] In some embodiments, the global domain discriminative loss function L g and the local domain discriminative loss function L l The interval distance between the source domain differential entropy feature and the target domain differential entropy feature is calculated, and then the interval distance is weighted to obtain a dynamic adversarial factor ω of the adversarial model, and the dynamic adversarial factor ω is input into the adversarial model, so that the loss of each training is dynamically adjusted by the dynamic adversarial factor ω, as shown by the dashed part in formula (5). Figure 4 In some embodiments, the dynamic adversarial factor ω can be calculated by the following formula, for example:

[0109]

[0110] The adversarial model is trained by the dynamic adversarial factor to obtain a model training result, and then the dynamic adversarial factor and the cross-entropy loss are adjusted according to the model training result until the adversarial model converges, and finally the adversarial model that reaches a preset state is output to obtain the emotion recognition model.

[0111] It should be noted that the dynamic adversarial factor initially has an initial value, and in some embodiments of the present application, the dynamic adversarial factor is trained to obtain a model training result, and then the dynamic adversarial factor and the cross-entropy loss are adjusted according to the model training result, and the trained emotion recognition model is obtained through multiple adversarial training until the adversarial model converges. The trained emotion recognition model is used to recognize the emotional state. When the adversarial model starts training, the adversarial model treats the global distribution state and the local distribution state of the input data equally, and then the adversarial model obtains an adjusted dynamic adversarial factor, i.e. a new dynamic adversarial factor, which is used for the next training. Such iteration is performed until the adversarial model completely learns the weight of the global distribution or the local distribution. After step S5 is completed, step S6 can be performed.

[0112] Step S6: inputting the target domain differential entropy feature into the emotion recognition model to perform detection on the target domain differential entropy feature and generate a detection result of the emotional state.

[0113] After the emotion recognition model is generated, the target domain differential entropy features can be input into the emotion recognition model. Since the emotion recognition model is obtained through the adversarial model, the emotion recognition model can generate the emotion recognition model through adversarial training, and more common information between domains can be obtained through the emotion recognition model. Therefore, the emotion recognition model can more accurately distinguish the emotional state, that is, the emotional state can be more accurately detected through the brain waves, thereby improving the accuracy of the emotional state detection. For example, the emotional state detection can assist in predicting a certain state, such as predicting a depressive state, predicting the risk of a depressive state, and the like. At the same time, in the process of detecting the emotional state, the brain wave signals do not need to be relabeled, thereby reducing the workload and improving the detection efficiency of the emotional state.

[0114] It should be noted that in the process of detecting the emotional state through the emotion recognition model, the information input into the emotion recognition model can be related information of the source domain or related information of the target domain. Therefore, after the information is input into the emotion recognition model for learning, the result learned by the emotion recognition model can have the following several cases: it can learn the target domain information through the source domain information, it can learn the source domain information through the target domain information, or it can learn some common information through the source domain information or the target domain information. That is, the result learned by the emotion recognition model is not simply the information of a certain domain, but the common information between the domains.

[0115] From the above technical solutions, it can be seen that the above embodiment provides an emotion state detection method based on adversarial training, which includes collecting brain wave signals, the brain wave signals being brain wave signals of different experimental paradigms and / or different subjects; extracting differential entropy features of the brain wave signals; dividing the differential entropy features into source domain data and target domain data; generating source domain differential entropy features according to the source domain data, and generating target domain differential entropy features according to the target domain data; adjusting a dynamic adversarial factor according to the source domain differential entropy features and the target domain differential entropy features, and generating an emotion recognition model based on the adjusted dynamic adversarial factor; inputting the target domain differential entropy features into the emotion recognition model to perform detection on the target domain differential entropy features, and generating a detection result of the emotional state. The method can generate an emotion recognition model through adversarial training, and more common information between domains can be obtained through the emotion recognition model. Therefore, the emotion recognition model can more accurately distinguish the emotional state, thereby improving the accuracy of the emotional state detection.

[0116] Part of the embodiments of the present application also provide an emotion state detection system based on adversarial training, which includes:

[0117] The collection module collects brain wave signals, and the brain wave signals are brain wave signals of different experimental paradigms and / or different subjects.

[0118] extracting a differential entropy feature of the brain wave signal;

[0119] dividing the differential entropy feature into source domain data and target domain data;

[0120] generating source domain differential entropy features according to the source domain data, and generating target domain differential entropy features according to the target domain data;

[0121] adjusting a dynamic adversarial factor according to the source domain differential entropy features and the target domain differential entropy features, and generating an emotion recognition model based on the adjusted dynamic adversarial factor;

[0122] inputting the target domain differential entropy features into the emotion recognition model to perform detection on the target domain differential entropy features, and generating a detection result of the emotion state.

[0123] It can be known from the above technical solutions that the emotion state detection system based on adversarial training provided by the above embodiment can generate an emotion recognition model through adversarial training, and more common information between domains can be obtained through the emotion recognition model, so that the emotion recognition model can more accurately distinguish the emotion state, and the accuracy of emotion state detection is improved.

[0124] The same and similar parts among the various embodiments in the specification can be referred to each other, and will not be described here.

[0125] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. This application is intended to cover any variations, uses or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice in the art to which the application pertains.

[0126] It should be understood that the application is not limited to the precise construction that has been described above and illustrated in the accompanying drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the application is limited only by the claims that follow.

Claims

1. An emotion state detection method based on adversarial training, characterized in that, The method comprises the following steps: Collecting an electroencephalogram signal, wherein the electroencephalogram signal is an electroencephalogram signal of different experimental paradigms and / or different subjects; Extracting differential entropy features of the electroencephalogram signal; Dividing the differential entropy features into source domain data and target domain data; Generating source domain differential entropy features according to the source domain data, and generating target domain differential entropy features according to the target domain data; Adjusting a dynamic adversarial factor according to the source domain differential entropy features and the target domain differential entropy features, and generating an emotion recognition model based on the adjusted dynamic adversarial factor; Calculating an optimal distance between the source domain differential entropy features and the target domain differential entropy features, wherein the optimal distance is the minimum distance between the source domain differential entropy features and the target domain differential entropy features; Generating a forward distribution transfer vector and a reverse distribution transfer vector based on the optimal distance; Inputting the forward distribution transfer vector and the reverse distribution transfer vector into a distribution transformation discriminator to obtain a domain prediction result; Calculating a cross-entropy loss of the optimal distance according to the domain prediction result; Back-propagating the gradient of the cross-entropy loss to the emotion recognition model; Inputting the target domain differential entropy features into the emotion recognition model to perform detection on the target domain differential entropy features, and generating a detection result of an emotional state. 2.The emotion state detection method based on adversarial training according to claim 1, wherein, The step of extracting differential entropy features of the electroencephalogram signal comprises the following steps: According to the collected electroencephalogram signal, the differential entropy features of the electroencephalogram signal in each frequency band of each lead are obtained. 3.The emotion state detection method based on adversarial training according to claim 2, characterized in that, The step of dividing the differential entropy features into source domain data and target domain data comprises the following steps: According to the differential entropy features of the electroencephalogram signal in each frequency band of each lead, the differential entropy features of the electroencephalogram signal are divided into source domain data and target domain data. The method further comprises the following steps: 4.The emotion state detection method based on adversarial training according to claim 1, wherein, Deleting baseline drift in the electroencephalogram signal; Using a band-limited filter to perform noise removal on the electroencephalogram signal after deleting the baseline drift; Using a low-pass filter to perform noise removal on the electroencephalogram signal after noise removal; Deleting artifact signals in the electroencephalogram signal after the second noise removal, to obtain effective electroencephalogram signals, wherein the artifact signals are electroencephalogram signals that do not meet the preset conditions. The step of generating source domain differential entropy features according to the source domain data, and generating target domain differential entropy features according to the target domain data comprises the following steps: 5.The emotion state detection method based on adversarial training according to claim 1, wherein, Inputting the source domain data and the target domain data into a feature extractor, respectively; Performing feature extraction on the source domain data in the feature extractor to obtain source domain differential entropy features; Performing feature extraction on the target domain data in the feature extractor to obtain target domain differential entropy features. The method further comprises the following steps:

6. The emotion state detection method based on adversarial training of claim 1, wherein, Inputting the source domain differential entropy features and the target domain differential entropy features into a global domain discriminator to obtain a global domain discrimination result; Calculating a global domain discrimination loss function according to the global domain discrimination result; Back-propagating the gradient of the global domain discrimination loss function through a gradient reversal layer. The method further comprises the following steps:

7. The emotion state detection method based on adversarial training of claim 6, wherein, Inputting the source domain differential entropy features and the target domain differential entropy features into at least one local domain discriminator to obtain a local domain discrimination result; ​ Calculate a local domain discrimination loss function according to the local domain discrimination result; Backpropagate the gradient of the local domain discrimination loss function through a gradient reversal layer.

8. The emotion state detection method based on adversarial training according to claim 7, characterized in that, Adjust a dynamic adversarial factor according to the source domain differential entropy feature and the target domain differential entropy feature, and generate an emotion recognition model based on the adjusted dynamic adversarial factor, including: Calculate a gap distance between the source domain differential entropy feature and the target domain differential entropy feature according to the global domain discrimination loss function and the local domain discrimination loss function; Perform weighted calculation on the gap distance to obtain a dynamic adversarial factor of an adversarial model; Input the dynamic adversarial factor into the adversarial model to train the adversarial model through the dynamic adversarial factor to obtain a model training result; Adjust the dynamic adversarial factor and the cross-entropy loss according to the model training result until the adversarial model converges; Output the adversarial model that reaches a preset state to obtain an emotion recognition model.

9. An emotion state detection system based on adversarial training, characterized by, Comprise: The acquisition module acquires the brain wave signal, and the brain wave signal is the brain wave signal of different experimental paradigms and / or different subjects; The extraction module extracts the differential entropy feature of the brain wave signal; The feature division module divides the differential entropy feature into source domain data and target domain data; The feature generation module generates a source domain differential entropy feature according to the source domain data, and generates a target domain differential entropy feature according to the target domain data; The model generation module adjusts a dynamic adversarial factor according to the source domain differential entropy feature and the target domain differential entropy feature, and generates an emotion recognition model based on the adjusted dynamic adversarial factor; It comprises: calculating an optimal distance between the source domain differential entropy feature and the target domain differential entropy feature, the optimal distance being the minimum distance between the source domain differential entropy feature and the target domain differential entropy feature; Generate a forward distribution transfer vector and an inverse distribution transfer vector based on the optimal distance; Input the forward distribution transfer vector and the inverse distribution transfer vector into a distribution transformation discriminator to obtain a domain prediction result; Calculate the cross-entropy loss of the optimal distance according to the domain prediction result; Backpropagate the gradient of the cross-entropy loss to the emotion recognition model; The result generation module inputs the target domain differential entropy feature into the emotion recognition model to perform detection on the target domain differential entropy feature and generate a detection result of the emotional state.

Images