Method and device for amplifying physiological data, emotion recognition system and terminal

Through data amplification and quality evaluation technology, the EEG signal is amplified and screened, which solves the problem of inconvenient acquisition of EEG signals, improves the accuracy of emotion recognition, and generates high-quality physiological data to enhance emotional prediction ability.

CN119046715APending Publication Date: 2024-11-29ROBERT BOSCH GMBH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310615240.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-29
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

In the prior art, when acquiring and using electroencephalography (EEG) signals, the signal acquisition is not convenient enough, resulting in limited accuracy of emotion recognition and it is difficult to effectively amplify physiological data to improve the accuracy of emotional judgment.

Method used

The preprocessed physiological data is amplified by using a data amplification model, and combined with visualization and clustering models for quality evaluation and screening, to generate physiological data for emotion recognition, and through the generation of adversarial network (GAN) and K-means algorithms, etc. Technology, remove data that does not match distribution deviations and clustering labels to improve data quality.

Benefits of technology

Through amplification and screening technology, more and high-quality physiological data are generated, which improves the accuracy of emotion recognition, enhances the ability to predict user emotions, and trains a higher-precision emotion prediction model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119046715A_ABST
    Figure CN119046715A_ABST
Patent Text Reader

Abstract

The invention provides a method for amplifying physiological data. The method comprises the following steps: receiving preprocessed first physiological data; amplifying the received first physiological data by a data amplification model to obtain second physiological data, wherein the second physiological data does not include the first physiological data; and performing first quality evaluation and screening and / or second quality evaluation and screening on the first physiological data and the second physiological data to generate physiological data for emotion recognition. The invention further provides a system for amplifying the physiological data, a corresponding device and a terminal.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to emotion recognition technology, and more specifically, to technology for augmenting physiological data. Background Art

[0002] Quantifying biological dynamic signals is of great value in detecting and judging personal emotions. To this end, technologies based on facial expression recognition, voice recognition, and physiological signal recognition have been developed and applied to varying degrees. However, facial expressions and voice can be disguised to hide real emotions. In contrast, physiological signals can more accurately reflect people's emotional state. Electroencephalogram (EEG) signals, as physiological signals, are often used to characterize people's emotional changes.

[0003] Generally speaking, the accuracy of emotion recognition is highly dependent on the number of EEG signals, but EEG signals are not as easy to obtain as visual or speech signals. Currently, EEG signals are mostly obtained through head-mounted devices, which can basically cover the entire cerebral cortex to obtain multi-channel data.

[0004] Considering the widespread application of EEG signals in emotion judgment and other aspects, it is necessary to further improve the acquisition of EEG signals. Summary of the invention

[0005] One aspect of the present application is to provide a method for augmenting physiological data, the method comprising: receiving preprocessed first physiological data; amplifying the received first physiological data by a data augmentation model to obtain second physiological data, the second physiological data not including the first physiological data; performing a first quality assessment and screening on the first physiological data and the second physiological data, and / or performing a second quality assessment and screening on the first physiological data and the second physiological data to generate physiological data for emotion recognition. Wherein, based on a visualization model, the first physiological data and the second physiological data are subjected to a first quality assessment and screening, the first quality assessment and screening comprising: assessing the distribution of the first physiological data and the second physiological data, and removing physiological data with a distribution deviation greater than a threshold; wherein, based on a clustering model, the first physiological data and the second physiological data are subjected to a second quality assessment and screening, the second quality assessment and screening comprising: assessing whether the clustering label of the first physiological data and the clustering label of the second physiological data are consistent, and removing data in the second physiological data whose clustering label does not conform to the clustering label of the first physiological data.

[0006] According to the method for augmenting physiological data of the application, in some examples, the visualization model may include a t-SNE algorithm model.

[0007] According to the method for augmenting physiological data of the application, in some examples, the clustering model includes a K-means algorithm.

[0008] According to the method for amplifying physiological data of the application, in some examples, the method also includes: generating characteristic data from the second physiological data that has undergone the first quality evaluation and screening and / or the physiological data that has undergone the second quality evaluation and screening; determining emotions based on the generated characteristic data; and based on the comparison between the determined emotions and the actual emotions, screening out the physiological data whose determined emotions are consistent with the actual emotions from the physiological data that has undergone the first quality evaluation and screening and / or the physiological data that has undergone the second quality evaluation and screening as the physiological data for emotion recognition.

[0009] According to another aspect of the present application, a system for amplifying physiological data is also provided, the system comprising: a data amplification module, for amplifying the received preprocessed first physiological data to obtain second physiological data, the second physiological data does not include the first physiological data; a screening module, for performing a first quality assessment and screening on the first physiological data and the second physiological data, and / or performing a second quality assessment and screening on the first physiological data and the second physiological data to generate physiological data for emotion recognition. The screening module may further include a visualization module, for performing a first quality assessment and screening on the first physiological data and the second physiological data, the first quality assessment and screening including assessing the distribution of the first physiological data and the second physiological data, and removing physiological data with a distribution deviation greater than a threshold; and a clustering module, for performing a second quality assessment and screening on the first physiological data and the second physiological data, the second quality assessment and screening including: assessing whether the clustering label of the first physiological data and the clustering label of the second physiological data are consistent, and removing data in the second physiological data whose clustering label does not conform to the clustering label of the first physiological data.

[0010] According to the system for augmenting physiological data of the application, in some examples, the system also includes a re-screening module for generating characteristic data from the physiological data that has undergone a first quality assessment and screening and / or the physiological data that has undergone a second quality assessment and screening, determining emotions based on the generated characteristic data, and, based on a comparison between the determined emotions and the actual emotions, screening out physiological data whose determined emotions are consistent with the actual emotions from the physiological data that has undergone a first quality assessment and screening and / or the physiological data that has undergone a second quality assessment and screening as physiological data for emotion recognition.

[0011] According to another aspect of the application, a device for augmenting physiological data is also provided, which includes a processor, wherein the processor is used to execute program instructions stored in a memory to implement the steps of the method according to any one of claims 1 to 4.

[0012] An emotion recognition system is also provided, the emotion recognition system comprising a data acquisition device and a pre-trained classifier. The data acquisition device is used to acquire user physiological data. The pre-trained classifier is used to classify user emotions based on the acquired user physiological data. The training data for the classifier used for training is physiological data for emotion recognition obtained by the device for amplifying physiological data as described above, or physiological data for emotion recognition obtained by the system for amplifying physiological data as described above, or physiological data for emotion recognition obtained by executing the method for amplifying physiological data as described above.

[0013] An emotion recognition system is also provided, comprising: a collection device for collecting physiological signals of a user as first physiological data; a physiological data amplification device for amplifying the first physiological data to obtain second physiological data, and performing a first quality assessment and screening, and / or a second quality assessment and screening on the second physiological data to generate second physiological data for emotion recognition, wherein the second physiological data does not include the first physiological data; and a classifier for performing emotion recognition based on the first physiological data and the generated second physiological data for emotion recognition. The physiological data amplification device is further used to: perform a first quality assessment and screening on the second physiological data based on a visualization model, wherein the first quality assessment and screening includes assessing the distribution of the second physiological data, and removing the second physiological data whose distribution deviation is greater than a threshold compared to the first physiological data; and / or, perform a second quality assessment and screening on the second physiological data based on a clustering model, wherein the second quality assessment and screening includes assessing whether the clustering label of the second physiological data is consistent with the clustering label of the first physiological data, and removing data in the second physiological data whose clustering label is inconsistent with the clustering label of the first physiological data.

[0014] The emotion recognition system, for example, the classifier is a pre-trained classifier using physiological data for emotion recognition obtained by the device for amplifying physiological data as described above, or physiological data for emotion recognition obtained by the system for amplifying physiological data as described above, or physiological data for emotion recognition obtained by executing the method for amplifying physiological data as described above.

[0015] An electronic device is also provided, comprising the system for amplifying physiological data as described above, or comprising the device as described above, or comprising any one of the emotion recognition systems as described above.

[0016] In all the above examples, the physiological data may be data of electroencephalogram signals, namely, EEG data. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The present application will be more fully understood by referring to the following detailed description of specific embodiments in conjunction with the accompanying drawings, in which the same reference numerals in the drawings refer to the same elements in the drawings.

[0018] Figure 1 is a flow chart of a method for augmenting physiological data according to some examples of the present application;

[0019] Figure 2 is a flow chart of further evaluating physiological data that has undergone a first quality assessment and a second quality assessment according to some examples of the present application;

[0020] Figure 3 is a schematic diagram of a system for amplifying physiological data according to some examples of the present application;

[0021] Figure 4 is a schematic diagram of a structure of a device for amplifying physiological data according to some examples of the present application;

[0022] Figure 5 is a schematic diagram of the structure of an emotion recognition system according to some examples of the present application;

[0023] Figure 6 It is a schematic diagram of the structure of the emotion recognition system according to some other examples of the present application. DETAILED DESCRIPTION

[0024] In order to help those skilled in the art to accurately understand the subject matter for which protection is sought in the present application, the specific implementation of the present application will be described in detail below in conjunction with the accompanying drawings.

[0025] Figure 1 is a flow chart of a method for augmenting physiological data according to some examples of the present application. The method is performed by a system, device or apparatus for predicting user emotions based on physiological data. Figure 1 The method shown in the figure can obtain more physiological data to train the prediction model of these systems, devices or equipment. Figure 1 It is also possible to predict the user's emotions directly using the method shown. In addition, in all examples of this application, EEG data is used as physiological data for illustration, but this does not mean that each solution according to the examples of this application is only suitable for processing EEG data.

[0026] In step S100, the preprocessed first physiological data is received. As an example, the user's EEG signal is collected by an EEG signal collection device such as a head-mounted collection device or an in-ear collection device. The collected original EEG signal is sent to a filter for filtering to preprocess the original EEG signal, thereby obtaining the first physiological data. For example, each group of sample data in the first physiological data is provided with a label, which indicates which emotion the emotion of the group of sample data belongs to. In the examples of the present application, emotions are divided into two types, happy and unhappy, so the labels include a label indicating happiness and a label indicating unhappiness.

[0027] In step S102, the received first physiological data is amplified by the data augmentation model to obtain second physiological data. By way of example and not limitation, the data augmentation model adopts a generative adversarial network (GAN) model. As is known, the GAN model includes a generator and a discriminator. With the first physiological data as input, the generator generates a sample close to the first physiological data based on the input. The discriminator receives the physiological data generated by the generator and the real first physiological data, and attempts to distinguish the generated physiological data from the received data. The generator and the discriminator are constantly confronted, so that the generator continuously adjusts parameters and trains itself to generate sample data close to the real physiological data, but the sample data is synthetic physiological data. In this example, the data generated by GAN and output outside the model is called the second physiological data, to distinguish it from the first physiological data as the real physiological data, that is, the second physiological data does not include the first physiological data. In addition, according to the example of the present application, the second physiological data may also be labeled.

[0028] In step S104, the first physiological data and the second physiological data are subjected to a first quality assessment and screening, and / or a second quality assessment and screening, to generate physiological data for emotion recognition. Specifically, step S104 includes the following possible processing: performing a first quality assessment and screening on the first physiological data and the second physiological data, and performing a second quality assessment and screening on the first physiological data and the second physiological data, so as to generate physiological data for emotion recognition; or, performing only a first quality assessment and screening on the first physiological data and the second physiological data, so as to generate physiological data for emotion recognition; or, performing only a second quality assessment and screening on the first physiological data and the second physiological data, so as to generate physiological data for emotion recognition.

[0029] In some examples, based on the visualization model, the first physiological data and the second physiological data are subjected to a first quality assessment and screening. The first quality assessment and screening includes assessing the distribution of the first physiological data and the second physiological data, and removing physiological data whose distribution deviation is greater than a threshold. Specifically, the preprocessed first physiological data and the second physiological data generated by the data amplification model are sent to the visualization model, which generates a visualization pattern. As is known, as a data analysis tool, the visualization model mainly uses computer graphics, image processing, signal processing and other methods to express the internal structure, information and knowledge of the data, by converting high-dimensional data into data with a dimension not greater than 3, and then drawing it into an image for display, so that the relationship between the data can be quickly analyzed visually, and outlier detection can be performed. After the visualization analysis, if a certain data distribution is presented as an outlier outside the clustered data, then for this data, the data distribution deviation is greater than the distribution threshold. In other words, the distribution threshold here refers to the distance between the distribution point of the data and the data distribution cluster for the analyst who analyzes the visualization, so that the data can be considered as an outlier, and the distance is considered to be the distribution threshold. By presenting the first physiological data and the second physiological data through the visualization model, outlier data of poor quality can be easily removed. The distribution threshold is not limited to a specific value, but depends on the analyst and the object of analysis in actual application.

[0030] By way of example and not limitation, in the examples of the present application, the visualization model is a t-SNE model. For the t-SNE model, both the first physiological data and the second physiological data are its inputs, and they do not differ because one is real data and the other is generated data. The t-SNE model intuitively represents the input physiological data as a clustering relationship of EEG data after visual analysis based on its adaptive kernel algorithm. As mentioned above, the second physiological data is generated based on the first physiological data and is very similar to it. Therefore, in theory, the second physiological data should be clustered with the first physiological data in the visualization analysis of t-SNE. If outliers appear, the data should be of poor quality in the second physiological data and can be removed. Data quality in this application refers to the degree of similarity to real data. The more similar it is to real data, the higher the quality.

[0031] In some other examples, based on the clustering model, the first physiological data and the second physiological data are subjected to a second quality assessment and screening. The second quality assessment and screening includes: assessing whether the clustering labels of the first physiological data and the clustering labels of the second physiological data are consistent, and removing data whose clustering labels in the second physiological data do not match the clustering labels of the first physiological data. Specifically, the preprocessed first physiological data and the second physiological data generated by the data augmentation model are sent to the clustering model, which clusters these data to obtain multiple cluster families.

[0032] In the examples of the present application, the K-means clustering model is used. K-means clusters the input sample data according to its clustering algorithm to generate different cluster families, each cluster family has its own label. In the description process of the present application, the label of each cluster family is called a cluster label to avoid confusion with the emotion label set to the first physiological data and the second physiological data discussed above. According to some embodiments of the present application, before the clustering model performs clustering processing on the first physiological data and the second physiological data, the label representing the emotion set to the physiological data can be removed. For example, in the example where a label is set for each group of sample data in the first physiological data in step S100, the label can be removed before being processed by the K-means clustering model. As mentioned above, the second physiological data is generated based on the first physiological data and is very similar to it. Therefore, in theory, the clustering analysis of the second physiological data should be consistent with the cluster label of the first physiological data. If the cluster label is inconsistent, it means that the sample data should be data of poor quality in the second physiological data and can be removed.

[0033] The received EEG data is amplified by a data augmentation model such as GAN, so that a very close second physiological data is obtained based on the first physiological data as the real data, increasing the number of samples of available EEG data. In addition, the first physiological data and the second physiological data are subjected to a first quality assessment and screening and / or a second quality assessment and screening, for example, to remove those discrete points whose data distribution deviation is greater than the distribution threshold after visual analysis and / or to remove those data that are not distributed in the cluster cluster after a clustering model analysis such as Kmeans. So far, the amplified data with high data quality is obtained by removing low-quality physiological data.

[0034] In some other examples of the present application, further evaluation is also performed on the data that has been evaluated in step S104. Figure 2 This is a flowchart of further evaluating physiological data that has undergone the first quality evaluation and the second quality evaluation according to some examples of the present application. It should be noted that: Figure 2 The explanation is given by taking physiological data that has undergone both the first quality evaluation and screening and the second quality evaluation and screening as an example, but the method is also applicable to physiological data that has only undergone the first quality evaluation and screening or physiological data that has only undergone the second quality evaluation and screening. In this way, the data received in the following step S200 is only the physiological data that has undergone the first quality evaluation and screening or only the data that has undergone the second quality evaluation and screening.

[0035] like Figure 2As shown, in step S200, physiological data after a first quality assessment and screening and physiological data after a second quality assessment and screening are received. Wherein, the physiological data include first physiological data and second physiological data amplified by the first physiological data. In this example, the physiological data after the second quality assessment and screening is, for example, physiological data after clustering assessment. In step S202, feature data is generated according to the received physiological data after visualization assessment and clustering assessment. As an example, generating feature data according to the evaluated first physiological data and second physiological data includes performing frequency domain transformation on them, and performing feature calculation on the transformed data to obtain feature data. In step S204, the classifier determines the emotion based on the generated feature data. By way of example and not limitation, the classifier in each example of the present application is a support vector machine (SVM). The SVM outputs the determined user emotion. In step S206, based on the comparison between the determined emotion and the actual emotion, the physiological data whose determined emotion is consistent with the actual emotion is screened out from the physiological data after the first quality assessment and screening and / or the second quality assessment and screening as the physiological data for emotion recognition.

[0036] Each sample in the first physiological data and the second physiological data may have an emotion label, and the emotion represented by the emotion label is used as the actual emotion of the user corresponding to each sample to be compared with the determined emotion. For example, the first physiological data received in step S100 is the physiological data of which the user's emotion is already known, and the first physiological data may be marked with an emotion label; the second physiological data amplified by the first physiological data also has an emotion label. In some cases, such annotated emotion labels may still exist after the quality assessment and screening of the physiological data, such as the assessment and screening of the visualization model mentioned above. In other cases, such annotated emotion labels are removed before quality assessment and screening, such as the assessment and screening of the clustering model mentioned above, but in this case, the filtered physiological data can be marked with labels representing the corresponding actual emotions according to their previous emotion labels.

[0037] In some examples, the first physiological data is labeled with an emotion tag for indicating the real emotion, and in some cases, the second physiological data generated from the first physiological data is also labeled with an emotion tag. However, in subsequent quality assessment and other processing, the emotion tag will be retained or removed as needed.

[0038] The process of obtaining characteristic data in step S202 can be implemented according to the following specific process. First, the evaluated first physiological data and second physiological data are converted to the frequency domain by wavelet transform, and the transformed frequency domain signal is output as 5 frequency bands, namely, delta (δ) wave with a frequency in the range of 1 Hz to 4 Hz, theta (θ) wave with a frequency in the range of 4 Hz to 7 Hz, alpha (α) wave with a frequency in the range of 7 Hz to 13 Hz, beta (β) wave with a frequency in the range of 13 Hz to 30 Hz, and gamma (γ) wave with a frequency in the range of 30 Hz to 45 Hz. Subsequently, the power spectral density (PSD) of the signal transformed into the frequency domain is calculated to obtain the characteristic data.

[0039] For example, in an example of using an in-ear EEG signal acquisition device to collect EEG signals, the EEG signal frequency is 128 Hz, the sampling frequency is 256, and each processing length is one second of EEG data. The EEG data input to the FFT is X 256 , the FFT process is shown in formula 1.1:

[0040] Y 256 =FFT(X 256 , 256) (1.1)

[0041] Among them, X 256 The superscript 256 in the figure represents the EEG data within one second, which corresponds to 256 sampling values. 256 Indicates that FFT returns the same 256 The corresponding 256 values ​​in the frequency domain.

[0042] After the FFT transformation as shown in formula 1.1, the frequency domain output Y of the EEG signal is 256 It will be divided into five frequency bands of δ, θ, α, β, and γ as described above, respectively indicated as Y 256 Value Y in the range of 1Hz to 4Hz δ , Y 256 Value Y in the range of 4 Hz to 7 Hz θ , Y 256 Value Y in the range of 7Hz to 13Hz α , Y 256 Value Y in the range of 13Hz to 30Hz β , and Y 256 Y is between 30Hz and 45Hz γ .

[0043] For the signal in each frequency band, its power spectrum density can be calculated according to equation (1.2) to obtain the characteristic data of the frequency band:

[0044]

[0045] Among them, the subscript i (1≤i≤5, i is an integer) represents δ (delta), θ (theta), α (alpha), β (beta), and γ (gamma) as described above; j represents the number of data sampled in the sampling period.

[0046] In the above-mentioned example in which an earbud-type EEG signal acquisition device is used as a physiological signal sampling device and the sampling frequency is 256Hz, the two earphones used for sampling have the same sampling frequency, and both sample 256 physiological data in one second. The 256 EEG data with a length of one second are divided into values ​​of 5 frequency bands, namely δ, θ, α, β, and γ, so as to calculate characteristic data for each frequency band. For the user's physiological signals collected by each earphone, 5 characteristic data are generated for it in this example, and 10 characteristic data are generated for the left and right earphones, which are regarded as a sample (characteristic data of the left ear and the right ear).

[0047] The classifier involved in step S204 is a support vector machine (SVM). Specifically, the SVM outputs the determined user emotion. The SVM can achieve the best classification effect by finding the optimal hyperplane. Specifically, the SVM uses the radial basis function RBF as its kernel function. The kernel function RBF is shown as K(x i , x j ) = exp(-γ||x i -x j ||), where γ is the hyperparameter of the Gaussian kernel function.

[0048] SVM finds the optimal hyperplane through the following formula (1.3):

[0049] <w,Φ(X)> +b=0 (1.3)

[0050] Among them, w is the weight coefficient, X is the support vector, Φ(x) is the Gaussian kernel function, and b is the offset vector.

[0051] In its prediction process, SVM determines the new sample according to formula (1.4) for feature data:

[0052]

[0053] Among them, class represents the classification of emotions, and categories 0 and 1 represent an emotion respectively, for example, 0 represents happiness and 1 represents unhappiness.

[0054] It should be noted that when executing Figure 2 In the process of the method described above, the classifier used in step S204 can be a classifier trained by other methods or a classifier in training, in addition to the SVM described above. Figure 2 The physiological data generated by the method shown for emotion recognition can be used to further train the classifier, making it a high-precision emotion classifier in the end.

[0055] In step S206, based on the comparison between the determined emotion and the actual emotion, the emotion data whose determined emotion is consistent with the actual emotion is screened out from the first physiological data and the second physiological data as the physiological data for emotion recognition. The determined emotion is the user emotion determined by the SVM based on the feature data of the input first physiological data and the second physiological data. It should be noted that the second physiological data is data obtained based on the first physiological data, so each sample in the second physiological data has corresponding sample data of the first physiological data. Because the second physiological data is generated from the first physiological data and is very close, in theory, the emotion determined based on the second physiological data should be consistent with the emotion determined by the corresponding first physiological data. If it is inconsistent, it means that the sample data in the second physiological data corresponding to the emotion deviates from the real physiological data and has low quality, and can be eliminated from the second physiological data.

[0056] Through Figure 2 After the processing shown, the physiological data subjected to the first quality assessment and screening and / or the second quality assessment and screening are further assessed to ensure that the obtained amplified data has extremely high quality while the sample data is amplified.

[0057] In summary, after executing the method for amplifying physiological data according to each example of the present application, more second physiological data can be amplified on the basis of the first physiological data collected from the user, so that physiological data with exponential growth and relatively high quality can be generated. Based on these large amounts of high-quality physiological data, an emotion prediction model with higher emotion prediction accuracy can be trained, such as a more accurate SVM.

[0058] Combination of the above Figure 1 and Figure 2 The method described can be performed by computers, portable electronic devices such as tablets and mobile phones, and various devices capable of performing data processing. For example, the method is implemented as a software module and executed by a processor of these devices. Figure 1 and Figure 2It is also feasible that the described method is implemented as a stand-alone device combining software and hardware.

[0059] Figure 3 : is a structural diagram of a system for amplifying physiological data according to some examples of the present application. The system includes a data amplification module 30 and a screening module 32. The data amplification module 30 is used to amplify the received preprocessed first physiological data to obtain second physiological data. As an example, each sample data in the first physiological data and each sample data in the second physiological data may be provided with a label. The screening module 32 is used to perform a first quality assessment and screening on the first physiological data and the second physiological data, and / or perform a second quality assessment and screening on the first physiological data and the second physiological data to generate physiological data for emotion recognition.

[0060] The screening module 32 may include a visualization module 320 and / or a clustering module 340. The visualization module 320 performs a first quality assessment and screening on the first physiological data and the second physiological data. Specifically, the distribution of the first physiological data and the second physiological data is assessed and the physiological data with a distribution deviation greater than a threshold is removed. Usually, the data removed should be the data in the second physiological data with a deviation from the first physiological data greater than the distribution threshold. The clustering module 340 is used to assess whether the clustering label of the first physiological data is consistent with the clustering label of the second physiological data, and remove the data in the second physiological data with a clustering label that does not conform to the clustering label of the first physiological data.

[0061] according to Figure 3 The system for amplifying physiological data shown can be used to perform a combination of Figure 1 The method for augmenting physiological data described herein, for example, is performed by the data augmentation module 30 in steps S100 and S102 ; and by the screening module 32 in step S104 .

[0062] In some examples, the system for augmenting physiological data further includes a re-screening module 36, which is used to generate feature data from the physiological data that has undergone the first quality assessment and screening and / or the physiological data that has undergone the second quality assessment and screening, determine emotions based on the generated feature data, and screen out physiological data whose determined emotions are consistent with the actual emotions from the physiological data that has undergone the first quality assessment and screening and / or the physiological data that has undergone the second quality assessment and screening as physiological data for emotion recognition based on the comparison between the determined emotions and the actual emotions. Wherein, both the physiological data that has undergone the first quality assessment and screening and the physiological data that has undergone the second quality assessment and screening can include the first physiological data and the second physiological data.

[0063] In a specific example, the re-screening module 36 is configured to generate feature data based on the physiological data output by the visualization module 320 and the physiological data output by the clustering module 340, and determine the user's emotions accordingly. The re-screening module 36 also screens out the physiological data whose determined emotions are consistent with the actual emotions as emotion recognition data based on the comparison between the determined emotions and the actual emotions. It should be noted that in this specific example, the physiological data of both the visualization module 320 and the clustering module 340 are processed at the same time. In actual applications, the re-screening module 36 may only process the physiological data output by the visualization module 320 or the physiological data output by the clustering module 340.

[0064] Also refer to Figure 3 , Figure 1 and Figure 2 . The data amplification module 30 receives (step S100) the preprocessed first physiological data, and amplifies it (step S102) to obtain second physiological data, and the second physiological data does not include the first physiological data. The quality assessment module 32 performs a first quality assessment and screening, and / or a second quality assessment and screening on the first physiological data and the second physiological data to generate physiological data for emotion recognition (step S104). Specifically, the visualization module 320 performs a first quality assessment and screening on the first physiological data and the second physiological data, and evaluates the distribution of the first physiological data and the second physiological data to remove physiological data whose distribution deviation is greater than a threshold. The visualization module 320, for example, uses a t-SNE model. The clustering module 340 is used to perform a second quality assessment and screening on the first physiological data and the second physiological data. The second quality assessment and screening includes evaluating whether the clustering label of the first physiological data and the clustering label of the second physiological data are consistent, and removing data whose clustering label in the second physiological data does not match the clustering label of the first physiological data. The clustering module 340, for example, uses a K-means clustering model.

[0065] The re-screening module 36 combines the evaluated and screened physiological data output by the visualization module 320 and the evaluated and screened physiological data output by the clustering module 340. Figure 2 The further evaluation described. The re-screening module 36 receives (step S200) the data after visualization evaluation and screening and receives the data after cluster evaluation and screening, generates (step S202) feature data, and determines (step S204) the user's emotions based on the feature data. Among them, the data after visualization evaluation is the data after the first quality evaluation, and the data after cluster evaluation is the data after the second quality evaluation, and they both include the first physiological data and the second physiological data. The re-screening module 36 further screens out the emotion recognition data from the first physiological data and the second physiological data based on the comparison between the determined emotion and the actual emotion (step S206).

[0066] Combination of the above Figure 1 , Figure 2 The features, combinations of features, definitions, etc. described herein are applicable to all examples of this application. Figure 1 The visualization model and clustering model described are suitable for combining Figure 3 Described system etc.

[0067] Combination Figure 3 The described system may be implemented as a module, such as a module combining software and hardware, so as to be disposed in other devices such as computers, tablets, mobile phones, etc. for use, or implemented as an independent device to be connected to these devices in a communicative manner for use.

[0068] Figure 4 4 is a schematic diagram of a device for amplifying physiological data according to some examples of the present application. The device includes a processor 40. In some cases, the device may include a memory 42 shown in dashed lines, and in other cases, the device may be connected to an external storage device that can be used as the memory 42 (shown in the figure as a solid line). The processor 40 is configured to be able to execute program instructions stored in the memory 42, thereby implementing any of the methods for amplifying physiological data described above.

[0069] This application also provides an emotion recognition system. Figure 5 : is a schematic diagram of the structure of the emotion recognition system. The system includes a data acquisition device 50 and a pre-trained classifier 52. The data acquisition device 50 is used to acquire user physiological data. In some examples, the data acquisition device is configured to use user physiological signals, filter the original physiological data carried by the collected physiological signals, extract features, and perform a series of processes to generate physiological data that is fed into the pre-trained classifier 52. Here, the pre-trained classifier 52 classifies the user's emotions according to the acquired user physiological data, and the classifier is, for example, SVM. Among them, the training data used to train the classifier 52 is physiological data obtained in one of the following ways: physiological data for emotion recognition obtained by the device for amplifying physiological data described above, or physiological data for emotion recognition obtained by the system for amplifying physiological data described above, or physiological data for emotion recognition obtained by executing the method for amplifying physiological data described above.

[0070] Figure 6 FIG. 1 is a schematic diagram of the structure of an emotion recognition system according to some other examples of the present application. Figure 6As shown, the emotion recognition system includes a collection device 60, a physiological data amplification device 62 and a classifier 64. The collection device 60 is used to collect the user's electroencephalogram signal as the first physiological data. The physiological data amplification device 62 is used to amplify the first physiological data to obtain the second physiological data, and perform a first quality assessment and screening, and / or a second quality assessment and screening on the second physiological data to generate the second physiological data for emotion recognition, wherein the second physiological data does not include the first physiological data. The classifier 64 performs emotion recognition based on the first physiological data and the generated second physiological data for emotion recognition.

[0071] Among them, the physiological data amplification device 62 is also used for: based on the visualization model, performing a first quality assessment and screening on the second physiological data, the first quality assessment and screening including evaluating the distribution of the second physiological data, and removing the second physiological data whose distribution deviation is greater than a threshold compared with the first physiological data; and / or, based on the clustering model, performing a second quality assessment and screening on the second physiological data, the second quality assessment and screening including evaluating whether the clustering label of the second physiological data is consistent with the clustering label of the first physiological data, and removing data in the second physiological data whose clustering label is inconsistent with the clustering label of the first physiological data.

[0072] For example Figure 6 In the example, the acquisition device 60 is, for example, an electroencephalogram signal acquisition device, and accordingly, the physiological signal is an electroencephalogram signal, and each physiological data is the data carried by the signal. The physiological data amplification device 62 can, for example, use a GAN model to generate the second physiological data, and can, for example, use the visualization model and / or clustering model described above to perform quality assessment and screening on the amplified data to obtain high-quality amplified data. As an example, the visualization model and / or clustering model at least performs quality assessment and screening on the second physiological data. As an example, the classifier 64 is a classifier that has been pre-trained based on the physiological data for emotion recognition generated by any one of the methods for amplifying physiological data described above, or the physiological data for emotion recognition generated by the system for amplifying physiological data described above, or the physiological data for emotion recognition generated by the device for amplifying physiological data described above. The classifier 64 can also be any other classifier for emotion classification, that is, using conventional or existing classifiers, based on the amplified, quality-assessed and screened physiological data obtained by the method according to the embodiment of the present invention, to perform emotion recognition. The amplified, quality-assessed and screened physiological data of the embodiment of the present invention can be used not only as training data in the training phase of the classifier, but also in the execution phase of the emotion recognition task.

[0073] The present application also provides an electronic device, which includes the system for amplifying physiological data as described above, or includes the device as described above, or includes any one of the emotion recognition systems as described above. The electronic device can be, for example, a computer, a tablet, a smart phone, etc. Here, the electronic device can also be, for example, an electronic system of a vehicle.

[0074] Although specific embodiments of the present application have been shown and described in detail to illustrate the principles of the present application, it should be understood that the present application can be implemented in other ways without departing from such principles.

Claims

1. A method for augmenting physiological data, characterized in that: The method comprises: receiving preprocessed first physiological data; amplifying the received first physiological data by a data augmentation model to obtain second physiological data, wherein the second physiological data does not include the first physiological data; Performing a first quality assessment and screening on the first physiological data and the second physiological data, and / or performing a second quality assessment and screening on the first physiological data and the second physiological data to generate physiological data for emotion recognition; wherein, based on the visualization model, a first quality assessment and screening is performed on the first physiological data and the second physiological data, wherein the first quality assessment and screening includes assessing the distribution of the first physiological data and the second physiological data, and removing physiological data with a distribution deviation greater than a threshold; and / or, Among them, based on the clustering model, a second quality assessment and screening is performed on the first physiological data and the second physiological data, and the second quality assessment and screening includes evaluating whether the clustering labels of the first physiological data and the clustering labels of the second physiological data are consistent, and removing data in the second physiological data whose clustering labels are inconsistent with the clustering labels of the first physiological data.

2. The method for augmenting physiological data according to claim 1, characterized in that: The visualization model includes a t-SNE algorithm model.

3. The method for augmenting physiological data according to claim 1, characterized in that: The clustering model includes a K-means algorithm.

4. The method for augmenting physiological data according to claim 1, characterized in that: The method further comprises: Generating characteristic data from the physiological data that has undergone the first quality assessment and screening and / or the physiological data that has undergone the second quality assessment and screening; determining a sentiment based on the generated feature data; Based on the comparison between the determined emotion and the actual emotion, physiological data whose determined emotion is consistent with the actual emotion is screened out from the physiological data that has undergone the first quality assessment and screening and / or the second quality assessment and screening as physiological data for emotion recognition, wherein each sample in the first physiological data and the second physiological data has an emotion label, and the emotion label serves as the actual emotion corresponding to each sample.

5. A system for augmenting physiological data, characterized in that: The system comprises: a data amplification module, configured to amplify the received preprocessed first physiological data to obtain second physiological data, wherein the second physiological data does not include the first physiological data; a screening module, configured to perform a first quality assessment and screening on the first physiological data and the second physiological data, and / or perform a second quality assessment and screening on the first physiological data and the second physiological data, so as to generate physiological data for emotion recognition; Wherein, the screening module further comprises: a visualization module, configured to perform a first quality assessment and screening on the first physiological data and the second physiological data, wherein the first quality assessment and screening includes assessing the distribution of the first physiological data and the second physiological data, and removing physiological data having a distribution deviation greater than a threshold; and / or A clustering module is used to perform a second quality assessment and screening on the first physiological data and the second physiological data, wherein the second quality assessment and screening includes: assessing whether the clustering labels of the first physiological data and the clustering labels of the second physiological data are consistent, and removing data in the second physiological data whose clustering labels do not match the clustering labels of the first physiological data.

6. The system for augmenting physiological data according to claim 5, characterized in that: The system further comprises: A re-screening module is used to generate characteristic data from the physiological data that has undergone a first quality assessment and screening and / or the physiological data that has undergone a second quality assessment and screening, determine emotions based on the generated characteristic data, and based on a comparison between the determined emotions and the actual emotions, screen out physiological data whose determined emotions are consistent with the actual emotions from the physiological data that has undergone a first quality assessment and screening and / or the physiological data that has undergone a second quality assessment and screening as physiological data for emotion recognition.

7. A device for augmenting physiological data, characterized in that: The apparatus comprises a processor configured to execute program instructions stored in a memory to implement the steps of the method according to any one of claims 1 to 4.

8. An emotion recognition system, characterized in that: The emotion recognition system comprises: A data acquisition device, used to acquire physiological data of a user; A pre-trained classifier for classifying user emotions based on the acquired user physiological data; The training data used to train the classifier is physiological data for emotion recognition obtained by the device for amplifying physiological data as described in claim 7, or physiological data for emotion recognition obtained by the system for amplifying physiological data as described in claim 5 or 6, or physiological data for emotion recognition obtained by executing the method for amplifying physiological data as described in any one of claims 1 to 4.

9. An emotion recognition system, characterized in that: include: A collection device, used for collecting a user's physiological signal as first physiological data; A physiological data amplification device, configured to amplify the first physiological data to obtain second physiological data, and perform a first quality assessment and screening, and / or a second quality assessment and screening on the second physiological data to generate second physiological data for emotion recognition, wherein the second physiological data does not include the first physiological data; a classifier, performing emotion recognition based on the first physiological data and the generated second physiological data for emotion recognition; Among them, the physiological data amplification device is also used for: based on the visualization model, performing a first quality assessment and screening on the second physiological data, the first quality assessment and screening includes evaluating the distribution of the second physiological data, and removing the second physiological data whose distribution deviation is greater than a threshold compared with the first physiological data; and / or, based on the clustering model, performing a second quality assessment and screening on the second physiological data, the second quality assessment and screening includes evaluating whether the clustering label of the second physiological data is consistent with the clustering label of the first physiological data, and removing data in the second physiological data whose clustering label is inconsistent with the clustering label of the first physiological data.

10. The emotion recognition system according to claim 9, characterized in that: The classifier includes physiological data for emotion recognition generated by the method according to any one of claims 1 to 4, or physiological data for emotion recognition generated by the system according to claim 5 or 6, or physiological data for emotion recognition generated by the device according to claim 7 or 8, a pre-trained classifier.

11. An electronic device, comprising the system for augmenting physiological data according to claim 5 or 6, or comprising the device according to claim 7, or comprising the emotion recognition system according to any one of claims 8 to 10.

12. The method for augmenting physiological data according to any one of claims 1 to 4, the system for augmenting physiological data according to claim 5 or 6, the device for augmenting physiological data according to claim 7, the emotion recognition system according to any one of claims 8 to 10, and the electronic device according to claim 11, wherein: The first physiological data includes collected brain electrical signal data of the user.