Emotion recognition method and system based on dynamic clustering and variational Bayesian element learning

Abstract

The invention discloses an emotion recognition method and system based on dynamic clustering and variational Bayesian element learning, and the method specifically comprises the steps: dynamically determining the number of crowd types through the process of a Chinese restaurant, and carrying out the emotion recognition of different crowd types according to the difference of modes for carrying out emotion expression through physiological signals. The thought of Bayesian meta learning is introduced to learn the commonality of crowd type emotion expression modes, and the characteristics of crowd individual emotion expression modes are learned through the sub-process of meta learning. The method specifically comprises the four steps of data arrangement and preprocessing, initialization of crowd dynamic clustering process parameters and meta parameters in a model, online training and parameter updating of an algorithm model, and model performance testing and putting into use. According to the method, emotion classification models with different parameter distributions can be established for different crowd types, so that emotion recognition with relatively high precision is realized, and meanwhile, the method has strong generalization ability of knowledge migration among different crowds.

Description

Technical Field

[0001] The present invention belongs to the field of information technology, and in particular relates to an emotion recognition method and system using electroencephalogram signals based on dynamic clustering and variational Bayesian meta-learning. Background Art

[0002] In recent years, as society as a whole has paid more and more attention to people's mental health diseases, the corresponding emotion recognition technology has also been developing continuously. As an important physiological feature that is different from traditional facial expressions, voices and other easy-to-disguise features, EEG signals have received extensive attention and research in the field of emotion recognition. However, when dealing with the emotion classification of different groups of people, existing emotion recognition methods often find it difficult to balance the accuracy of emotion classification and the generalization ability of the model when dealing with the physiological signal characteristics of different groups of people.

[0003] Traditional emotion recognition methods usually assume that all individuals have the same emotion classification pattern (such as the general population, psychologically sensitive people, or people with special mental illnesses). This makes traditional emotion recognition methods often less than ideal in practical applications. A simple example is that different people will have different emotions when experiencing the same event. If an emotion classification model is trained for a certain group of people, there will be two difficulties. First, there is no unified metric to divide the group in terms of the dimension of emotion expression patterns. Second, it is often difficult to obtain a large amount of experimental data from the corresponding group in practical applications to train the model in a targeted manner. Therefore, it is of great significance to study a method that can dynamically adapt to the emotion classification patterns of different groups of people and obtain higher emotion classification performance with the support of a small number of samples in the field of emotion research and application. Summary of the invention

[0004] The purpose of the present invention is to solve the problem that the emotion recognition methods in the prior art often find it difficult to balance the classification accuracy of emotions and the generalization ability of the model in dealing with the physiological signal characteristics of different groups of people when dealing with the emotion classification of different groups of people, and that traditional emotion recognition methods usually assume that the emotion classification patterns of all individuals are the same, which leads to the problem that traditional emotion recognition methods are often less than ideal in practical applications.

[0005] In order to solve the above problems, the present invention provides an emotion recognition method based on dynamic clustering and variational Bayesian meta-learning, comprising the following steps:

[0006] Step S1: data sorting and preprocessing;

[0007] Step S1-1: sorting the existing EEG signals of the subjects, uniformly downsampling the EEG signals to 128 Hz, filtering the EEG signals using a 5 to 45 Hz bandpass filter, and obtaining the physiological signal feature data called by the method;

[0008] Step S1-2: The subject number is t, and the set of physiological signal characteristic data sequences obtained from the subject is recorded as:

[0009]

[0010] The subject's true emotion label set is recorded as:

[0011]

[0012] make

[0013] X t ∈R C×N

[0014] Where, X t represents the multi-channel physiological signal obtained from the subject in a single emotional stimulation experiment, C represents the number of characteristic channels of the physiological signal, N represents the product of the duration and frequency of the physiological signal, that is, the number of time points at which the physiological signal is finally obtained, R represents a real number set, Nt represents the number of physiological signal samples with emotion labels obtained from the tth subject, and the corresponding emotion label Y t It is given in the form of one-hot encoding. Taking the dimension of like degree in emotion annotation of DEAP dataset as an example, the subject's emotions are divided into two categories: like and dislike, which are represented by vectors [0,1] and [1,0] respectively.

[0015] Step S1-3: The sample set of the subject Split into support set samples for training the subject's own emotion classifier and a sample queryset for testing The basis for splitting is to randomly shuffle and then split according to the proportion;

[0016] Step S2: Initialization of parameters and meta-parameters of the crowd dynamic clustering process in the model;

[0017] Step S2-1: The crowd dynamic clustering parameter initialization process based on the Chinese restaurant process CRP is to set a hyperparameter α to determine the crowd category of the newly entered i-th subject, and use zi to represent the label of its category. The formula is as follows:

[0018]

[0019] Where n k It represents the number of subjects expected to be divided into the kth population. α>0 is used as a concentration hyperparameter to control the number of population types. A larger α will lead to more population types. Each population will be described as CRP-CROWD, and the number of population types is confirmed by CRP.

[0020] Step S2-2: The probability of classifying a subject into a certain population category needs to be completed by constructing a broken stick representation for each population category. According to the principle of Dirichlet process, the formula of the initialization process is:

[0021] β k |α~Beta(1,α) (2)

[0022]

[0023] In the formula, π k Represents a probability value vector, π = [π1, π2, ...], where the value of each element represents the probability of classifying the subject into different population categories. It is necessary to ensure that the value of each element in the vector is non-negative and the sum of all values ​​is 1. The probability of the kth category is expressed as π k ;

[0024] Step S2-3: Traditional meta-learning assumes that there is a meta-model with parameter θ. The meta-model is used to generate an untrained emotion classifier with initialized parameters adapted to the population for each subject belonging to the population category, which is regarded as a sub-task in this method. The emotion classifier is trained with a small amount of data of the sub-task and then applied to the sub-task.

[0025] Use the meta parameters to generate a corresponding emotion classifier. The generation process is:

[0026] z t ∣π~Categorical ∞ (π), (4)

[0027]

[0028] In the formula, represents a mixed Gaussian distribution, and Categorical(π) represents sampling from each component of the probability value vector π according to the corresponding probability value to obtain a certain z t = k, that is, the current subject t is clearly divided into the corresponding population category, and the meta-parameter distribution of the population category to which the subject belongs is determined; after sampling, the meta-parameters of the corresponding population category are obtained, and the population category to which the current subject belongs is determined to be z t , and sample a corresponding meta-parameter framework from the classifier parameter distribution of the corresponding population It indicates that it obeys the mean vector of the mixed distribution. Represents the semi-positive covariance matrix, and the Gaussian distribution of the meta-parameters of the corresponding population category is determined by the two. Finally, by using the meta-parameter distribution of the corresponding population obtained by sampling, a corresponding emotion classifier sub-model with initialization parameters is generated for the current subject and invested in the emotion classification task of the subject. The process is:

[0029]

[0030] Step S3: Online training and parameter update of algorithm model;

[0031] Step S3-1: Through structured inference, through online training, that is, supporting the data of the subjects to be input one by one, the model updates the parameters accordingly. When the sample data of the t-th subject is input, the distribution parameters obeyed by the four parameters are output through the posterior method, and the β in Formula 2 is replaced by k The parameter of the Beta distribution is denoted by γ k , replace z in Formula 4 t The parameter of the Categorical distribution is denoted as η t , replace θ in Formula 5 t The parameter of the Gaussian mixture distribution is denoted by ψ t , replace the task-specific parameter φ in Formula 6 t The variational parameter of t , the model parameters can be continuously updated or put into use online. The specific formula is:

[0032]

[0033] When the tth subject arrives, the observed evidence lower bound ELOB is further described according to Formula 8:

[0034]

[0035] In the formula, represents mathematical expectation, KL[·|·] represents Kullback–Leibler divergence. By maximizing this loss function, the goal of optimizing model parameters can be achieved. q represents the distribution predicted by the model, and p represents the actual distribution.

[0036] Step S3-2: γ k , η t , t , t Generation and optimization process of four distribution parameters:

[0037] (1) Variational distribution of β. Considering only the items related to β in ELOB, its optimal variational distribution is:

[0038] q(βk ; γ k )=Beta(β k ; γ k,1 ,γ k,2 )

[0039] That is, a Beta distribution with parameters, where γ k,1 =1+η t,k ,

[0040] (2) z t The variational distribution of t The related items derive their optimal variational distribution as:

[0041] q(z t ; η t )=Categorical K (z t ; η t )

[0042] That is, a classifier distribution with parameters:

[0043]

[0044] In the formula, is the variational distribution of β;

[0045] (3)θ t When the tth subject arrives, it is inferred that the kth population category z t After that, at this time, not only the physiological signal samples of the current subjects need to be considered, but also the knowledge parameters of the corresponding population types learned in the past learning process need to be applied, that is, the common emotional expression patterns of the corresponding population types, and the meta-learning parameters of the corresponding population types The distribution is expressed as:

[0046]

[0047] Assume that each population type is subject to a parameter:

[0048] Gaussian distribution, without loss of generality, after the arrival of the tth subject, the new The prior knowledge of The a posteriori knowledge of is given, namely:

[0049]

[0050] After the current subject's data training is completed, the new posterior knowledge is replaced to accept the new subject emotion classification task, and the parameters are updated by stochastic gradient descent;

[0051] (4)φ t The variational distribution of , that is, the generated emotion classifier with initialization parameters, is formulated as:

[0052]

[0053] In the formula, φ t represents the neural network parameters of the emotion classifier generated by this method for the current subject, λ t represents the variational parameters of the sampling. In order to reduce the training pressure of the neural network, with the help of the idea of ​​amortized variational reasoning, the variational parameters of the population category to which the current subject belongs have been determined Training data for and current subject Perform several steps of gradient descent, namely:

[0054]

[0055] In the formula, J is the step of gradient descent, ∈ is the learning rate, and the gradient descent strategy uses stochastic gradient descent;

[0056] Step S4: performance testing and commissioning;

[0057] When a subject's physiological signal sample set has only a few sample labels After being introduced into this method, the neural network trained in step S3 is called to accurately and dynamically divide the subjects into corresponding groups, and the meta-learning framework that has been trained in the corresponding group is called to generate an emotion classifier with initialization parameters for the subject. The physiological signal is used as input in an end-to-end manner, and the emotion classification result is output. The emotion classifier is quickly trained using a small number of corresponding samples and can be put into use or tested. At this time, the test step is that when one or a batch of physiological signal samples without emotion annotations are passed in, the corresponding physiological signal samples are input into the corresponding emotion classifier, and the emotion prediction output of the meta-parameter model for the physiological signal sample can be obtained.

[0058] In a preferred embodiment, the emotion classifier sub-model is a replaceable component, and in this method, a module based on a multi-layer convolutional neural network is selected.

[0059] In a preferred manner, in a real scenario, any component can be replaced as long as it is a trainable parameter-containing network that can encode the original multi-channel physiological signal into an embedded vector through a parameterized neural network.

[0060] In a preferred embodiment, the split ratio in step S1-3 is 60% and 40%.

[0061] An emotion recognition system based on dynamic clustering and variational Bayesian meta-learning includes: a data sorting and data preprocessing module, a dynamic clustering parameter module based on the Chinese restaurant process CRP, a meta-parameter module, a model parameter updating module, and a model commissioning module:

[0062] Step M1: The implementation of the data sorting and preprocessing module specifically includes:

[0063] Step M1-1: sort out the existing EEG signals of the subjects, uniformly downsample the EEG signals to 128 Hz, and filter the EEG signals using a 5 to 45 Hz bandpass filter to obtain the physiological signal feature data called by the method;

[0064] Step M1-2: The subject number is t, and the set of physiological signal characteristic data sequences obtained from the subject is recorded as:

[0065]

[0066] The subject's true emotion label set is recorded as:

[0067]

[0068] make

[0069] X t ∈R C×N

[0070] In the formula, X t represents the multi-channel physiological signals collected from the subjects in a single emotional stimulation experiment, C represents the number of characteristic channels of the EEG signal, N represents the product of the duration and frequency of the physiological signal, that is, the number of time points at which the physiological signal is finally obtained, R represents a real number set, Nt represents the number of physiological signal samples with emotion labels obtained from the tth subject, and the corresponding emotion label Y t It is given in the form of one-hot encoding. Taking the dimension of the degree of sentiment annotation of the DEAP dataset as an example, it is divided into two emotions: like and dislike, and represented by vectors [0,1] and [1,0] respectively;

[0071] Step M1-3: The sample set of the subject Split into support set samples for training the subject's own emotion classifier and a sample queryset for testing The basis for splitting is to randomly shuffle and split according to the ratio of 60% and 40% respectively;

[0072] Step M2: Dynamic clustering parameter module based on Chinese restaurant process CRP;

[0073] Step M2-1: The Chinese restaurant process controls the category judgment of the newly entered i-th subject by setting a hyperparameter α and uses z i To represent its category label, the formula is as follows:

[0074]

[0075] Where n k It represents the number of subjects expected to be divided into the kth population. α>0 is used as a concentration hyperparameter to control the number of population types. A larger α will lead to more population types. Each population will be described as CRP-CROWD, and the number of population types is confirmed by CRP.

[0076] Step M2-2: The probability of classifying a subject into a certain population category needs to be completed by constructing a broken stick representation for each population category. According to the principle of the Dirichlet process, the specific formula is:

[0077] β k |α~Beta(1,α), (2)

[0078]

[0079] In the formula, π k Represents a probability value vector, π = [π1, π2, ...], where the value of each element represents the probability of classifying the subject into different population categories. It is necessary to ensure that the value of each element in the vector is non-negative and the sum of all values ​​is 1. The probability of the kth category is expressed as π k ;

[0080] Step M3: The specific implementation of the meta-parameter module is as follows: for each k-th population category, a corresponding Bayesian meta-parameter distribution must be maintained for it, that is, the traditional meta-learning assumes that there is a meta-model with a parameter θ, and the meta-model is used to generate an emotion classifier with initialized parameters but untrained for each subject belonging to the population category. The emotion classifier is trained with a small amount of data of the subtask and then applied to the subtask;

[0081] The value of the meta-model θ is uncertain, but obeys the Gaussian distribution. Each time Bayesian meta-learning is used to generate parameters for the sub-model, a meta-parameter value is first sampled from the Gaussian distribution obeyed by the meta-model, and a corresponding emotion classifier is generated using the meta-parameter. The generation process is as follows:

[0082] z t ∣π~Categorical ∞(π), (4)

[0083]

[0084] In the formula, represents a mixed Gaussian distribution, and Categorical(π) represents sampling from each component of the probability value vector π according to the corresponding probability value to obtain a certain z t = k, that is, the current subject t is clearly divided into the corresponding population category, and the meta-parameter distribution of the population category to which the subject belongs is determined. After sampling, the meta-parameters of the corresponding population category are obtained. At this time, it is determined that the population category to which the current subject belongs is z t , and sample a corresponding meta-parameter framework from the classifier parameter distribution of the corresponding population represents the mean vector, Represents the semi-positive covariance matrix, and the Gaussian distribution of the meta-parameters of the corresponding population category is determined by the two. Finally, by using the meta-parameter distribution of the corresponding population obtained by sampling, a corresponding emotion classifier sub-model with initialization parameters is generated for the current subject and invested in the emotion classification task of the subject. The process is:

[0085]

[0086] Step M4: model parameter updating module;

[0087] Step M4-1: Directly use the process described by Formula 2 to Formula 7. When using the Monte Carlo method to update the model, the β k ,z t ,θ t ,φ t The problem of difficulty in estimating the specific parameters of the posterior distribution obeyed by the proposed method is that this method uses structured inference to input the sample data of the tth subject and output the distribution parameters obeyed by the four parameters. k The parameter of the Beta distribution is denoted by γ k , replace z in Formula 4 t The parameter of the Categorical distribution is denoted as η t , replace θ in Formula 5 t The parameter of the Gaussian mixture distribution is denoted by ψ t , replace the task-specific parameter φ in Formula 6 t The variational parameter of t , the model parameters can be continuously updated or put into use online, and the real distribution parameters are approximated using structured variational posterior. The specific formula is:

[0088]

[0089] When the tth subject arrives, the observed evidence lower bound ELOB is further described according to Formula 8:

[0090]

[0091] In the formula, represents mathematical expectation, KL[·|·] represents Kullback–Leibler divergence. The goal of optimizing model parameters can be achieved by maximizing this loss function. p represents the distribution predicted by the model, and q represents the actual distribution.

[0092] Step M4-2: γ k , η t , t , t Generation and optimization process of four distribution parameters:

[0093] (1) Variational distribution of β. Considering only the items related to β in ELOB, the optimal variational distribution is:

[0094] q(β k ; γ k )=Beta(β k ; γ k,1 ,γ k,2 )

[0095] That is, a Beta distribution with parameters, where γ k,1 =1+η t,k ,

[0096] (2) z t The variational distribution of t The related items derive their optimal variational distribution as:

[0097] q(z t ; η t )=Categorical K (z t ; η t )

[0098] That is, a classifier distribution with parameters:

[0099]

[0100] In the formula is the variational distribution of β;

[0101] (3)θ tWhen the tth subject arrives, it is inferred that the kth population category z t After that, at this time, not only the physiological signal samples of the current subjects need to be considered, but also the knowledge parameters of the corresponding population types learned in the past learning process need to be applied, that is, the common emotional expression patterns of the corresponding population types. Therefore, the meta-learning parameters of the corresponding population types The distribution is expressed as:

[0102]

[0103] Assume that each population type is subject to a parameter Gaussian distribution, without loss of generality, after the arrival of the tth subject, the new The prior knowledge of The a posteriori knowledge of is given, namely:

[0104]

[0105] After the current subject's data training is completed, the new posterior knowledge is replaced to accept the new subject emotion classification task, and the parameters are updated by stochastic gradient descent;

[0106] (4)φ t The variational distribution of , that is, the generated emotion classifier with initialization parameters, is in the formula:

[0107]

[0108] In the formula, φ t represents the neural network parameters of the emotion classifier generated by this method for the current subject, λ t represents the variational parameters of the sampling. In order to reduce the training pressure of the neural network, this method uses the idea of ​​amortized variational reasoning. The variational parameters of the population category to which the current subject belongs have been determined Training data for and current subject Perform several steps of gradient descent, namely:

[0109]

[0110] In the formula, J is the step of gradient descent, ∈ is the learning rate, and the gradient descent strategy uses stochastic gradient descent;

[0111] Step M5: Model put into use module;

[0112] When a subject's physiological signal sample set has only a few sample labels After being introduced into this method, the neural network trained in step S3 is called to accurately and dynamically divide the subjects into corresponding groups, and the meta-learning framework that has been trained in the corresponding group is called to generate an emotion classifier with initialization parameters for the subject. The physiological signal is used as input in an end-to-end manner, and the emotion classification result is output. The emotion classifier is quickly trained using a small number of corresponding samples and can be put into use or tested. At this time, the test step is that when one or a batch of physiological signal samples without emotion annotations are passed in, the corresponding physiological signal samples are input into the corresponding emotion classifier, and the emotion prediction output of the meta-parameter model for the physiological signal sample can be obtained.

[0113] The Monte Carlo method involves randomly generating a large number of parameter samples and applying them to the model to observe the classification performance.

[0114] The EEG signal in step S1 is information that has been stored and recorded by existing equipment, or information obtained through other means. The invention of the present invention is not about collecting signals, but about subsequent processing of existing signals.

[0115] Likewise, the present invention relates to the following contents:

[0116] A set of physiological signal characteristic data sequences obtained from a subject; X t represents the multi-channel physiological signal obtained from the subject in a single emotional stimulation experiment, C represents the number of characteristic channels of the physiological signal, N represents the product of the duration and frequency of the physiological signal, that is, the number of time points at which the physiological signal is finally obtained, R represents a real number set, Nt represents the number of physiological signal samples with emotion labels obtained from the tth subject, and the corresponding emotion label Y t The data are given in the form of one-hot encoding. The present invention does not involve the process of obtaining the above data, and the present invention only uses the existing data to perform subsequent steps.

[0117] To be more specific, the present invention does not involve the process of directly acting on the human body to obtain signals. The concepts of "obtaining signals" or "collecting signals" involved in the present invention are merely to use existing signal data, not to collect signals by acting on the human body.

[0118] The present invention can achieve the following beneficial effects: in terms of the performance of emotion classification, using the accuracy index ACC commonly used in the field of emotion classification, compared with other traditional machine learning methods that do not distinguish between people, the classic machine learning model such as the support vector machine can reach 69.85%; the traditional CNN convolutional network can reach an accuracy of 76.45%; the latest neural network based on deep learning, such as the Tsception framework, can reach a performance of about 82.37%: and the present invention can reach a classification accuracy of about 85.23% under the same experimental setting and using the same experimental environment, and the emotion classification performance is achieved in the following aspects that exceed the existing methods. Compared with other traditional studies, the present invention supports model testing on individuals of the population with a small number of samples. For example, in the DEAP data set, the present invention can be put into application after training the method framework using a small number of individual samples of the population (40 emotion samples of 16 subjects), and the accuracy of emotion classification can reach about 75%, which effectively proves the effectiveness of the present invention in a small number of sample scenarios and the effectiveness of the innovative idea of ​​distinguishing different populations. BRIEF DESCRIPTION OF THE DRAWINGS

[0119] Figure 1 is a flow chart of the method of the present invention;

[0120] Figure 2 The figure shows the difference in parameter setting between the present invention and traditional meta-learning, as well as a schematic diagram of parameter generation. DETAILED DESCRIPTION

[0121] Combination Figure 1 The method flow of the present invention is shown, comprising the following steps:

[0122] Step S1: data sorting and preprocessing;

[0123] Step S1-1: sorting the existing EEG signals of the subjects, uniformly downsampling the EEG signals to 128 Hz, and filtering the EEG signals using a 5 to 45 Hz bandpass filter to obtain physiological signal feature data that can be called by the method;

[0124] Step S1-2: The subject number is t, and the physiological signal characteristic data sequence set obtained from the subject is recorded as:

[0125]

[0126] The subject's true emotion label set is recorded as:

[0127]

[0128] make

[0129] X t ∈RC×N

[0130] In the formula, X t represents the multi-channel physiological signals obtained from the subjects in a single emotional stimulation experiment, C represents the number of characteristic channels of the physiological signal, N represents the product of the duration and frequency of the physiological signal, that is, the number of time points at which the physiological signal is finally obtained, R represents a real number set, Nt represents the number of physiological signal samples with emotion labels obtained from the tth subject, and the corresponding emotion label Y t It is given in the form of one-hot encoding. Taking the dimension of like degree in emotion annotation of DEAP dataset as an example, the subject's emotions are divided into two categories: like and dislike, and represented by vectors [0,1] and [1,0] respectively;

[0131] Step S1-3: The sample set of the subject Split into support set samples for training the subject's own emotion classifier and a sample queryset for testing The basis for splitting is to randomly shuffle and split according to the ratio of 60% and 40% respectively;

[0132] Step S2: Initialization of parameters and meta-parameters of the crowd dynamic clustering process in the model;

[0133] Step S2-1: The Chinese restaurant process sets a hyperparameter α to classify the newly entered i-th subject and uses z i To represent the label of its category, the formula is as follows:

[0134]

[0135] Where n k It represents the number of subjects expected to be divided into the kth population. α>0 is used as a concentration hyperparameter to control the number of population types. A larger α will lead to more population types. Each population will be described as CRP-CROWD, and the number of population types will be confirmed by CRP.

[0136] Step S2-2: The probability of classifying a subject into a certain population category needs to be completed by constructing a broken stick representation for each population category. According to the principle of the Dirichlet process, the specific formula is:

[0137] β k |α~Beta(1,α), (2)

[0138]

[0139] In the formula, π kRepresents a probability value vector, π = [π1, π2, ...], where the value of each element represents the probability of classifying the subject into different population categories. It is necessary to ensure that the value of each element in the vector is non-negative and the sum of all values ​​is 1. The probability of the kth category is expressed as π k ;

[0140] Step S2-3: For each k-th population category, a corresponding Bayesian meta-parameter distribution is maintained for it. That is, the traditional meta-learning assumes that there is a meta-model with parameter θ. The meta-model is used to generate an untrained emotion classifier with initialized parameters adapted to the population for each subject belonging to the population category. The emotion classifier is trained with a small amount of data of the subtask and then applied to the subtask.

[0141] The value of the meta-model θ is uncertain, but obeys the Gaussian distribution. Each time Bayesian meta-learning is used to generate parameters for the sub-model, a meta-parameter value is first sampled from the Gaussian distribution obeyed by the meta-model, and a corresponding emotion classifier is generated using the meta-parameter. The generation process is as follows:

[0142] z t ∣π~Categorical ∞ (π), (4)

[0143]

[0144] In the formula, represents a mixed Gaussian distribution, and Categorical(π) represents sampling from each component of the probability value vector π according to the corresponding probability value to obtain a certain z t = k, that is, the current subject t is clearly divided into the corresponding population category, and the meta-parameter distribution of the population category to which the subject belongs is determined. After sampling, the meta-parameters of the corresponding population category are obtained. At this time, it can be determined that the current subject belongs to the population category z t , and sample a corresponding meta-parameter framework from the classifier parameter distribution of the corresponding population represents the mean vector, represents the semi-positive covariance matrix. The Gaussian distribution of the meta-parameters of the corresponding population category can be determined by the two. Finally, by using the meta-parameter distribution of the corresponding population obtained by sampling, a corresponding emotion classifier sub-model with initialization parameters can be generated for the current subject and put into the emotion classification task of the subject. The process is:

[0145]

[0146] Step S3: Online training and parameter update of algorithm model;

[0147] Step S3-1: The present invention uses structured inference to input the sample data of the t-th subject, output the distribution parameters obeyed by the four parameters, and replace β in Formula 2 with k The parameter of the Beta distribution is denoted by γ k , replace z in Formula 4 t The parameter of the Categorical distribution is denoted as η t , replace θ in Formula 5 t The parameter of the Gaussian mixture distribution is denoted by ψ t , replace the task-specific parameter φ in Formula 6 t The variational parameter of t , the model parameters can be continuously updated or put into use online. The real distribution parameters can be approximated using structured variational posterior. The specific formula is:

[0148]

[0149] When the tth subject arrives, the observed evidence lower bound ELOB can be further described according to Formula 8:

[0150]

[0151] In the formula, represents mathematical expectation, KL[·|·] represents Kullback–Leibler divergence. The goal of optimizing model parameters can be achieved by maximizing this loss function. q represents the distribution predicted by the model, and p represents the true distribution.

[0152] Step S3-2: Describing γ k , η t , t , t Generation and optimization process of four distribution parameters:

[0153] (1) Variational distribution of β. Considering only the items related to β in ELOB, the optimal variational distribution is:

[0154] q(β k ; γ k )=Beat(β k ; γ k,1 ,γ k,2 )

[0155] That is, a Beta distribution with parameters, where γ k,1 =1+η t,k ,

[0156] (2) z tThe variational distribution of t The related items derive their optimal variational distribution as:

[0157] q(z t ; η t )=Categorical K (z t ; η t )

[0158] That is, a classifier distribution with parameters:

[0159]

[0160] In the formula, is the variational distribution of β;

[0161] (3)θ t When the tth subject arrives, it is inferred that the kth population category z t After that, at this time, not only the physiological signal samples of the current subjects need to be considered, but also the knowledge parameters of the corresponding population types learned in the past learning process need to be applied, that is, the common emotional expression patterns of the corresponding population types. Therefore, the meta-learning parameters of the corresponding population types The distribution is expressed as:

[0162]

[0163] Assume that each population type is subject to a parameter Gaussian distribution, without loss of generality, after the arrival of the tth subject, the new The prior knowledge of The a posteriori knowledge of After the current subject's data training is completed, the new posterior knowledge is replaced to accept the new subject emotion classification task, and the parameters are updated by stochastic gradient descent;

[0164] (4)φ t The variational distribution of , that is, the generated emotion classifier with initialization parameters, is In, φ t represents the neural network parameters of the emotion classifier generated by the present invention for the current subject, λ t represents the variational parameters of the sample. In order to reduce the training pressure of the neural network, the present invention uses the idea of ​​amortized variational reasoning to calculate the variational parameters of the population category to which the current subject has been determined. Training data for and current subject Perform several steps of gradient descent, namely:

[0165]

[0166] In the formula, J is the step of gradient descent, ∈ is the learning rate, and the gradient descent strategy uses stochastic gradient descent;

[0167] Step S4: performance testing and commissioning;

[0168] When a subject's physiological signal sample set with only a small number of sample labels is passed into the method, the small sample set is recorded as At this time, by calling the neural network trained in the previous step, the subject can be accurately and dynamically divided into the corresponding population, and by calling the meta-learning framework that has been trained in the corresponding population, an emotion classifier with initialization parameters can be generated for the subject. The physiological signal can be used as input in an end-to-end manner to output the emotion classification result, and the emotion classifier can be quickly trained using a small number of corresponding samples and put into use or testing. At this time, the test steps become that when one or a batch of physiological signal samples without emotion annotations are passed in, the corresponding physiological signal samples are input into the corresponding emotion classifier, and the emotion prediction output of the meta-parameter model for the physiological signal sample can be obtained.

[0169] The above is a preferred embodiment of the present invention, but the present invention is not limited to the above embodiments. It should be pointed out that other improvements and changes directly derived or associated by technicians in this technical field without departing from the spirit and concept of the present application should be considered to be included in the scope of protection of the present invention.

Claims

1. An emotion recognition method based on dynamic clustering and variational Bayesian meta-learning, characterized in that: The steps include: Step S1: data sorting and preprocessing; Step S1-1: sorting the existing EEG signals of the subjects, uniformly downsampling the EEG signals to 128 Hz, filtering the EEG signals using a 5 to 45 Hz bandpass filter, and obtaining the physiological signal feature data called by the method; Step S1-2: The subject number is t, and the set of physiological signal characteristic data sequences obtained from the subject is recorded as: The subject's true emotion label set is recorded as: make X t ∈R C×N Where, X t represents the multi-channel physiological signal obtained from the subject in a single emotional stimulation experiment, C represents the number of characteristic channels of the physiological signal, N represents the product of the duration and frequency of the physiological signal, that is, the number of time points at which the physiological signal is finally obtained, R represents a real number set, Nt represents the number of physiological signal samples with emotion labels obtained from the tth subject, and the corresponding emotion label Y t It is given in the form of one-hot encoding. Taking the dimension of like degree in emotion annotation of DEAP dataset as an example, the subject's emotions are divided into two categories: like and dislike, which are represented by vectors [0,1] and [1,0] respectively. Step S1-3: The sample set of the subject Split into support set samples for training the subject's own emotion classifier and a sample queryset for testing The basis for splitting is to randomly shuffle and then split according to the proportion; Step S2: Initialization of parameters and meta-parameters of the crowd dynamic clustering process in the model; Step S2-1: Initialize the dynamic clustering parameters of the Chinese restaurant process CRP, set a hyperparameter α to determine the population category of the newly entered i-th subject, and use z i To represent the label of its category, the formula is as follows: Where n k represents the number of subjects expected to be divided into the kth population, and α>0 is used as a concentration hyperparameter to control the number of population types; Step S2-2: The probability of classifying a subject into a certain population category needs to be completed by constructing a broken stick representation for each population category. According to the principle of the Dirichlet process, the formula for the initialization process is: b k ∣α~Beta(1,α) (2) In the formula, π k Represents a probability value vector, π = [π1, π2, ...], where the value of each element represents the probability of classifying the subject into different population categories. It is necessary to ensure that the value of each element in the vector is non-negative and the sum of all values ​​is 1. The probability of the kth category is expressed as π k ; Step S2-3: Traditional meta-learning assumes that there is a meta-model with a parameter θ, and uses the meta-parameters in the meta-model to generate a corresponding emotion classifier. The generation process is: z t ∣π~Categorical ∞ (π), (4) In the formula, represents a mixed Gaussian distribution, and Categorical(π) represents sampling from each component of the probability value vector π according to the corresponding probability value to obtain a certain z t = k, that is, the current subject t is clearly divided into the corresponding population category, and the meta-parameter distribution of the population category to which the subject belongs is determined; z t Indicates the population category to which the current subject belongs. It means sampling a corresponding meta-parameter framework from the classifier parameter distribution of the corresponding population. It indicates that it obeys the mean vector of Gaussian mixture distribution, represents the semi-positive covariance matrix; Generate a corresponding emotion classifier sub-model with initialization parameters for the current subject. The specific process is as follows: Step S3: Online training and parameter update of algorithm model; Step S3-1: By means of structured inference, when the sample data of the t-th subject is input, the distribution parameters obeyed by the four parameters are outputted by means of posterior method, and β in formula 2 is replaced by k The parameter of the Beta distribution is denoted by γ k , replace z in Formula 4 t The parameter of the Categorical distribution is denoted as η t , replace θ in Formula 5 t The parameter of the Gaussian mixture distribution is denoted by ψ t , replace the task-specific parameter φ in Formula 6 t The variational parameter of t , the model parameters can be continuously updated or put into use online. The specific formula is: When the tth subject arrives, the observed evidence lower bound ELOB is further described according to Formula 8: In the formula, represents mathematical expectation, KL[·|·] represents Kullback–Leibler divergence. By maximizing this loss function, the goal of optimizing model parameters can be achieved. q represents the distribution predicted by the model, and p represents the actual distribution. Step S3-2: γ k , η t , t , t Generation and optimization process of four distribution parameters: (1) Variational distribution of β. Considering only the items related to β in ELOB, its optimal variational distribution is: q(β k ;c k )=Beta(β k ;c k,1 ,c k,2 ) That is, a Beta distribution with parameters, where γ k,1 =1+η t,k , (2) z t The variational distribution of t The related items derive their optimal variational distribution as: q(z t ;η t )=Categorical K (z t ;η t ) That is, a classifier distribution with parameters: In the formula, is the variational distribution of β; (3)θ t When the tth subject arrives, it is inferred that the kth population category z t Afterwards, the meta-learning parameters corresponding to the population The distribution is expressed as: Assume that each population type is subject to a parameter: Gaussian distribution, without loss of generality, after the arrival of the tth subject, the new The prior knowledge of The a posteriori knowledge of is given, namely: After the current subject's data training is completed, the new posterior knowledge is replaced to accept the new subject emotion classification task, and the parameters are updated by stochastic gradient descent; (4)φ t The variational distribution of , that is, the generated emotion classifier with initialization parameters, is formulated as: In the formula, φ t represents the neural network parameters of the emotion classifier generated by this method for the current subject, λ t Represents the variational parameters of the sampling, the variational parameters of the population category to which the current subject has been determined Training data for and current subject Perform several steps of gradient descent, namely: In the formula, J is the step of gradient descent, ∈ is the learning rate, and the gradient descent strategy uses stochastic gradient descent; Step S4: performance testing and commissioning; When a subject's physiological signal sample set has only a few sample labels After being introduced into this method, the neural network trained in step S3 is called to accurately and dynamically divide the subjects into corresponding groups, and the meta-learning framework that has been trained in the corresponding group is called to generate an emotion classifier with initialization parameters for the subject. The physiological signal is used as input in an end-to-end manner, and the emotion classification result is output. The emotion classifier is quickly trained using a small number of corresponding samples and can be put into use or tested. At this time, the test step is that when one or a batch of physiological signal samples without emotion annotations are passed in, the corresponding physiological signal samples are input into the corresponding emotion classifier, and the emotion prediction output of the meta-parameter model for the physiological signal sample can be obtained.

2. The emotion recognition method based on dynamic clustering and variational Bayesian meta-learning according to claim 1, characterized in that: The emotion classifier sub-model is a replaceable component, and in this method a module based on a multi-layer convolutional neural network is selected.

3. The emotion recognition method based on dynamic clustering and variational Bayesian meta-learning according to claim 2, characterized in that: In real scenarios, any component can be replaced as long as it is a trainable parameter-containing network that can encode the original multi-channel physiological signal into an embedded vector through a parameterized neural network.

4. The emotion recognition method based on dynamic clustering and variational Bayesian meta-learning according to claim 1, characterized in that: The split ratio in step S1-3 is 60% and 40%.

5. An emotion recognition system based on dynamic clustering and variational Bayesian meta-learning, characterized in that: include: Data sorting and data preprocessing module, dynamic clustering parameter module based on Chinese restaurant process CRP, meta-parameter module, model parameter update module, model commissioning module: Step M1: The implementation of the data sorting and preprocessing module specifically includes: Step M1-1: sort out the existing EEG signals of the subjects, uniformly downsample the EEG signals to 128 Hz, and filter the EEG signals using a 5 to 45 Hz bandpass filter to obtain the physiological signal feature data called by the method; Step M1-2: The subject number is t, and the set of physiological signal characteristic data sequences obtained from the subject is recorded as: The subject's true emotion label set is recorded as: make X t ∈R C×N Where, X t represents the multi-channel physiological signals collected from the subjects in a single emotional stimulation experiment, C represents the number of characteristic channels of the EEG signal, N represents the product of the duration and frequency of the physiological signal, that is, the number of time points at which the physiological signal is finally obtained, R represents a real number set, Nt represents the number of physiological signal samples with emotion labels obtained from the tth subject, and the corresponding emotion label Y t It is given in the form of one-hot encoding. Taking the dimension of the degree of sentiment annotation of the DEAP dataset as an example, it is divided into two emotions: like and dislike, and represented by vectors [0,1] and [1,0] respectively; Step M1-3: The sample set of the subject Split into support set samples for training the subject's own emotion classifier and a sample queryset for testing The basis for splitting is to randomly shuffle and split according to the ratio of 60% and 40% respectively; Step M2: Dynamic clustering parameter module based on Chinese restaurant process CRP; Step M2-1: The Chinese restaurant process controls the category judgment of the newly entered i-th subject by setting a hyperparameter α and uses z i To represent its category label, the formula is as follows: Where n k represents the number of subjects expected to be divided into the kth population, and α>0 is used as a concentration hyperparameter to control the number of population types; Step M2-2: The probability of classifying a subject into a certain population category needs to be completed by constructing a broken stick representation for each population category. According to the principle of the Dirichlet process, the specific formula is: b k ∣α~Beta(1,α), (2) In the formula, π k Represents a probability value vector, π = [π1, π2, ...], where the value of each element represents the probability of classifying the subject into different population categories. It is necessary to ensure that the value of each element in the vector is non-negative and the sum of all values ​​is 1. The probability of the kth category is expressed as π k ; Step M3: The specific implementation of the meta-parameter module is as follows: Traditional meta-learning assumes that there is a meta-model with a parameter θ, and uses the meta-parameter module in the meta-model to generate a corresponding emotion classifier. The generation process is: z t ∣π~Categorical ∞ (π), (4) In the formula, represents a mixed Gaussian distribution, and Categorical(π) represents sampling from each component of the probability value vector π according to the corresponding probability value to obtain a certain z t = k, that is, the current subject t is clearly divided into the corresponding population category, and the meta-parameter distribution of the population category to which the subject belongs is determined. After sampling, the meta-parameters of the corresponding population category are obtained. At this time, it is determined that the population category to which the current subject belongs is z t , and sample a corresponding meta-parameter framework from the classifier parameter distribution of the corresponding population represents the mean vector, represents the semi-positive covariance matrix; Generate a corresponding emotion classifier sub-model with initialization parameters for the current subject. The specific process is as follows: Step M4: model parameter updating module; Step M4-1: This method uses structured inference to input the sample data of the tth subject and output the distribution parameters obeyed by the four parameters. k The parameter of the Beta distribution is denoted by γ k , replace z in Formula 4 t The parameter of the Categorical distribution is denoted as η t , replace θ in Formula 5 t The parameter of the Gaussian mixture distribution is denoted by ψ t , replace the task-specific parameter φ in Formula 6 t The variational parameter of t , the model parameters can be continuously updated or put into use online. The specific formula is: When the tth subject arrives, the observed evidence lower bound ELOB is further described according to Formula 8: In the formula, represents mathematical expectation, KL[·|·] represents Kullback–Leibler divergence. The goal of optimizing model parameters can be achieved by maximizing this loss function. p represents the distribution predicted by the model, and q represents the actual distribution. Step M4-2: γ k , η t , t , t Generation and optimization process of four distribution parameters: (1) Variational distribution of β. Considering only the items related to β in ELOB, the optimal variational distribution is: q(β k ;c k )=Beta(β k ;c k,1 ,c k,2 ) That is, a Beta distribution with parameters, where γ k,1 =1+η t,k , (2) z t The variational distribution of t The related items derive their optimal variational distribution as: q(z t ;η t )=Categorical K (z t ;η t ) That is, a classifier distribution with parameters: In the formula is the variational distribution of β; (3)θ t When the tth subject arrives, it is inferred that the kth population category z t Afterwards, the meta-learning parameters corresponding to the population The distribution is expressed as: Assume that each population type is subject to a parameter Gaussian distribution, without loss of generality, after the arrival of the tth subject, the new The prior knowledge of The a posteriori knowledge of is given, namely: After the current subject's data training is completed, the new posterior knowledge is replaced to accept the new subject emotion classification task, and the parameters are updated by stochastic gradient descent; (4)φ t The variational distribution of , that is, the generated emotion classifier with initialization parameters, is in the formula: In the formula, φ t represents the neural network parameters of the emotion classifier generated by this method for the current subject, λ t Represents the variational parameters of the sampling, the variational parameters of the population category to which the current subject has been determined Training data for and current subject Perform several steps of gradient descent, namely: In the formula, J is the step of gradient descent, ∈ is the learning rate, and the gradient descent strategy uses stochastic gradient descent; Step M5: Model put into use module; When a subject's physiological signal sample set has only a few sample labels After being introduced into this method, the neural network trained in step S3 is called to accurately and dynamically divide the subjects into corresponding groups, and the meta-learning framework that has been trained in the corresponding group is called to generate an emotion classifier with initialization parameters for the subject. The physiological signal is used as input in an end-to-end manner, and the emotion classification result is output. The emotion classifier is quickly trained using a small number of corresponding samples and can be put into use or tested. At this time, the test step is that when one or a batch of physiological signal samples without emotion annotations are passed in, the corresponding physiological signal samples are input into the corresponding emotion classifier, and the emotion prediction output of the meta-parameter model for the physiological signal sample can be obtained.

Images