Speech emotion recognition method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a voice emotion recognition method and device, electronic equipment and a storage medium, and relates to the technical field of data processing, and the method comprises the steps: obtaining target voice data and an emotion recognition model used for voice emotion recognition, and the emotion recognition model at least comprises a feature extraction model, a feature dimension reduction model and a classifier; inputting the target voice data into a feature extraction model for feature extraction to obtain voice features corresponding to the target voice data; inputting the voice features into a feature dimensionality reduction model for feature dimensionality reduction to obtain low-dimensional features corresponding to the voice features; and the low-dimensional features are input into the classifier for prediction to obtain the emotion category of the voice data, so that the voice emotion recognition accuracy is effectively improved by performing feature extraction, feature dimension reduction, classification and other processing on the voice data.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method for recognizing speech emotions, a device for recognizing speech emotions, an electronic device and a computer-readable storage medium. Background Art

[0002] With the development of human-computer interaction technology, emotion recognition has become increasingly important in various intelligent systems. As an important way to identify and understand human emotions, speech emotion recognition is widely used in customer service, emotion monitoring, smart assistants, interactive entertainment and other scenarios. However, existing speech emotion recognition technology faces many challenges in practical applications. First, it is difficult to obtain emotional speech data, and the number of samples is limited, which makes it difficult for the model to have sufficient generalization ability. Secondly, in the process of feature extraction, conventional neural network algorithms are prone to problems such as gradient disappearance and gradient explosion, which affects the effective extraction of features and the stability of the model. Thirdly, feature dimensionality reduction technology is difficult to retain important information while reducing the data dimension, resulting in a decrease in subsequent classification accuracy. Finally, when processing complex emotional data, traditional classifier algorithms find it difficult to effectively capture and utilize the inherent diversity and uncertainty of data, resulting in insufficient accuracy in emotion recognition. Summary of the invention

[0003] The embodiments of the present invention provide a method, device, electronic device and computer-readable storage medium for recognizing speech emotions, so as to solve or partially solve the problem of inaccurate speech emotion recognition.

[0004] The embodiment of the present invention discloses a method for recognizing speech emotion, comprising:

[0005] Obtain target speech data, an emotion recognition model for speech emotion recognition, and a data expansion model for data expansion, wherein the emotion recognition model at least includes a feature extraction model, a feature dimensionality reduction model, and a classifier, wherein the data expansion model is a model trained based on dynamic symmetry breaking theory, the feature extraction model is a model trained based on a neural network algorithm optimized by dynamic population evolution, the feature dimensionality reduction model is a model trained based on a feature dimensionality reduction model based on feature refinement, and the classifier is a classifier trained based on a random forest classifier using quantum entropy coding;

[0006] Inputting the target speech data into the feature extraction model to perform feature extraction, and obtaining speech features corresponding to the target speech data;

[0007] Inputting the speech feature into the feature dimensionality reduction model to perform feature dimensionality reduction, and obtaining a low-dimensional feature corresponding to the speech feature;

[0008] The low-dimensional features are input into the classifier for prediction to obtain the emotion category of the speech data.

[0009] The embodiment of the present invention also discloses a speech emotion recognition device, comprising:

[0010] A data acquisition module, used to acquire target speech data, an emotion recognition model for speech emotion recognition, and a data expansion model for data expansion, wherein the emotion recognition model at least includes a feature extraction model, a feature dimensionality reduction model, and a classifier, wherein the data expansion model is a model trained based on dynamic symmetry breaking theory, the feature extraction model is a model trained based on a neural network algorithm optimized by dynamic population evolution, the feature dimensionality reduction model is a model trained based on a feature dimensionality reduction model based on feature refinement, and the classifier is a classifier trained based on a random forest classifier using quantum entropy coding;

[0011] A feature extraction module, used for inputting the target speech data into the feature extraction model to perform feature extraction, and obtaining speech features corresponding to the target speech data;

[0012] A feature dimension reduction module, used for inputting the speech feature into the feature dimension reduction model to perform feature dimension reduction, and obtaining a low-dimensional feature corresponding to the speech feature;

[0013] The prediction module is used to input the low-dimensional features into the classifier for prediction to obtain the emotion category of the speech data.

[0014] The embodiment of the present invention further discloses an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0015] The memory is used to store computer programs;

[0016] The processor is used to implement the method described in the embodiment of the present invention when executing the program stored in the memory.

[0017] The embodiment of the present invention further discloses a computer-readable storage medium having instructions stored thereon, which, when executed by one or more processors, enables the processors to execute the method described in the embodiment of the present invention.

[0018] The embodiments of the present invention include the following advantages:

[0019] In an embodiment of the present invention, in the process of emotion recognition of speech, an emotion recognition model for speech emotion recognition is obtained, the emotion recognition model at least includes a feature extraction model, a feature dimensionality reduction model and a classifier, the feature extraction model is a model obtained by training a neural network algorithm based on dynamic population evolution optimization, the feature dimensionality reduction model is a model obtained by training a feature dimensionality reduction model based on feature refinement, and the classifier is a classifier obtained by training a random forest classifier based on quantum entropy coding. The target speech data is first input into the feature extraction model for feature extraction to obtain speech features corresponding to the target speech data, and then the speech features are input into the feature dimensionality reduction model for feature dimensionality reduction to obtain low-dimensional features corresponding to the speech features. Finally, the low-dimensional features are input into the classifier for prediction to obtain the emotion category of the speech data, thereby effectively improving the recognition accuracy of speech emotion by performing feature extraction, feature dimensionality reduction and classification on the speech data. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a flow chart of the steps of a method for recognizing speech emotion provided in an embodiment of the present invention;

[0021] Figure 2 is a schematic diagram of the training process of the neural network algorithm provided in an embodiment of the present invention;

[0022] Figure 3 Schematic diagram of the training process of the autoencoding neural network algorithm provided in an embodiment of the present invention;

[0023] Figure 4 It is a structural block diagram of a speech emotion recognition device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0025] As an example, for speech emotion recognition, during the recognition process, firstly, it is difficult to obtain emotional speech data, and the number of samples is limited, which makes it difficult for the model to have sufficient generalization ability. Secondly, during the feature extraction process, conventional neural network algorithms are prone to problems such as gradient vanishing and gradient explosion, which affect the effective extraction of features and the stability of the model. Thirdly, feature dimensionality reduction technology is difficult to retain important information while reducing the data dimension, resulting in a decrease in subsequent classification accuracy. Finally, when processing complex emotional data, traditional classifier algorithms find it difficult to effectively capture and utilize the inherent diversity and uncertainty of the data, resulting in insufficient accuracy in emotion recognition.

[0026] In this regard, in the present invention, the emotion recognition model is optimized, and by optimizing the training process of the model, the trained emotion recognition model can accurately identify the emotion category corresponding to the speech data. Specifically, in the recognition process, by obtaining the emotion recognition model for speech emotion recognition, the emotion recognition model at least includes a feature extraction model, a feature dimensionality reduction model and a classifier. The feature extraction model is a model trained by a neural network algorithm based on dynamic population evolution optimization, the feature dimensionality reduction model is a model trained by a feature dimensionality reduction model based on feature refinement, and the classifier is a classifier trained by a random forest classifier based on quantum entropy coding. The target speech data is first input into the feature extraction model for feature extraction to obtain the speech features corresponding to the target speech data, and then the speech features are input into the feature dimensionality reduction model for feature dimensionality reduction to obtain the low-dimensional features corresponding to the speech features. Finally, the low-dimensional features are input into the classifier for prediction to obtain the emotion category of the speech data, thereby effectively improving the recognition accuracy of speech emotion by performing feature extraction, feature dimensionality reduction and classification on the speech data.

[0027] Reference Figure 1 , shows a flow chart of the steps of a method for recognizing speech emotions provided in an embodiment of the present invention, which may specifically include the following steps:

[0028] Step 101, obtaining target speech data and an emotion recognition model for speech emotion recognition, wherein the emotion recognition model at least includes a feature extraction model, a feature dimensionality reduction model and a classifier, wherein the feature extraction model is a model trained by a neural network algorithm based on dynamic population evolution optimization, the feature dimensionality reduction model is a model trained by a feature dimensionality reduction model based on feature refinement, and the classifier is a classifier trained by a random forest classifier based on quantum entropy coding;

[0029] Step 102, inputting the target speech data into the feature extraction model to perform feature extraction, and obtaining speech features corresponding to the target speech data;

[0030] Step 103, inputting the speech feature into the feature dimensionality reduction model to perform feature dimensionality reduction, and obtaining a low-dimensional feature corresponding to the speech feature;

[0031] Step 104: input the low-dimensional features into the classifier for prediction to obtain the emotion category of the speech data.

[0032] In an embodiment of the present invention, the emotion recognition model may at least include a feature extraction model, a feature dimensionality reduction model and a classifier, wherein the feature extraction model is a model obtained by training a neural network algorithm based on dynamic population evolution optimization, the feature dimensionality reduction model is a model obtained by training a feature dimensionality reduction model based on feature refinement, and the classifier is a classifier obtained by training a random forest classifier based on quantum entropy coding.

[0033] After the corresponding emotion recognition model is obtained through training, the emotion data is input into the emotion recognition model, and the feature extraction model is used to extract the speech features corresponding to the target speech data, and then the feature dimension reduction model is used to reduce the feature dimension, and the low-dimensional features corresponding to the speech features are obtained, and then the classifier is used to predict and obtain the emotion category of the speech data, thereby effectively improving the recognition accuracy of speech emotion by performing feature extraction, feature dimension reduction and classification on the speech data. Among them, the emotion categories may include happiness, sadness, anger, surprise and neutrality, etc., and the present invention is not limited to this.

[0034] It should be noted that the training process of the emotion recognition model may include a data expansion process and a model training process, etc. The data expansion process may include expanding the training data through a data expansion model, and the model training process may include the training process of corresponding models such as a data expansion model, a feature extraction model, a feature dimensionality reduction model, and a classifier.

[0035] Before model training, data collection and labeling can be performed first. The training data can include a variety of public emotional speech databases and live speech data collected through legal channels, covering multiple languages, accents and emotion types. All collected data are stored in a lossless compression format and are accompanied by metadata tags, including but not limited to the recording environment, the speaker's age, gender, and accent information. The collected data is stored in a structured database, and each file is indexed by a unique identifier to facilitate subsequent data processing and access.

[0036] For speech emotion recognition, the emotion categories can be annotated as: happy, sad, angry, surprised, neutral, and each emotion category is carefully reviewed and annotated by emotion recognition experts to ensure the consistency and accuracy of the annotations. In an example, the annotated speech data can be shown in Table 1 below:

[0037]

[0038] Table 1

[0039] After the training data is determined, the collection, acquisition, labeling and preprocessing of the training data are very time-consuming and labor-intensive, and insufficient training samples can easily lead to poor generalization of the model, affecting the prediction accuracy of the model. In this regard, in an embodiment of the present invention, a generative adversarial network algorithm based on dynamic symmetry breaking theory can be used to generate samples, thereby achieving data expansion, thereby enhancing the richness of the data and the robustness of the model.

[0040] In addition, considering that speech data is often interfered by environmental noise in practical applications, in this regard, in an embodiment of the present invention, a time-frequency mask is also combined in the generative adversarial network framework to combat various noises in the data expansion stage, thereby enhancing the model's ability to process speech data in complex environments.

[0041] In some feasible implementations, a data augmentation model to be trained is obtained, the data augmentation model includes at least a generator and a discriminator, and a symmetry breaking excitation term is configured in the data augmentation model. Then, the first target speech data for training the data augmentation model is determined, and then the first network parameters corresponding to the generator and the second network parameters corresponding to the discriminator can be randomly generated. Then, the first target speech data, the first network parameters and the second network parameters are used to alternately train the generator and the discriminator to obtain a generator loss function corresponding to the generator and a discriminator loss function corresponding to the discriminator. Then, the generator loss function and the symmetry breaking excitation term are used to iterate the parameters of the generator until a preset stop iteration condition is met to obtain a target generator. Then, the discriminator loss function is used to iterate the parameters of the discriminator until a preset stop iteration condition is met to obtain a target discriminator. Finally, the target generator and the target discriminator are combined to obtain the data augmentation model.

[0042] In the specific implementation, for the generator G c and the discriminator D c Network parameters and is randomly initialized, the goal of the generator is to learn a mapping function The random noise vector z c Converted to the distribution of speech data, the discriminator aims to distinguish between generated speech data and real speech data, and the initialization method is expressed as:

[0043]

[0044] in, is the parameter of the generator; is the parameter of the discriminator; σ is the standard deviation of the initialization parameters; the init() function is responsible for initializing the network parameters according to the normal distribution; ← is the parameter update operation.

[0045] In adversarial training, the generator and discriminator are trained alternately. The discriminator updates its parameters to better identify generated data and real data; the generator updates its parameters to generate speech samples that are more consistent with the distribution of real data. In this way, the generator and discriminator promote each other in constant confrontation and improve the performance of the entire model.

[0046] During adversarial training, it is necessary to minimize the generator’s loss function L Gc and maximize the discriminator loss function Specifically, the generator loss function corresponding to the generator can be calculated by formula (1):

[0047]

[0048] The discriminator loss function corresponding to the discriminator is calculated by formula (2):

[0049]

[0050] in, is the generator loss function; is the discriminator loss function; z c From the prior distribution Random noise sampled from c is the real data; p data is the real data distribution; G c () function is a generator function; D c () function is the discriminator function; represents the expectation operation; ~ represents obedience to a specific distribution.

[0051] After the corresponding loss function is calculated through the above process, the symmetry breaking adjustment can be performed dynamically. In each training cycle, the symmetry breaking excitation term is dynamically adjusted according to the difference between the generated data and the real data, so that the generator can avoid the problem of mode collapse while maintaining data diversity. Specifically, the symmetry breaking excitation term can be calculated by formula (3):

[0052] Δ c =λ c ·(∥G c (z c )-x c ∥-γ c ) Formula (3)

[0053] Then, the parameters of the generator are updated by formula (4):

[0054]

[0055] Among them, Δc represents the symmetry breaking excitation term; Ω c is the gradient adjustment function; c is the symmetry breaking excitation adjustment coefficient; γ c is the preset symmetry breaking threshold; ∥G c (z c )-x c ∥ is the Euclidean distance between the generated data and the real data; ∥∥ is the L2 norm, which is calculated in the same way as the Euclidean distance; is the learning rate of the generative adversarial network; Express Optionally, λ c Set to 0.1, Set to 0.02, γ c Set to 0.5.

[0056] In some feasible implementations, during the training of the data augmentation model, an adaptive gradient modulation mechanism can be used to dynamically adjust the gradient update rate of the generator during the training process, so as to more finely capture and simulate complex speech emotion features. Specifically, during the training of the generator, the adjustment gradient of the generator is adjusted based on the gradient adjustment function through formula (5):

[0057] Ω c =exp(-κ c ·(1-Acc(G c (z c )))) Formula (5)

[0058] Among them, Acc(G c (z c )) represents the classification accuracy of the generated sample obtained by the discriminator, κ c is the parameter for adjusting sensitivity; Ω c Ensure that when the accuracy of the emotional expression of the generated sample reaches a preset threshold, the adjustment is stopped; when the accuracy is less than the preset threshold, the gradient adjustment is enhanced to quickly improve the generator performance. Optionally, κ c Can be set to 2.

[0059] As shown in the above embodiment, in order to enhance the model's ability to process voice data in complex environments, the time-frequency mask is also combined in the generative adversarial network framework to combat various noises in the data expansion stage. Specifically, the first voice data used to train the data expansion model can be obtained, and then the time-frequency mask and random noise vector for the first voice data are obtained, and then the first voice data is optimized according to the video mask and the random noise vector to obtain the first target voice data.

[0060] In a specific implementation, the time-frequency mask is used to optimize the speech data in each batch of generated data so that the speech data can still maintain a high quality in a noisy environment. The first speech data can be optimized by formula (6) and formula (7) to obtain the first target speech data:

[0061] M c =f c (G c (Z c )) Formula (6)

[0062] G′ c (z c )=M c ⊙G c (Z c ) Formula (7)

[0063] Among them, Z c is the random noise vector of the entire batch, M c is the time-frequency mask; f c () function is the mask generation function; ⊙ represents element-by-element multiplication; G′ c (Z c ) is the generated data after applying the mask.

[0064] Furthermore, the mask generation function depends not only on a single generated sample but also on the statistical characteristics of the entire data batch to improve the dynamic adaptability of the mask. Specifically, the time-frequency mask for the first speech data can also be calculated by formula (8):

[0065]

[0066] Among them, Z c is the random noise vector of the entire batch, the Var() function and the Mean() function calculate the variance and mean of the random noise vector respectively, and the Max() function is used to calculate the maximum value.

[0067] It should be noted that in the process of training the data expansion model, the above process can be repeated until the preset stop iteration condition is met, which means that the model training is completed, so that the generative adversarial network based on the dynamic symmetry breaking theory performs data expansion, and by adopting the symmetry breaking excitation term, the mode collapse problem that occurs in the sample generation of the traditional generative adversarial network is solved, thereby enhancing the diversity and richness of the generated samples. In one embodiment, the preset stop iteration condition is to reach the preset maximum number of iterations. For example, the preset maximum number of iterations can be set to 1000 times.

[0068] For the feature extraction model, after the training data is expanded by the data expansion model, the expanded training data can be input into the feature extraction model to train the feature extraction model. Optionally, in an embodiment of the present invention, a 6-layer fully connected neural network is used for feature extraction, while in related technologies, some schemes use neural networks for feature extraction. In some neural network structures, problems such as gradient disappearance, gradient explosion, or falling into local optimal solutions may be encountered, affecting the stability of training and the performance of the model. In this regard, in an embodiment of the present invention, a neural network algorithm based on dynamic population evolution optimization is used as a feature extraction model. In traditional population evolution algorithms, the evolution of all individuals is based on fixed rules. The present invention uses a self-correction mechanism to automatically adjust the evolution rules according to the characteristics of the current training data, and adjusts the probability of crossover and mutation according to the changing trend of the loss function, so that the algorithm can adapt to different data distributions more flexibly and improve the generalization ability and training efficiency of the model.

[0069] In some feasible implementations, the second target speech data for training the feature extraction model can be determined first. The second target speech data can be speech data expanded based on the data expansion model. Then, according to the bionic algorithm initialization method, an initial population can be generated in the initialization stage. The population includes several individuals, each individual represents a configuration of network weights in a feature extraction model, and the corresponding weights and biases of the individuals are obtained. The composite loss function of the individuals in the second target speech data is calculated based on the weights and biases. Then, the fitness of the individuals is obtained, and the target individuals are selected from the individuals based on the fitness. The target individuals are cross-operated and mutated to generate new individuals. Then, the feature extraction model is iterated according to the composite loss function and the new individuals until the preset stop iteration condition is met to obtain the trained feature extraction model.

[0070] In the specific implementation, according to the bionic algorithm initialization method, in the initialization stage, an initial population is generated, each individual represents a configuration of network weights, and the population size is represented as N p , initialize the weights and biases for the i-th individual, expressed as:

[0071]

[0072] Among them, W pi is the weight of the neural network corresponding to the i-th individual, b pi is the bias of the neural network corresponding to the i-th individual, Represents the weight matrix of the i-th individual in the initial state; represents the bias of the i-th individual in the initial state; σ 2 represents the initialization variance; Indicates that the mean is 0 and the variance is σ2 Normal distribution of is a normal distribution. Optionally, σ 2 Can be set to 0.01.

[0073] For each individual in the population, its corresponding neural network configuration can be used to process the input training data, calculate the output of the model, and evaluate its performance according to a predetermined loss function. Specifically, the loss function of the extraction model on the training set can be calculated for the i-th individual according to the weight and bias corresponding to the individual through formula (9):

[0074]

[0075] Among them, L pi is the loss of the neural network corresponding to the i-th individual; mps represents the number of samples input in the current batch; the l() function represents the composite loss function, and the fsig() function represents the neural network model function; Represents the characteristics of the jth sample; represents the label of the jth sample.

[0076] In an optional example, the composite loss function may include a regularization term, and the generalization ability of the model may be increased by the regularization term. Specifically, the regularization term is calculated by formula (10):

[0077]

[0078] Among them, W pi,k is the kth weight of the neural network corresponding to the i-th individual;

[0079] The composite loss function is calculated by formula (11):

[0080]

[0081] Among them, the MSE() function is the mean square error function, Reg(W pi ) is the regularization term, λ ps is the regularization parameter.

[0082] After the composite loss function is determined, the best performing individuals can be selected from the current population according to their fitness and retained as candidate solutions for the next generation. Specifically, based on the fitness of the individuals, excellent individuals have a higher probability of being selected. The probability of the individuals being selected can be calculated by formula (12), and the individuals with a probability greater than or equal to a preset threshold are taken as target individuals:

[0083]

[0084] Among them, Pselect (i) represents the probability of the i-th individual being selected; γ pse is the parameter that controls the selection pressure; L pk is the loss of the neural network corresponding to the kth individual.

[0085] Furthermore, new individuals can be generated through crossover and mutation operations. The crossover operation allows two excellent individuals to exchange some genes to generate new offspring; the mutation operation randomly changes some genes in an individual to increase the diversity of the population. Specifically, the crossover operation randomly selects two individuals for gene exchange, which is expressed as:

[0086] W′ pi =α pcs W p1 +(1-α pcs )W p2

[0087] b′ pi =α pcs b p1 +(1-α pcs )b p2

[0088] Among them, α pcs is the crossover rate, W p1 is the weight of the neural network corresponding to the first individual selected, b p1 is the bias of the neural network corresponding to the first individual selected, W p2 is the weight of the neural network corresponding to the second individual selected, b p2 is the bias of the neural network corresponding to the second individual selected, W′ pi is the weight of the neural network corresponding to the individual after the crossover operation, b′ pi is the bias of the neural network corresponding to the individual after the crossover operation; and the mutation operation performs a small random perturbation on the newly generated individual weights, expressed as:

[0089]

[0090] Among them, τ 2 represents the variance of variation, W″ pi is the weight of the neural network corresponding to the individual after the mutation operation, b″ pi is the bias of the neural network corresponding to the individual after the mutation operation.

[0091] It should be noted that the above steps are repeated until the preset stop iteration condition is met, which means that the model training is completed. In one embodiment, the preset stop iteration condition is to reach the preset maximum number of iterations, so that in the training process of the feature extraction model, the feature extraction model based on the dynamic population evolution optimization is used for feature extraction, and the evolution rules are automatically adjusted according to the characteristics of the training data through the self-correction mechanism, thereby improving the adaptability of the model to different data distributions and significantly enhancing the stability and efficiency of feature extraction. Optionally, the preset maximum number of iterations is set to 1000 times.

[0092] After the feature extraction model obtained through the above training extracts the corresponding feature data, the extracted feature data can be input into the feature dimensionality reduction model to train the feature dimensionality reduction model. Specifically, the present invention adopts an autoencoding neural network based on feature refinement as a dimensionality reduction model. The autoencoding neural network based on feature refinement consists of three parts: an encoder, a decoder, and a feature adjustment unit. The encoder is responsible for mapping high-dimensional input data to a low-dimensional feature space, and the decoder is used to reconstruct the reduced-dimensional features back to the original space to ensure the reversibility of the dimensionality reduction process. The feature adjustment unit dynamically adjusts the reduced-dimensional feature space through recursive feature adaptive optimization, so that important features can be strengthened and secondary features are gradually weakened, so that the reduced-dimensional feature representation has better simplicity while retaining data information.

[0093] In some feasible implementations, the first feature data for training the feature dimensionality reduction model can be determined first, and then the first feature data is input into the encoder for mapping to obtain the corresponding initial low-dimensional features, and then the initial low-dimensional features are input into the feature adjustment unit for feature adjustment, and the feature weights corresponding to the low-dimensional features are determined first, and the initial low-dimensional features are adjusted to the target low-dimensional features corresponding to the feature weights, and then the target low-dimensional features are recursively optimized. In each round of iteration, the feature weights of each low-dimensional feature are adjusted according to the performance of each low-dimensional feature in the previous round to obtain the target feature weights corresponding to each low-dimensional feature, and then the target low-dimensional features and the corresponding target feature weights are input into the decoder for reconstruction to obtain the second feature data corresponding to the target low-dimensional feature, and finally the first feature data and the second feature data are matched, and the feature dimensionality reduction model is iterated based on the matching results until the preset stop iteration condition is met to obtain the trained feature dimensionality reduction model.

[0094] In the specific implementation, it can be assumed that the data input to the autoencoder neural network is X r , the encoder adopts a multi-layer nonlinear mapping structure to map the high-dimensional data to the initial low-dimensional feature space. Then, the first feature data can be mapped to the initial low-dimensional feature space to obtain the corresponding low-dimensional features through formula (13):

[0095] Zr =Sig enc (W r X r +b r ) Formula (13)

[0096] Among them, Z r represents the initial low-dimensional features, W r is the weight matrix of the encoder, b r is the encoder bias vector, Sig enc () function is the multi-layer Sigmoid activation function of the encoder.

[0097] After the low-dimensional features are generated, the feature adjustment module automatically generates feature weights according to the feature importance in the current feature space. The module is initialized to assign the same initial weights to all features so that they can be gradually adjusted according to feature contributions in subsequent steps. Specifically, the initial weight matrix for the low-dimensional features can be obtained; then the low-dimensional features are input into the feature adjustment unit, and the initial low-dimensional features are adjusted by the initial weight matrix to obtain the corresponding target low-dimensional features.

[0098] Wherein, the initial weight matrix is:

[0099] A r =diag(α r )

[0100] Among them, A r is the initial weight matrix; α r is the feature weight vector, α r Each element α in r,i Initialized to the same value, indicating that all features have the same importance in the initial stage; diag() is a function for extracting the diagonal elements of the matrix;

[0101] Among them, the target low-dimensional feature is expressed as:

[0102] Z′ r =A r Z r

[0103] Among them, Z′ r is the target low-dimensional feature after feature weight adjustment.

[0104] After the target low-dimensional features are input into the feature adjustment unit, the feature adjustment module recursively optimizes the initially generated low-dimensional features. In each round of iteration, the module adjusts the weights of each feature according to the performance of the previous round of features, gradually enhances those features with important influences, and gradually weakens redundant or noise features. The target low-dimensional features are recursively optimized. In the tth round of iteration, the feature weights are updated by formula (14):

[0105]

[0106] in, represents the feature weight of the t+1th iteration, represents the feature weight of the tth iteration, η r is the learning rate of the feature dimensionality reduction model, L r () function is the loss function of the feature dimensionality reduction model, Y r is the label data, The function represents the gradient of the loss function with respect to the feature weight;

[0107] Among them, the loss function of the feature dimensionality reduction model can be a reconstruction error loss function.

[0108] In order to ensure the effectiveness of the dimensionality reduction process, the decoder remaps the low-dimensional features back to the high-dimensional space to ensure that no important information is lost in the dimensionality reduction process. The target low-dimensional features can be input into the decoder for reconstruction. Specifically, the weight matrix and bias vector corresponding to the decoder can be obtained, and the target low-dimensional features, the weight matrix and the bias vector can be processed according to the multi-layer activation function configured in the decoder to construct the second feature data corresponding to the target low-dimensional features.

[0109] For example, the reconstruction process of the decoder is expressed as:

[0110] X′ r =Sig dec (W′ r Z′ r +b′ r )

[0111] Among them, X′ r is the reconstructed high-dimensional data, W′ r is the weight matrix of the decoder, b′ r is the bias vector of the decoder, Sig dec () function is the multi-layer Sigmoid activation function of the decoder.

[0112] It should be noted that in the process of training the feature dimensionality reduction model, the above steps can be repeated until the preset stop iteration condition is met, which means that the model training is completed. Therefore, in the process of training the feature dimensionality reduction model, the autoencoder neural network based on feature refinement is used as the feature dimensionality reduction model. Through recursive feature adaptive optimization, the importance of the features after dimensionality reduction is dynamically adjusted, and effective feature compression is achieved. While retaining important information, redundancy is reduced. In one embodiment, the preset stop iteration condition is to reach a preset maximum number of iterations. Optionally, the preset maximum number of iterations can be set to 1000 times.

[0113] After the training of the feature dimensionality reduction model is completed through the above process, the data after the dimensionality reduction of the feature dimensionality reduction model can be further input into the classifier to train the classifier model. Specifically, the present invention adopts a random forest based on quantum entropy coding as a classification algorithm, and uses a quantum entropy coding mechanism to optimize the selection process of tree nodes to significantly improve the accuracy and efficiency of classification. Quantum entropy coding is a technology that uses the probability amplitude of quantum bits to encode information. Through the quantum entropy coding mechanism, the present invention can more effectively capture and utilize the inherent uncertainty and diversity of data, thereby optimizing the construction process of the decision tree.

[0114] In some feasible implementations, the training process of the random forest algorithm based on quantum entropy coding may include: initializing a random forest classifier composed of multiple decision trees, the decision tree includes several nodes, and in the construction process of each decision tree, obtaining the quantum entropy corresponding to the node, and then using quantum entropy to calculate the information gain for the decision tree, and performing node segmentation according to the information gain, determining the result growth tree structure corresponding to the decision tree, and then comparing the result growth tree structure with the preset true structure, iterating the decision tree based on the comparison result until the preset stop iteration condition is met, and obtaining a trained decision tree, and finally integrating all trained decision trees to obtain the final random forest classifier.

[0115] In the specific implementation, we can first initialize the random forest composed of multiple decision trees, and the parameter initialization method is expressed as:

[0116]

[0117] in, Represents the parameters of the i-th tree; N T is the total number of trees; σ u is the initialization standard deviation; the init() function is used to initialize the parameters according to the Gaussian distribution. Optionally, N T Set to 100.

[0118] In the process of building each decision tree, the node selection is realized through the quantum entropy coding mechanism. By evaluating the quantum entropy change after each feature split, the split that maximizes the quantum entropy is selected to enhance the model's sensitivity to data and classification accuracy. Specifically, the quantum entropy corresponding to the node can be calculated by formula (14):

[0119]

[0120] Among them, S represents the data set of the current node; f is the considered segmentation feature; θ is the segmentation threshold; p k is the probability of the k-th label in S; K is the total number of categories; is the subset after segmentation; u is the balance coefficient, which is used to adjust the weight of the segmentation uniformity and information gain.

[0121] In one embodiment, p k The frequency of occurrence of each category and the statistical weight of the category at a specific node are taken into account, and the calculation method is expressed as:

[0122]

[0123] In the formula, w i is the weight of the ith data point, y i is the category of the i-th data point, [y i =k] is the indicator function, when y i The value is 1 when it is equal to k, otherwise it is 0.

[0124] Further, Considering the impact of unbalanced data, the correction coefficient μ is used j To adjust the influence of each subset, the correction coefficient can be calculated based on the diversity and distribution density of the subset, expressed as:

[0125]

[0126] In the formula, is the category k in the subset The relative frequency in Representation Subset The volume or coverage in the feature space; the Div() function is the diversity function; the Den() function is the density function.

[0127] After the corresponding quantum entropy is calculated through the above process, a tree structure can be grown according to the result of quantum entropy coding. The growth process of each tree is to recursively split the data set until the stop condition is met. The decision of node splitting is based on maximizing information gain. Specifically, the information gain for the decision tree can be calculated by formula (15):

[0128]

[0129] Among them, IG u is the information gain, which reflects the degree of reduction in entropy after using feature f to split the node.

[0130] Furthermore, through iterative training, the parameters and structure of each tree are continuously optimized so that the entire model gradually adapts to the characteristics of the training data. In each round of iteration, the quantum entropy coding strategy and game strategy are adjusted and optimized by evaluating the classification performance of the entire forest to achieve the best classification effect. Among them, the game theory strategy can be used to adjust the growth strategy between decision trees to optimize the performance of the entire forest. Specifically, in each round of iteration, the growth strategy between the decision trees is adjusted by formula (16):

[0131]

[0132] in, Represents the utility function of the i-th decision tree, which depends on its own parameters and the parameters θ of other decision trees -i Γ u Represents the optimization function used to maximize the synergy of the entire random forest.

[0133] Finally, through the ensemble learning strategy, all optimized tree models are integrated to form the final random forest classifier, and the decisions of all trees are integrated through the voting mechanism to determine the final classification result, which is expressed as:

[0134]

[0135] Among them, x is the input feature vector, specifically the feature vector after feature dimensionality reduction; is the prediction result of the i-th tree; the mode() function selects the most frequently occurring class label as the final prediction. Thus, in the process of training the classifier model, the random forest classifier based on quantum entropy coding optimizes the tree node selection process, takes quantum entropy maximization as the goal, improves the classification accuracy and efficiency, and realizes efficient classification of complex data.

[0136] After the corresponding emotion recognition model is trained through the above process, the trained model can be used to process new samples to achieve speech emotion recognition. In one embodiment, the collected raw data is input into the trained feature extraction and feature dimension reduction model for feature processing, and further, the processed features are input into the classifier model for classifier training to obtain the classification results. In this embodiment, the classification categories include: happy, sad, angry, surprised, and neutral. Among them, the processed emotion recognition results can be output to the application system or end user, and can be used in various application scenarios, such as customer service, emotion monitoring, interactive entertainment, etc.

[0137] It should be noted that the embodiments of the present invention include but are not limited to the above examples. It is understandable that those skilled in the art can also make settings according to actual needs under the guidance of the ideas of the embodiments of the present invention, and the present invention is not limited to this.

[0138] In an embodiment of the present invention, in the process of emotion recognition of speech, an emotion recognition model for speech emotion recognition is obtained, the emotion recognition model at least includes a feature extraction model, a feature dimensionality reduction model and a classifier, the feature extraction model is a model obtained by training a neural network algorithm based on dynamic population evolution optimization, the feature dimensionality reduction model is a model obtained by training a feature dimensionality reduction model based on feature refinement, and the classifier is a classifier obtained by training a random forest classifier based on quantum entropy coding. The target speech data is first input into the feature extraction model for feature extraction to obtain speech features corresponding to the target speech data, and then the speech features are input into the feature dimensionality reduction model for feature dimensionality reduction to obtain low-dimensional features corresponding to the speech features. Finally, the low-dimensional features are input into the classifier for prediction to obtain the emotion category of the speech data, thereby effectively improving the recognition accuracy of speech emotion by performing feature extraction, feature dimensionality reduction and classification on the speech data.

[0139] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the following examples are used for exemplary description:

[0140] Reference Figure 2, showing a schematic diagram of the training process of the neural network algorithm provided in an embodiment of the present invention, the feature extraction model can be a model built based on the neural network algorithm, and the corresponding training process can include: initializing the population - calculating the loss for each individual - evaluating the performance based on the loss - selecting the best performing individual - checking the operation to generate a new individual - mutating the operation to increase the diversity - checking the conditions for stopping the iteration - if the conditions are met, the model training is completed, if not, the loss is continued to be calculated for each individual to complete the model training, thereby performing feature extraction based on the neural network algorithm optimized by dynamic population evolution, automatically adjusting the evolution rules according to the characteristics of the training data through the self-correction mechanism, improving the adaptability of the model to different data distributions, and significantly enhancing the stability and efficiency of feature extraction.

[0141] Reference Figure 3 , showing a schematic diagram of the training process of the autoencoder neural network algorithm provided in an embodiment of the present invention, the feature dimensionality reduction model can be obtained by training based on the autoencoder neural network algorithm, and the corresponding training process may include: input data - encoder dimensionality reduction - generate low-dimensional features - feature adjustment module - feature initial weight - weight optimization - generate optimized features - decoder reconstructs high-dimensional features - calculates loss function - determines whether the iteration condition is satisfied - if satisfied, stops iteration, if not satisfied, continues weight optimization - ends training, so that the autoencoder neural network based on feature refinement is used as a feature dimensionality reduction model, and the importance of the features after dimensionality reduction is dynamically adjusted through recursive feature adaptive optimization, thereby achieving effective feature compression, retaining important information while reducing redundancy.

[0142] In addition, for other model training processes, you can refer to the description in the aforementioned embodiments and will not go into details here. Therefore, through the implementation of data augmentation technology, the number of samples is significantly increased, the generalization ability and robustness of the model are improved, and the emotion recognition effect in a variety of environments is improved; and, in the feature extraction process, by dynamically adjusting the feature weights, the representation ability of important features is improved, and the instability in the training process is reduced, thereby improving the overall recognition accuracy; and, in the feature dimensionality reduction stage, the dimension of the data is effectively reduced, making the subsequent classification process more efficient, while ensuring the integrity of important information and optimizing feature expression; finally, with the support of quantum entropy coding, the random forest classifier can better capture the inherent uncertainty of the data, improve the accuracy of classification decisions, and enhance the adaptability of the model in practical applications.

[0143] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0144] Reference Figure 4 , shows a structural block diagram of a speech emotion recognition device provided in an embodiment of the present invention, which may specifically include the following modules:

[0145] A data acquisition module 401 is used to acquire target speech data, an emotion recognition model for speech emotion recognition, and a data expansion model for data expansion, wherein the emotion recognition model at least includes a feature extraction model, a feature dimensionality reduction model, and a classifier, wherein the data expansion model is a model trained based on dynamic symmetry breaking theory, the feature extraction model is a model trained based on a neural network algorithm optimized by dynamic population evolution, the feature dimensionality reduction model is a model trained based on a feature dimensionality reduction model based on feature refinement, and the classifier is a classifier trained based on a random forest classifier using quantum entropy coding;

[0146] A data expansion module 402 is used to input the voice data into the data expansion model for data expansion to obtain target voice data;

[0147] A feature extraction module 403 is used to input the target speech data into the feature extraction model to perform feature extraction, and obtain speech features corresponding to the target speech data;

[0148] A feature dimension reduction module 404 is used to input the speech feature into the feature dimension reduction model to perform feature dimension reduction, and obtain a low-dimensional feature corresponding to the speech feature;

[0149] The prediction module 405 is used to input the low-dimensional features into the classifier for prediction to obtain the emotion category of the speech data.

[0150] In some feasible embodiments, the device further comprises:

[0151] A model acquisition module, used to acquire a data expansion model to be trained, wherein the data expansion model includes at least a generator and a discriminator, and a symmetry breaking excitation term is configured in the data expansion model;

[0152] A first data determination module, used to determine first target speech data for training the data augmentation model;

[0153] A first parameter generation module, used to randomly generate a first network parameter corresponding to the generator;

[0154] A second parameter generation module, used to randomly generate a second network parameter corresponding to the discriminator;

[0155] A loss function calculation module, used to alternately train the generator and the discriminator using the first target speech data, the first network parameters, and the second network parameters to obtain a generator loss function corresponding to the generator and a discriminator loss function corresponding to the discriminator;

[0156] A first iteration module, used to iterate the parameters of the generator using the generator loss function and the symmetry breaking excitation term until a preset stop iteration condition is met to obtain a target generator;

[0157] A first processing module, used to iterate the parameters of the discriminator using the discriminator loss function until a preset stop iteration condition is met, thereby obtaining a target discriminator;

[0158] The model determination module is used to combine the target generator and the target discriminator to obtain a data expansion model.

[0159] In some feasible embodiments, the loss function calculation module is specifically used to:

[0160] The generator loss function corresponding to the generator is calculated by formula (1):

[0161]

[0162] The discriminator loss function corresponding to the discriminator is calculated by formula (2):

[0163]

[0164] in, is the generator loss function; is the discriminator loss function; z c From the prior distribution Random noise sampled from c is the real data; p data is the real data distribution; G c () is the generator function; D c () is the discriminator function; represents the expectation operation; ~ represents obedience to a specific distribution.

[0165] In some feasible embodiments, the first iteration module is specifically used for:

[0166] The symmetry breaking excitation term is calculated by formula (3):

[0167]

[0168] The parameters of the generator are updated by formula (4):

[0169]

[0170] Among them, Δ c represents the symmetry breaking excitation term; Ω c is the gradient adjustment function; c is the symmetry breaking excitation adjustment coefficient; γ c is the preset symmetry breaking threshold; ∥G c (z c )-x c ∥ is the Euclidean distance between the generated data and the real data; ∥∥ is the L2 norm, which is calculated in the same way as the Euclidean distance; is the learning rate of the generative adversarial network; Express gradient.

[0171] In some possible embodiments, λ c Set to 0.1, Set to 0.02, γ c Set to 0.5.

[0172] In some feasible embodiments, it also includes:

[0173] A gradient adjustment module is used to adjust the adjustment gradient of the generator based on the gradient adjustment function through formula (5) during the training of the generator:

[0174] Ω c =exp(-κ c ·(1-Acc(G c (z c )))) Formula (5)

[0175] Among them, Acc(G c (z c )) represents the classification accuracy of the generated sample obtained by the discriminator, κ c is the parameter for adjusting sensitivity; Ω c Ensure that when the accuracy of the emotional expression of the generated sample reaches the preset threshold, the adjustment is stopped; when the accuracy is less than the preset threshold, the gradient adjustment is enhanced to quickly improve the generator performance.

[0176] In some feasible embodiments, the first data determination module is specifically used to:

[0177] Acquire first speech data for training the data augmentation model;

[0178] Acquire a time-frequency mask and a random noise vector for the first speech data;

[0179] The first speech data is optimized according to the video mask and the random noise vector to obtain first target speech data.

[0180] In some feasible embodiments, the first data determination module is specifically used to:

[0181] The first speech data is optimized by formula (6) and formula (7) to obtain the first target speech data:

[0182] M c =f c (G c (Z c )) Formula (6)

[0183] G′ c (z c )=M c ⊙G c (Z c ) Formula (7)

[0184] Among them, Z c is the random noise vector of the entire batch, M c is the time-frequency mask; f c () is the mask generation function; ⊙ represents element-by-element multiplication; G′ c (Z c ) is the generated data after applying the mask.

[0185] In some feasible embodiments, the first data determination module is specifically used to:

[0186] The time-frequency mask for the first speech data is calculated by formula (8):

[0187]

[0188] Among them, Z c is the random noise vector of the entire batch, Var() and Mean() calculate the variance and mean of the random noise vector respectively, and Max() calculates the maximum value.

[0189] In some feasible embodiments, it also includes:

[0190] A second data determination module, used to determine second target speech data for training the feature extraction model;

[0191] An initialization module, used to generate an initial population according to the bionic algorithm initialization method in the initialization stage, wherein the population includes a plurality of individuals, each of which represents a configuration of the network weights in the feature extraction model;

[0192] A calculation module, used to obtain the weight and bias corresponding to the individual, and calculate the composite loss function of the individual in the second target speech data according to the weight and the bias;

[0193] An individual selection module, used to obtain the fitness of the individuals and select a target individual from the individuals based on the fitness;

[0194] An individual processing module, used for performing crossover operation and mutation processing on the target individual to generate a new individual;

[0195] The second iteration module is used to iterate the feature extraction model according to the composite loss function and the new individual until a preset stop iteration condition is met to obtain a trained feature extraction model.

[0196] In some feasible embodiments, the initialization module is specifically used to:

[0197] Let the population size be N p , initialize the weights and biases for the i-th individual, expressed as:

[0198]

[0199] Among them, W pi is the weight of the neural network corresponding to the i-th individual, b pi is the bias of the neural network corresponding to the i-th individual, Represents the weight matrix of the i-th individual in the initial state; represents the bias of the i-th individual in the initial state; σ 2 represents the initialization variance; Indicates that the mean is 0 and the variance is σ 2 Normal distribution of is a normal distribution.

[0200] In some feasible embodiments, the computing module is specifically used for:

[0201] For the i-th individual, the loss function of the extraction model on the training set is calculated according to the weight and bias corresponding to the individual through formula (9):

[0202]

[0203] Among them, L piis the loss of the neural network corresponding to the i-th individual; mps represents the number of samples input in the current batch; l() represents the composite loss function, and fsig() represents the neural network model function; Represents the characteristics of the jth sample; represents the label of the jth sample.

[0204] In some feasible embodiments, the composite loss function includes a regularization term, and the regularization term is calculated by formula (10):

[0205]

[0206] Among them, W pi,k is the kth weight of the neural network corresponding to the i-th individual;

[0207] The composite loss function is calculated by formula (11):

[0208]

[0209] Among them, MSE() is the mean square error function, Reg(W pi ) is the regularization term, λ ps is the regularization parameter.

[0210] In some feasible embodiments, the individual selection module is specifically used to:

[0211] The probability of the individual being selected is calculated by formula (12), and the individual whose probability is greater than or equal to the preset threshold is taken as the target individual:

[0212]

[0213] Among them, P select (i) represents the probability of the i-th individual being selected; γ pse is the parameter that controls the selection pressure; L pk is the loss of the neural network corresponding to the kth individual.

[0214] In some feasible embodiments, the individual processing module is specifically used for:

[0215] The crossover operation randomly selects two individuals for gene exchange, which is expressed as:

[0216] W′ pi =α pcs W p1 +(1-α pcs )W p2

[0217] b′ pi =α pcs bp1 +(1-α pcs )b p2

[0218] Among them, α pcs is the crossover rate, W p1 is the weight of the neural network corresponding to the first individual selected, b p1 is the bias of the neural network corresponding to the first individual selected, W p2 is the weight of the neural network corresponding to the second individual selected, b p2 is the bias of the neural network corresponding to the second individual selected, W′ pi is the weight of the neural network corresponding to the individual after the crossover operation, b′ pi is the bias of the neural network corresponding to the individual after the crossover operation; and the mutation operation performs a small random perturbation on the newly generated individual weights, expressed as:

[0219]

[0220] Among them, τ 2 represents the variance of variation, W p ″ i is the weight of the neural network corresponding to the individual after the mutation operation, b″ pi is the bias of the neural network corresponding to the individual after the mutation operation.

[0221] In some feasible embodiments, the feature dimension reduction model includes at least an encoder, a decoder, and a feature adjustment unit, and the device further includes:

[0222] A third data determination module, used to determine first feature data for training the feature dimensionality reduction model;

[0223] A first mapping module, used for inputting the first feature data into the encoder for mapping to obtain corresponding initial low-dimensional features;

[0224] A feature adjustment module, used for inputting the initial low-dimensional feature into the feature adjustment unit for feature adjustment, first determining a feature weight corresponding to the low-dimensional feature, and adjusting the initial low-dimensional feature to a target low-dimensional feature corresponding to the feature weight;

[0225] An iteration module, used for recursively optimizing the target low-dimensional features, and in each round of iteration, adjusting the feature weights of the low-dimensional features according to the performance of the low-dimensional features in the previous round, to obtain the target feature weights corresponding to the low-dimensional features;

[0226] A reconstruction module, used for inputting the target low-dimensional feature and the corresponding target feature weight into the decoder for reconstruction, so as to obtain second feature data corresponding to the target low-dimensional feature;

[0227] The model building module is used to match the first feature data with the second feature data, and iterate the feature dimensionality reduction model based on the matching result until a preset stop iteration condition is met to obtain a trained feature dimensionality reduction model.

[0228] In some feasible embodiments, the first mapping module is specifically used for:

[0229] The first feature data is mapped to the initial low-dimensional feature space by formula (13) to obtain the corresponding low-dimensional features:

[0230] Z r =Sig enc (W r X r +b r ) Formula (13)

[0231] Among them, Z r represents the initial low-dimensional features, W r is the weight matrix of the encoder, b r is the encoder bias vector, Sig enc () is the multi-layer Sigmoid activation function of the encoder.

[0232] In some feasible embodiments, the feature adjustment module is specifically used to:

[0233] Obtaining an initial weight matrix for the low-dimensional features;

[0234] Inputting the low-dimensional features into the feature adjustment unit, adjusting the initial low-dimensional features by using the initial weight matrix, and obtaining corresponding target low-dimensional features;

[0235] Wherein, the initial weight matrix is:

[0236] A r =diag(α r )

[0237] Among them, A r is the initial weight matrix; α r is the feature weight vector, α r Each element α in r,i Initialized to the same value, indicating that all features have the same importance in the initial stage; diag() is a function for extracting the diagonal elements of the matrix;

[0238] Among them, the target low-dimensional feature is expressed as:

[0239] Z′ r =A r Z r

[0240] Among them, Z′ r is the target low-dimensional feature after feature weight adjustment.

[0241] In some feasible embodiments, the iteration module is specifically used for:

[0242] The target low-dimensional features are recursively optimized, and in the tth iteration, the feature weights are updated by formula (14):

[0243]

[0244] in, represents the feature weight of the t+1th iteration, represents the feature weight of the tth iteration, η r is the learning rate of the feature dimensionality reduction model, L r () is the loss function of the feature dimensionality reduction model, Y r is the label data, Represents the gradient of the loss function with respect to the feature weight;

[0245] Among them, the loss function of the feature dimensionality reduction model is a reconstruction error loss function.

[0246] In some feasible embodiments, the reconstruction module is specifically used to:

[0247] Obtaining a weight matrix and a bias vector corresponding to the decoder;

[0248] The target low-dimensional feature, the weight matrix, and the bias vector are processed according to the multi-layer activation function configured in the decoder to construct second feature data corresponding to the target low-dimensional feature.

[0249] In some feasible embodiments, it also includes:

[0250] A classifier initialization module, used to initialize a random forest classifier composed of multiple decision trees, wherein the decision tree includes a plurality of nodes;

[0251] A quantum entropy determination module, used to obtain the quantum entropy corresponding to the node during the construction process of each decision tree;

[0252] A segmentation module, used to calculate the information gain of the decision tree using the quantum entropy, and perform node segmentation according to the information gain to determine the result growth tree structure corresponding to the decision tree;

[0253] A decision tree iteration module, used to compare the resulting growth tree structure with a preset real structure, and iterate the decision tree based on the comparison result until a preset stop iteration condition is met to obtain a trained decision tree;

[0254] The integration module is used to integrate all trained decision trees to obtain the final random forest classifier.

[0255] In some feasible embodiments, the quantum entropy determination module is specifically used to:

[0256] The quantum entropy corresponding to the node is calculated by formula (14):

[0257]

[0258] Among them, S represents the data set of the current node; f is the considered segmentation feature; θ is the segmentation threshold; p k is the probability of the k-th label in S; K is the total number of categories; is the subset after segmentation; u is the balance coefficient, which is used to adjust the weight of the segmentation uniformity and information gain.

[0259] In some feasible embodiments, the segmentation module is specifically used to:

[0260] The information gain for the decision tree is calculated by formula (15):

[0261]

[0262] Among them, IG u is the information gain, which reflects the degree of reduction in entropy after using feature f to split the node.

[0263] In some feasible embodiments, the decision tree iteration module is specifically used for:

[0264] In each round of iteration, the growth strategy between the decision trees is adjusted by formula (16):

[0265]

[0266] in, Represents the utility function of the i-th decision tree, which depends on its own parameters and the parameters θ of other decision trees -i Γ u Represents the optimization function used to maximize the synergy of the entire random forest.

[0267] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0268] In addition, an embodiment of the present invention also provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the various processes of the above-mentioned speech emotion recognition method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0269] The embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, each process of the above-mentioned speech emotion recognition method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0270] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0271] It should be understood by those skilled in the art that the embodiments of the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, EEPROM, Flash, and eMMC, etc.) containing computer-usable program codes.

[0272] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.

[0273] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0274] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0275] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0276] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0277] The above is a detailed introduction to a speech emotion recognition method and a speech emotion recognition device provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for recognizing speech emotion, characterized in that: include: Acquire target speech data and an emotion recognition model for speech emotion recognition, wherein the emotion recognition model at least includes a feature extraction model, a feature dimensionality reduction model and a classifier, wherein the feature extraction model is a model trained by a neural network algorithm based on dynamic population evolution optimization, the feature dimensionality reduction model is a model trained by a feature dimensionality reduction model based on feature refinement, and the classifier is a classifier trained by a random forest classifier based on quantum entropy coding; Inputting the target speech data into the feature extraction model to perform feature extraction, and obtaining speech features corresponding to the target speech data; Inputting the speech feature into the feature dimensionality reduction model to perform feature dimensionality reduction, and obtaining a low-dimensional feature corresponding to the speech feature; The low-dimensional features are input into the classifier for prediction to obtain the emotion category of the speech data.

2. The method according to claim 1, characterized in that The method further comprises: Obtaining a data expansion model to be trained, wherein the data expansion model includes at least a generator and a discriminator, and a symmetry breaking excitation term is configured in the data expansion model; Determining first target speech data for training the data augmentation model; Randomly generate a first network parameter corresponding to the generator; Randomly generate a second network parameter corresponding to the discriminator; Using the first target speech data, the first network parameters, and the second network parameters to alternately train the generator and the discriminator to obtain a generator loss function corresponding to the generator and a discriminator loss function corresponding to the discriminator; Iterate the parameters of the generator using the generator loss function and the symmetry breaking excitation term until a preset stop iteration condition is met to obtain a target generator; Iterate the parameters of the discriminator using the discriminator loss function until a preset stop iteration condition is met to obtain a target discriminator; The target generator is combined with the target discriminator to obtain a data augmentation model.

3. The method according to claim 2, characterized in that The step of alternately training the generator and the discriminator using the first target speech data, the first network parameters, and the second network parameters to obtain a generator loss function corresponding to the generator and a discriminator loss function corresponding to the discriminator includes: The generator loss function corresponding to the generator is calculated by formula (1): The discriminator loss function corresponding to the discriminator is calculated by formula (2): in, is the generator loss function; is the discriminator loss function; z c From the prior distribution Random noise sampled from c is the real data; p data is the real data distribution; G c () is the generator function; D c () is the discriminator function; represents the expectation operation; ~ represents obedience to a specific distribution.

4. The method according to claim 2 or 3, characterized in that: The generator loss function and the symmetry breaking excitation term are used to iterate the parameters of the generator until a preset stop iteration condition is met to obtain a target generator, including: The symmetry breaking excitation term is calculated by formula (3): Δ c = λ c · (||G c (z c ) - x c || - γ c ) Equation (3) The parameters of the generator are updated by formula (4): Among them, Δ c represents the symmetry breaking excitation term; Ω c is the gradient adjustment function; c is the symmetry breaking excitation adjustment coefficient; γ c is the preset symmetry breaking threshold; ||G c (z c )-x c || is the Euclidean distance between the generated data and the real data; |||| is the L2 norm, calculated the same as the Euclidean distance; is the learning rate of the generative adversarial network; Express gradient.

5. The method according to claim 4, characterized in that λ c Set to 0.1, Set to 0.02, γ c Set to 0.

5.

6. The method according to claim 2 or 3, characterized in that: Also includes: In the process of training the generator, the adjustment gradient of the generator is adjusted based on the gradient adjustment function by formula (5): Ω c =exp(-κ c ·(1-Acc(G c (z c )))) Formula (5) Among them, Acc(G c (z c )) represents the classification accuracy of the generated sample obtained by the discriminator, κ c is the parameter for adjusting sensitivity; Ω c Ensure that when the accuracy of the emotional expression of the generated sample reaches the preset threshold, the adjustment is stopped; when the accuracy is less than the preset threshold, the gradient adjustment is enhanced to quickly improve the generator performance.

7. The method according to claim 2 or 3, characterized in that: The determining of first target speech data for training the data augmentation model comprises: Acquire first speech data for training the data augmentation model; Acquire a time-frequency mask and a random noise vector for the first speech data; The first speech data is optimized according to the video mask and the random noise vector to obtain first target speech data.

8. The method according to claim 7, characterized in that The step of optimizing the first speech data according to the video mask and the random noise vector to obtain first target speech data includes: The first speech data is optimized by formula (6) and formula (7) to obtain the first target speech data: M c =f c (G c (Z c )) Formula (6) G′ c (z c )=M c ⊙G c (Z c ) Formula (7) Among them, Z c is the random noise vector of the entire batch, M c is the time-frequency mask; f c () is the mask generation function; ⊙ represents element-by-element multiplication; G′ c (Z c ) is the generated data after applying the mask.

9. The method according to claim 8, characterized in that The acquiring of a time-frequency mask for the first speech data includes: The time-frequency mask for the first speech data is calculated by formula (8): Among them, Z c is the random noise vector of the entire batch, Var() and Mean() calculate the variance and mean of the random noise vector respectively, and Max() calculates the maximum value.

10. The method according to claim 1, characterized in that Also includes: Determining second target speech data for training the feature extraction model; According to the bionic algorithm initialization method, in the initialization stage, an initial population is generated, the population includes a number of individuals, each individual represents a configuration of the network weight in the feature extraction model; Obtaining a weight and a bias corresponding to the individual, and calculating a composite loss function of the individual in the second target speech data according to the weight and the bias; Acquiring the fitness of the individuals, and selecting a target individual from the individuals based on the fitness; Performing crossover and mutation processing on the target individual to generate a new individual; The feature extraction model is iterated according to the composite loss function and the new individual until a preset stop iteration condition is met, thereby obtaining a trained feature extraction model.

11. The method according to claim 10, characterized in that According to the bionic algorithm initialization method, in the initialization stage, an initial population is generated, including: Let the population size be N p , initialize the weights and biases for the i-th individual, expressed as: Among them, W pi is the weight of the neural network corresponding to the i-th individual, b pi is the bias of the neural network corresponding to the i-th individual, Represents the weight matrix of the i-th individual in the initial state; represents the bias of the i-th individual in the initial state; σ 2 represents the initialization variance; Indicates that the mean is 0 and the variance is σ 2 Normal distribution of is a normal distribution.

12. The method according to claim 10 or 11, characterized in that: The calculating, according to the weight and the bias, a composite loss function of the individual in the second target speech data comprises: For the i-th individual, the loss function of the extraction model on the training set is calculated according to the weight and bias corresponding to the individual through formula (9): Among them, L pi is the loss of the neural network corresponding to the i-th individual; mps represents the number of samples input in the current batch; represents the composite loss function, and fsig() represents the neural network model function; Represents the characteristics of the jth sample; represents the label of the jth sample.

13. The method according to claim 10, characterized in that The composite loss function includes a regularization term, which is calculated by formula (10): Among them, W pi,k is the kth weight of the neural network corresponding to the i-th individual; The composite loss function is calculated by formula (11): Among them, MSE() is the mean square error function, Reg(W pi ) is the regularization term, λ ps is the regularization parameter.

14. The method according to claim 10, characterized in that The step of selecting a target individual from the individuals based on the fitness comprises: The probability of the individual being selected is calculated by formula (12), and the individual whose probability is greater than or equal to the preset threshold is taken as the target individual: Among them, P select (i) represents the probability of the i-th individual being selected; γ pse is the parameter that controls the selection pressure; L pk is the loss of the neural network corresponding to the kth individual.

15. The method according to claim 10, characterized in that The performing a crossover operation and a mutation process on the target individual to generate a new individual includes: The crossover operation randomly selects two individuals for gene exchange, which is expressed as: W′ pi =a pcs W p1 +(1-a pcs )W p2 b′ pi =a pcs b p1 +(1-a pcs )b p2 Among them, α pcs is the crossover rate, W p1 is the weight of the neural network corresponding to the first individual selected, b p1 is the bias of the neural network corresponding to the first individual selected, W p2 is the weight of the neural network corresponding to the second individual selected, b p2 is the bias of the neural network corresponding to the second individual selected, W′ pi is the weight of the neural network corresponding to the individual after the crossover operation, b′ pi is the bias of the neural network corresponding to the individual after the crossover operation; and the mutation operation performs a small random perturbation on the newly generated individual weights, expressed as: Among them, τ 2 represents the variance of variation, W″ pi is the weight of the neural network corresponding to the individual after the mutation operation, b″ pi is the bias of the neural network corresponding to the individual after the mutation operation.

16. The method according to claim 1, characterized in that The feature dimension reduction model at least includes an encoder, a decoder and a feature adjustment unit, and the method further includes: Determining first feature data for training the feature dimensionality reduction model; Inputting the first feature data into the encoder for mapping to obtain corresponding initial low-dimensional features; Inputting the initial low-dimensional features into the feature adjustment unit for feature adjustment, first determining feature weights corresponding to the low-dimensional features, and adjusting the initial low-dimensional features to target low-dimensional features corresponding to the feature weights; Recursively optimizing the target low-dimensional features, in each round of iteration, adjusting the feature weights of the low-dimensional features according to the performance of the low-dimensional features in the previous round, and obtaining the target feature weights corresponding to the low-dimensional features; Inputting the target low-dimensional feature and the corresponding target feature weight into the decoder for reconstruction to obtain second feature data corresponding to the target low-dimensional feature; The first feature data and the second feature data are matched, and the feature dimensionality reduction model is iterated based on the matching result until a preset stop iteration condition is met, so as to obtain a trained feature dimensionality reduction model.

17. The method according to claim 16, characterized in that The step of inputting the first feature data into the encoder for mapping to obtain the corresponding initial low-dimensional features includes: The first feature data is mapped to the initial low-dimensional feature space by formula (13) to obtain the corresponding low-dimensional features: Z r =Sig enc (W r X r +b r ) Formula (13) Among them, Z r represents the initial low-dimensional features, W r is the weight matrix of the encoder, b r is the encoder bias vector, Sig enc () is the multi-layer Sigmoid activation function of the encoder.

18. The method according to claim 16 or 17, characterized in that The step of inputting the initial low-dimensional feature into the feature adjustment unit for feature adjustment, first determining a feature weight corresponding to the low-dimensional feature, and adjusting the initial low-dimensional feature to a target low-dimensional feature corresponding to the feature weight, comprises: Obtaining an initial weight matrix for the low-dimensional features; Inputting the low-dimensional features into the feature adjustment unit, adjusting the initial low-dimensional features by using the initial weight matrix, and obtaining corresponding target low-dimensional features; Wherein, the initial weight matrix is: A r =diag(a r ) Among them, A r is the initial weight matrix; α r is the feature weight vector, α r Each element α in r,i Initialized to the same value, indicating that all features have the same importance in the initial stage; diag() is a function for extracting the diagonal elements of the matrix; Among them, the target low-dimensional feature is expressed as: Z′ r =A r From r Among them, Z′ r is the target low-dimensional feature after feature weight adjustment.

19. The method according to claim 16 or 17, characterized in that The target low-dimensional features are recursively optimized, and in each round of iteration, the feature weights of the low-dimensional features are adjusted according to the performance of the low-dimensional features in the previous round to obtain the target feature weights corresponding to the low-dimensional features, including: The target low-dimensional features are recursively optimized, and in the tth iteration, the feature weights are updated by formula (14): in, represents the feature weight of the t+1th iteration, represents the feature weight of the tth iteration, η r is the learning rate of the feature dimensionality reduction model, L r () is the loss function of the feature dimensionality reduction model, Y r is the label data, Represents the gradient of the loss function with respect to the feature weight; Among them, the loss function of the feature dimensionality reduction model is a reconstruction error loss function.

20. The method according to claim 16 or 17, characterized in that The step of inputting the target low-dimensional feature and the corresponding target feature weight into the decoder for reconstruction to obtain second feature data corresponding to the target low-dimensional feature includes: Obtaining a weight matrix and a bias vector corresponding to the decoder; The target low-dimensional feature, the weight matrix, and the bias vector are processed according to the multi-layer activation function configured in the decoder to construct second feature data corresponding to the target low-dimensional feature.

21. The method according to claim 1, characterized in that Also includes: Initializing a random forest classifier consisting of multiple decision trees, wherein the decision tree includes a plurality of nodes; In the process of constructing each decision tree, obtaining the quantum entropy corresponding to the node; The information gain of the decision tree is calculated by using the quantum entropy, and nodes are segmented according to the information gain to determine a result growth tree structure corresponding to the decision tree; The resulting growth tree structure is compared with a preset real structure, and the decision tree is iterated based on the comparison result until a preset stop iteration condition is met to obtain a trained decision tree; All trained decision trees are integrated to obtain the final random forest classifier.

22. The method according to claim 21, characterized in that The obtaining of the quantum entropy corresponding to the node includes: The quantum entropy corresponding to the node is calculated by formula (14): Among them, S represents the data set of the current node; f is the considered segmentation feature; θ is the segmentation threshold; p k is the probability of the k-th label in S; K is the total number of categories; is the subset after segmentation; u is the balance coefficient, which is used to adjust the weight of the segmentation uniformity and information gain.

23. The method according to claim 21, characterized in that The using the quantum entropy to calculate the information gain for the decision tree includes: The information gain for the decision tree is calculated by formula (15): Among them, IG u is the information gain, which reflects the degree of reduction in entropy after using feature f to split the node.

24. The method according to claim 21, characterized in that The iterating the decision tree based on the comparison result comprises: In each round of iteration, the growth strategy between the decision trees is adjusted by formula (16): in, Represents the utility function of the i-th decision tree, which depends on its own parameters and the parameters θ of other decision trees -i Γ u Represents the optimization function used to maximize the synergy of the entire random forest.

25. A speech emotion recognition device, characterized in that: include: A data acquisition module, used to acquire target speech data, an emotion recognition model for speech emotion recognition, and a data expansion model for data expansion, wherein the emotion recognition model at least includes a feature extraction model, a feature dimensionality reduction model, and a classifier, wherein the data expansion model is a model trained based on dynamic symmetry breaking theory, the feature extraction model is a model trained based on a neural network algorithm optimized by dynamic population evolution, the feature dimensionality reduction model is a model trained based on a feature dimensionality reduction model based on feature refinement, and the classifier is a classifier trained based on a random forest classifier using quantum entropy coding; A feature extraction module, used for inputting the target speech data into the feature extraction model to perform feature extraction, and obtaining speech features corresponding to the target speech data; A feature dimension reduction module, used for inputting the speech feature into the feature dimension reduction model to perform feature dimension reduction, and obtaining a low-dimensional feature corresponding to the speech feature; The prediction module is used to input the low-dimensional features into the classifier for prediction to obtain the emotion category of the speech data.

26. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; The memory is used to store computer programs; The processor is used to implement the method according to any one of claims 1 to 24 when executing the program stored in the memory.

27. A computer-readable storage medium having instructions stored thereon, which, when executed by one or more processors, cause the processors to perform the method according to any one of claims 1 to 24.

Images