Speech emotion recognition model training method and device, computer equipment and medium

Abstract

The invention relates to the field of voice processing and financial science and technology, and particularly discloses a training method and device of a voice emotion recognition model, computer equipment and a medium. According to the invention, the feature learning module and the classifier learning module based on supervised comparative learning are trained at the same time through the sample audio features, and the feature learning module and the classifier learning module are subjected to weighted combination through the target weight coefficient. The method can dynamically adjust the attention to the feature learning module and the classifier learning module in the training process according to the target weight coefficient, enables the model to fully utilize the sample audio features, learns the feature representation with higher emotion discrimination, improves the model performance, and improves the accuracy of speech emotion recognition.

Description

Technical Field

[0001] The present application relates to the fields of speech processing and financial technology, and in particular to a training method, device, computer equipment and medium for a speech emotion recognition model. Background Art

[0002] With the development of computer technology, intelligent voice customer service systems are widely used in customer service in the financial field. Intelligent voice customer service systems automatically identify the speaker's emotions through voice emotion recognition technology, which can help business personnel better understand the user's voice content and emotional changes, and then provide more appropriate and personalized services based on the user's emotional state. For example, the intelligent voice customer service system records the communication between the business personnel and the user, and identifies the user's emotional state from the communication voice through voice emotion recognition technology, so as to better understand the user's needs and situation. However, most of the current voice emotion recognition is end-to-end deep neural networks, or first learns features and then uses frozen feature extractors to learn classifiers, ignoring the connection between the same emotion categories, and cannot fully utilize audio features for learning, resulting in low performance of the trained model and low recognition accuracy. Therefore, how to optimize the model training process, improve model performance, and then improve the accuracy of voice emotion recognition has become an urgent problem to be solved. Summary of the invention

[0003] The present application provides a method, apparatus, computer equipment and medium for training a speech emotion recognition model to optimize the model training process, improve model performance, and thereby improve the accuracy of speech emotion recognition.

[0004] In a first aspect, the present application provides a method for training a speech emotion recognition model, the method comprising:

[0005] Acquire an audio sample set, and perform feature extraction on audio samples in the audio sample set based on a preset feature extractor to obtain sample audio features;

[0006] Based on the feature learning module and the classifier learning module of the pre-trained model respectively, the sample audio features are processed to obtain a first loss function and a second loss function;

[0007] Performing a weighted combination of the first loss function and the second loss function according to a target weight coefficient to obtain a target loss function;

[0008] Based on the target loss function, the feature learning module and the classifier learning module are trained to obtain a speech emotion recognition model.

[0009] In a second aspect, the present application also provides a training device for a speech emotion recognition model, the device comprising:

[0010] A sample audio feature acquisition module, used to acquire an audio sample set, and perform feature extraction on the audio samples in the audio sample set based on a preset feature extractor to obtain sample audio features;

[0011] A sample audio feature processing module, used to process the sample audio features based on the feature learning module and the classifier learning module of the pre-trained model, respectively, to obtain a first loss function and a second loss function;

[0012] An objective loss function combination module, used for weighted combination of the first loss function and the second loss function according to an objective weight coefficient to obtain an objective loss function;

[0013] The emotion recognition model acquisition module is used to train the feature learning module and the classifier learning module based on the target loss function to obtain a speech emotion recognition model.

[0014] In a third aspect, the present application also provides a computer device, comprising a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and implement the training method of the speech emotion recognition model as described above when executing the computer program.

[0015] In a fourth aspect, the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor implements the training method of the speech emotion recognition model as described above.

[0016] The present application discloses a training method, device, computer equipment and medium for a speech emotion recognition model, which obtains an audio sample set, and extracts features from the audio samples in the audio sample set based on a preset feature extractor to obtain sample audio features; processes the sample audio features based on a feature learning module and a classifier learning module of a pre-trained model to obtain a first loss function and a second loss function; weights and combines the first loss function and the second loss function according to a target weight coefficient to obtain a target loss function; trains the feature learning module and the classifier learning module based on the target loss function to obtain a speech emotion recognition model. The present application simultaneously trains the feature learning module and the classifier learning module through sample audio features, and weights and combines the feature learning module and the classifier learning module through a target weight coefficient, and can dynamically adjust the attention paid to the feature learning module and the classifier learning module during the training process according to the target weight coefficient, so that the model can make full use of the sample audio features, learn a more emotionally discriminative feature representation, improve the model performance, and thus improve the accuracy of speech emotion recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 This is a schematic flow chart of a first embodiment of a method for training a speech emotion recognition model provided by an embodiment of the present application;

[0019] Figure 2 is a schematic flow chart of a second embodiment of a method for training a speech emotion recognition model provided in an embodiment of the present application;

[0020] Figure 3 is a schematic flow chart of a third embodiment of a method for training a speech emotion recognition model provided by an embodiment of the present application;

[0021] Figure 4 A schematic block diagram of a training device for a speech emotion recognition model provided in an embodiment of the present application;

[0022] Figure 5 A schematic block diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0024] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0025] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0026] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0027] The embodiments of the present application provide a method, device, computer equipment and medium for training a speech emotion recognition model. Among them, the training method of the speech emotion recognition model can be applied to a server, and the feature learning module and the classifier learning module are trained simultaneously through sample audio features, and the feature learning module and the classifier learning module are weighted and combined through the target weight coefficient. The attention paid to the feature learning module and the classifier learning module during the training process can be dynamically adjusted according to the target weight coefficient, so that the model can make full use of the sample audio features and learn feature representations with more emotional discrimination, thereby improving the model performance and further improving the accuracy of speech emotion recognition. Among them, the server can be an independent server or a server cluster.

[0028] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0029] See also Figure 1 , Figure 1 This is a schematic flow chart of a method for training a speech emotion recognition model provided by an embodiment of the present application. The method for training a speech emotion recognition model can be applied to a server, and is used to train a feature learning module and a classifier learning module simultaneously through sample audio features, and weightedly combine the feature learning module and the classifier learning module through a target weight coefficient, and can dynamically adjust the attention paid to the feature learning module and the classifier learning module during the training process according to the target weight coefficient, so that the model can make full use of the sample audio features, learn a more emotionally discriminative feature representation, improve the model performance, and thus improve the accuracy of speech emotion recognition.

[0030] like Figure 1 As shown, the training method of the speech emotion recognition model specifically includes steps S101 to S104.

[0031] S101, obtaining an audio sample set, and performing feature extraction on audio samples in the audio sample set based on a preset feature extractor to obtain sample audio features;

[0032] In one embodiment, the audio sample can be a public data set, self-recorded audio, or an audio file obtained from other legal channels. All audio files are formatted in a unified manner and organized into an audio sample set.

[0033] In one embodiment, the audio samples may also be preprocessed, including silence removal: removing the silent part in the audio and retaining only the valid speech segment; segmentation and cropping: segmenting the long audio file into shorter segments, each segment corresponding to an audio sample.

[0034] In one embodiment, according to preset configuration parameters, the parameters of the feature extractor are set, such as sampling rate, window size, step size, etc. The feature extractor can select ResNet18 (Res i dual Network, deep residual network), specifically, all network layers before the last layer FC (Fully Connected Layer, fully connected layer) of ResNet18 are taken as feature extractors.

[0035] In one embodiment, feature extraction is performed using a feature extractor. Specifically, the audio sample is converted into a Mel spectrogram, and the feature extractor is used to extract features from the Mel spectrogram to obtain sample audio features.

[0036] S102, processing the sample audio features based on the feature learning module and the classifier learning module of the pre-training model respectively to obtain a first loss function and a second loss function;

[0037] In one embodiment, the feature learning module MLP1 and the classifier learning module MLP2 based on the pre-trained model process the sample audio features respectively to obtain a first loss function and a second loss function. The feature learning module is constructed based on supervised contrastive learning, the first loss function is a supervised contrastive learning loss function, and the second loss function is a cross entropy loss function.

[0038] In one embodiment, the sample audio features are input to the feature learning module MLP1, wherein MLP1 comprises a plurality of fully connected layers, each of which comprises a Sigmad activation function for extracting and optimizing feature representation.

[0039] In one embodiment, the feature learning module aims to bring the feature representations of samples of the same emotion category closer and push the feature representations of samples of different emotion categories further away. For each sample anchor point, find sample features of the same type (positive sample features) and sample features of different classes (negative sample features). Through the positive sample features and the negative sample features, a first loss function is obtained to encourage the model to reduce the distance between positive examples and increase the distance between negative examples based on the first loss function. Avoid ignoring the connection between features of the same category and improve the accuracy of feature learning.

[0040] In one embodiment, the sample audio features are simultaneously input into the classifier learning module MLP2, wherein MLP2 comprises a fully connected layer including a Softmax activation function for mapping the features to the probability distribution of the emotion category.

[0041] In one embodiment, the second loss function is used to measure the difference between the probability distribution output by MLP2 and the true label. For each sample, the cross entropy between its predicted probability distribution and the true label is calculated to obtain the second loss function.

[0042] Furthermore, the feature learning module based on the pre-trained model processes the sample audio features to obtain a first loss function, including: based on the feature learning module, processing the audio sample features to obtain target sample features; based on the emotion category of each audio sample in the audio sample set, searching for positive sample features and negative sample features of each sample anchor point from the target sample features; based on a preset contrast loss function formula, the sample anchor point, the positive sample features and the negative sample features, obtaining the first loss function; based on the classifier learning module, processing the audio sample features to obtain a predicted emotion label of the audio sample and a predicted probability of each predicted emotion label; based on a preset cross entropy loss function formula, the emotion category of the audio sample, the predicted emotion label and the predicted probability, obtaining the second loss function.

[0043] In one embodiment, for each sample anchor point, other samples with the same label are found as positive samples according to its sentiment category label.

[0044] In one embodiment, the positive sample is a set of samples with the same sentiment category label as the anchor sample, and the negative sample is a set of samples with different sentiment category labels from the anchor sample.

[0045] In one embodiment, for each anchor point sample, the number of positive samples with the same label as the anchor point sample is calculated. For each anchor point sample, the number of negative samples with different labels as the anchor point sample is calculated.

[0046] In one embodiment, the supervised contrast loss function formula may be:

[0047]

[0048] Where N is the size of the sample batch, ﹒ represents the inner product, is the i-th anchor point z i The number of all positive examples, z j is the jth positive sample feature, z k represents the feature of the kth negative sample, and τ is the temperature parameter used for parameter optimization.

[0049] In one embodiment, for each anchor point sample, the contrast loss function formula is used to calculate the loss between it and the positive sample and the negative sample. The losses of all anchor point samples are summed up to obtain a first loss function.

[0050] In one embodiment, the sample audio features are processed by a classifier learning module to obtain a predicted emotion label and a predicted probability of the predicted emotion label for each sample audio.

[0051] The cross entropy loss function is used to measure the difference between the probability distribution predicted by the model and the true label. The formula is:

[0052]

[0053] Among them, C is the total number of sentiment labels, y i represents the true label of the sample audio with predicted emotion label i, p i Represents the predicted probability of predicting sentiment label i.

[0054] The predicted emotion label of each sample audio obtained by the classifier learning module, the predicted probability of the predicted emotion label, and the true emotion label of the sample audio are substituted into the cross entropy loss function formula to obtain the second loss function.

[0055] In this embodiment, the first loss function is a supervised contrast loss function, which is used to shorten the distance between positive samples and increase the distance between negative samples. The second loss function is a cross entropy loss function, which is used to optimize the output of the classifier to make it closer to the true label.

[0056] S103, performing weighted combination of the first loss function and the second loss function according to a target weight coefficient to obtain a target loss function;

[0057] In one embodiment, the target weight coefficient is a hyperparameter between 0 and 1 that controls the relative importance of the two loss functions.

[0058] According to the target weight coefficient, the loss function association formula can be generated: L = aL SCL +(1-a)L CE The first loss function and the second loss function are calculated and combined through the loss function association formula to obtain the target loss function. That is, the first loss function and the second loss function are weighted and combined through a target weight coefficient a to balance their impact on the overall training process. Among them, L SCL is the first loss function, L CE is the second loss function.

[0059] S104. Based on the target loss function, the feature learning module and the classifier learning module are trained to obtain a speech emotion recognition model.

[0060] Furthermore, based on the target loss function, the feature learning module and the classifier learning module are trained to obtain a speech emotion recognition model, including: based on the target loss function, obtaining a target loss value of the current training round; when the target loss value is greater than a preset loss threshold, performing gradient calculations on each parameter in the feature learning module and the classifier learning module respectively to obtain gradient values ​​corresponding to each parameter; based on the gradient values, the parameters of the feature learning module and the classifier learning module are updated, and model training for the next training round is performed based on the sample audio features, until the target loss value is less than or equal to the loss threshold, and the speech emotion recognition model is obtained.

[0061] In one embodiment, the audio sample set can be divided into a training set and a validation set. In the current training round, forward propagation is performed on the audio samples in the training set to calculate feature representations and classification probabilities. A first loss function value is calculated based on the obtained feature representation, a second loss value is calculated based on the obtained classification probability, and finally the target loss value is substituted into the target loss function.

[0062] In one embodiment, it is checked whether the calculated target loss value is greater than a preset loss threshold. If the target loss value is greater than the loss threshold, the parameters of the feature learning module and the classifier learning module are gradient calculated. The gradient of the target loss with respect to the model parameters is calculated using a back propagation algorithm.

[0063] In one embodiment, the parameters of the feature learning module and the classifier learning module are updated according to the calculated gradient value and the selected optimizer. Exemplarily, for each parameter θ, the update rule may be: Among them, α is the learning rate, is the gradient of the loss function with respect to the parameter θ.

[0064] In one embodiment, the updated parameters are used to perform model training for the next training round based on the sample audio features. After each training round, it is checked whether the target loss value L is less than or equal to a preset loss threshold.

[0065] In another embodiment, the model performance may also be evaluated on the validation set to check whether other termination conditions are met, such as the performance no longer improving.

[0066] In one embodiment, when the target loss value is less than or equal to a preset loss threshold, or other termination conditions are met, the training is stopped. The trained speech emotion recognition model is output, and the parameters of the model have been optimized based on the training data. The trained model is deployed in practical applications, such as an intelligent voice customer service system, to perform speech emotion recognition.

[0067] The above embodiment provides a training method, device, computer equipment and medium for a speech emotion recognition model, obtains an audio sample set, and extracts features from the audio samples in the audio sample set based on a preset feature extractor to obtain sample audio features; processes the sample audio features based on the feature learning module and the classifier learning module of the pre-trained model to obtain a first loss function and a second loss function; weightedly combines the first loss function and the second loss function according to the target weight coefficient to obtain a target loss function; trains the feature learning module and the classifier learning module based on the target loss function to obtain a speech emotion recognition model. The present application trains the feature learning module and the classifier learning module simultaneously through sample audio features, and weightedly combines the feature learning module and the classifier learning module through the target weight coefficient, and can dynamically adjust the attention paid to the feature learning module and the classifier learning module during the training process according to the target weight coefficient, so that the model can make full use of the sample audio features and learn feature representations with more emotional discrimination, thereby improving the model performance and thus improving the accuracy of speech emotion recognition.

[0068] See also Figure 2 , Figure 2 This is a schematic flow chart of a training method for a speech emotion recognition model provided by an embodiment of the present application. The training method for the speech emotion recognition model can be applied to a server, and is used to associate a first loss function with a second loss function, and adjust the target weight coefficient of each training round, so that the model can focus on different learning tasks at different stages of training, and the model can gradually learn and optimize feature representation and classification to improve recognition accuracy.

[0069] like Figure 2 As shown, the training method of the speech emotion recognition model specifically includes steps S201 to S202.

[0070] S201, obtaining target weight coefficients of the first loss function and the second loss function in each training round, and obtaining a loss function association formula for each training round based on the target weight coefficients;

[0071] S202. Based on the loss function association formula, combine the first loss function and the second loss function of each training round to obtain a target loss function of each training round.

[0072] In this embodiment, the target weight coefficient is used to balance the relative importance of the feature learning module and the classifier learning module. The target weight coefficient is a value that changes with the training round. For example, a course learning strategy can be used, and the target weight coefficient gradually changes with the training progress.

[0073] In one embodiment, according to the target weight coefficient a, a loss function association formula for each training round is generated, and based on the loss function association formula for each training round, the first loss function L of each training round is converted into SCL and the second loss function L CE Combined together, we form the target loss function L for each training round. The loss function association formula is L = aL SCL +(1-a)L CE .

[0074] Further, the obtaining of the target weight coefficients of the first loss function and the second loss function in each training round, and obtaining the loss function association formula of each training round based on the target weight coefficients, includes: obtaining a current training round and a preset maximum training round; based on the current training round and the maximum training round, calculating and obtaining the target weight coefficient corresponding to the current training round;

[0075] Among them, the target weight coefficient T is the current training round, T max is the maximum number of training rounds.

[0076] In one embodiment, the current training round T is the number of the training round currently being performed. max It is freely set by the user according to needs and is the maximum number of rounds required during the training process.

[0077] In one embodiment, the value of the target weight coefficient in each training round is given by the formula Calculate and obtain. Substitute the current training round T and the maximum training round T max Substitute into the formula to obtain the target weight coefficient a corresponding to the current training round.

[0078] In one embodiment, the first loss function L is combined using the calculated target weight coefficient a SCL and the second loss function L CE , forming the target loss function L: L = aL SCL +(1-a)L CE .

[0079] In the above embodiment, by associating the first loss function and the second loss function and adjusting the target weight coefficient of each training round, the model can focus on different learning tasks at different stages of training. The model can gradually learn and optimize feature representation and classification to improve recognition accuracy.

[0080] See also Figure 3 , Figure 3 This is a schematic flow chart of a method for training a speech emotion recognition model provided in an embodiment of the present application. The method for training a speech emotion recognition model can be applied to a server to process initial audio features based on a feature learning module of supervised contrastive learning to obtain target audio features with higher emotion discrimination, thereby improving the recognition accuracy of a classifier learning module.

[0081] like Figure 3 As shown, the training method of the speech emotion recognition model specifically includes steps S301 to S304.

[0082] S301, obtaining audio to be recognized, and converting the audio to be recognized into a target spectrogram;

[0083] S302, extracting features from the target spectrogram based on the feature extractor to obtain initial audio features;

[0084] S303, based on the feature learning module in the speech emotion recognition model, processing the initial audio features to obtain target audio features;

[0085] S304: Perform emotion prediction based on the classifier learning module in the speech emotion recognition model and the target audio features to obtain an emotion label of the audio to be recognized.

[0086] In this embodiment, the audio to be recognized is a voice file that needs to be emotion recognized. The audio may also be pre-processed as necessary, including denoising, normalizing the volume, and the like.

[0087] In one embodiment, an audio signal is converted into a spectrogram using an audio processing technique. Specifically, a fast Fourier transform is performed on the audio signal to obtain a frequency domain representation, and a me l filter bank is applied to convert the frequency domain signal into a me l spectrogram as a target spectrogram.

[0088] In one embodiment, a predefined feature extractor is used to extract features from the target spectrogram to obtain initial audio features.

[0089] In one embodiment, the initial audio features are input into the feature learning module of the speech emotion recognition model. The feature learning module is a supervised contrastive learning module that can transform and abstract the initial audio features and extract deeper feature representations. The target audio features processed by the feature learning module are output, and these features are more suitable for emotion classification.

[0090] In one embodiment, the target audio features are input into a classifier learning module of a speech emotion recognition model. The classifier learning module performs emotion classification based on the target audio features and outputs a probability distribution of each emotion category. The category with the highest probability in the probability distribution is selected as the emotion label of the audio to be recognized.

[0091] Furthermore, the emotion prediction is performed based on the classifier learning module in the speech emotion recognition model and the target audio features to obtain the emotion label of the audio to be recognized, including: based on the classifier learning module, processing the target audio features to obtain at least one emotion category of the audio to be recognized and the probability corresponding to each emotion category, and taking the emotion category with the highest probability as the emotion label of the audio to be recognized.

[0092] In one embodiment, the target audio features are used as input parameters of the classifier learning module of the speech emotion recognition model. In the classifier learning module, forward propagation is performed based on the target audio features to calculate the predicted probability of each emotion category. And the Softmax activation function is used in the last layer of the classifier learning module to convert the output into a probability distribution. Specifically, the Softmax function converts the output raw score into a probability distribution, and each emotion category corresponds to a probability value.

[0093] In one embodiment, the emotion category with the maximum probability value is found in the probability distribution, and the emotion category with the maximum probability is used as the emotion label of the audio to be recognized.

[0094] In the above embodiment, the feature learning module based on supervised contrastive learning processes the initial audio features to obtain target audio features with better emotion discrimination, thereby improving the recognition accuracy of the classifier learning module.

[0095] See also Figure 4 , Figure 4 The embodiment of the present application provides a schematic block diagram of a training device for a speech emotion recognition model, wherein the training device for the speech emotion recognition model is used to execute the aforementioned training method for the speech emotion recognition model. The training device for the speech emotion recognition model can be configured on a server.

[0096] like Figure 4 As shown, the training device 400 of the speech emotion recognition model includes:

[0097] The sample audio feature acquisition module 401 is used to acquire an audio sample set, and extract features from the audio samples in the audio sample set based on a preset feature extractor to obtain sample audio features;

[0098] A sample audio feature processing module 402, used to process the sample audio features based on the feature learning module and the classifier learning module of the pre-trained model, respectively, to obtain a first loss function and a second loss function;

[0099] A target loss function combining module 403 is used to perform a weighted combination of the first loss function and the second loss function according to a target weight coefficient to obtain a target loss function;

[0100] The emotion recognition model acquisition module 404 is used to train the feature learning module and the classifier learning module based on the target loss function to obtain a speech emotion recognition model.

[0101] Furthermore, the target loss function joint module 403 includes:

[0102] A weight coefficient acquisition unit, used to acquire target weight coefficients of the first loss function and the second loss function in each training round, and obtain a loss function association formula for each training round based on the target weight coefficients;

[0103] The target loss function obtaining unit is used to combine the first loss function and the second loss function of each training round based on the loss function association formula to obtain the target loss function of each training round.

[0104] Furthermore, the target weight coefficient acquisition unit includes:

[0105] The round acquisition subunit is used to obtain the current training round and the preset maximum training round;

[0106] A weight coefficient obtaining subunit, configured to calculate the target weight coefficient corresponding to the current training round based on the current training round and the maximum training round;

[0107] Among them, the target weight coefficient T is the current training round, T max is the maximum number of training rounds.

[0108] Furthermore, the emotion recognition model acquisition module 404 includes:

[0109] A target loss value obtaining unit, used to obtain a target loss value of a current training round based on the target loss function;

[0110] A gradient value obtaining unit, used to perform gradient calculation on each parameter in the feature learning module and the classifier learning module respectively to obtain a gradient value corresponding to each parameter when the target loss value is greater than a preset loss threshold;

[0111] A parameter updating unit is used to update the parameters of the feature learning module and the classifier learning module based on the gradient value, and to perform model training for the next training round based on the sample audio features until the target loss value is less than or equal to the loss threshold, thereby obtaining the speech emotion recognition model.

[0112] Furthermore, the training device 400 for the speech emotion recognition model further includes a speech emotion recognition module, and the speech emotion recognition module includes:

[0113] A target spectrum acquisition unit, used to acquire the audio to be recognized and convert the audio to be recognized into a target spectrum;

[0114] An initial audio feature obtaining unit, used for extracting features from the target spectrogram based on the feature extractor to obtain initial audio features;

[0115] A target audio feature obtaining unit, configured to process the initial audio feature based on a feature learning module in the speech emotion recognition model to obtain a target audio feature;

[0116] The emotion label obtaining unit is used to perform emotion prediction based on the classifier learning module in the speech emotion recognition model and the target audio features to obtain the emotion label of the audio to be recognized.

[0117] Furthermore, the emotion label acquisition unit is specifically used to process the target audio features based on the classifier learning module, obtain at least one emotion category of the audio to be identified and the probability corresponding to each emotion category, and take the emotion category with the highest probability as the emotion label of the audio to be identified.

[0118] Furthermore, the sample audio feature processing module 402 includes:

[0119] A target sample feature obtaining unit, configured to process the audio sample feature based on the feature learning module to obtain a target sample feature;

[0120] A sample search unit, configured to search for positive sample features and negative sample features of each sample anchor point from the target sample features based on the emotion category of each audio sample in the audio sample set;

[0121] A first loss function obtaining unit, configured to obtain the first loss function based on a preset contrast loss function formula, the sample anchor point, the positive sample feature, and the negative sample feature;

[0122] A prediction probability obtaining unit, used to process the audio sample features based on the classifier learning module to obtain the predicted emotion label of the audio sample and the prediction probability of each predicted emotion label;

[0123] The second loss function obtaining unit is used to obtain the second loss function based on a preset cross entropy loss function formula, the emotion category of the audio sample, the predicted emotion label and the predicted probability.

[0124] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device and each module can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0125] The above-mentioned device can be implemented in the form of a computer program. Figure 5 Runs on the computer device shown.

[0126] See also Figure 5 , Figure 5 1 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device may be a server.

[0127] See also Figure 5 The computer device includes a processor, a memory and a network interface connected through a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.

[0128] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any training method for a speech emotion recognition model.

[0129] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0130] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any training method for the speech emotion recognition model.

[0131] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 5The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0132] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0133] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:

[0134] Acquire an audio sample set, and perform feature extraction on audio samples in the audio sample set based on a preset feature extractor to obtain sample audio features;

[0135] Based on the feature learning module and the classifier learning module of the pre-trained model respectively, the sample audio features are processed to obtain a first loss function and a second loss function;

[0136] Performing a weighted combination of the first loss function and the second loss function according to a target weight coefficient to obtain a target loss function;

[0137] Based on the target loss function, the feature learning module and the classifier learning module are trained to obtain a speech emotion recognition model.

[0138] In one embodiment, when the processor performs weighted combination of the first loss function and the second loss function according to the target weight coefficient to obtain the target loss function, it is used to implement:

[0139] Obtain target weight coefficients of the first loss function and the second loss function in each training round, and based on the target weight coefficients, obtain a loss function association formula for each training round;

[0140] Based on the loss function association formula, the first loss function and the second loss function of each training round are combined to obtain the target loss function of each training round.

[0141] In one embodiment, the processor is used to implement, when obtaining the target weight coefficients of the first loss function and the second loss function in each training round, and obtaining the loss function association formula of each training round based on the target weight coefficient:

[0142] Get the current training round and the preset maximum training round;

[0143] Based on the current training round and the maximum training round, calculate and obtain the target weight coefficient corresponding to the current training round;

[0144] Among them, the target weight coefficient T is the current training round, T max is the maximum number of training rounds.

[0145] In one embodiment, when the processor trains the feature learning module and the classifier learning module based on the target loss function to obtain the speech emotion recognition model, it is used to implement:

[0146] Based on the target loss function, obtaining a target loss value for the current training round;

[0147] When the target loss value is greater than a preset loss threshold, performing gradient calculations on each parameter in the feature learning module and the classifier learning module respectively to obtain a gradient value corresponding to each parameter;

[0148] The parameters of the feature learning module and the classifier learning module are updated based on the gradient value, and the model training of the next training round is performed based on the sample audio features until the target loss value is less than or equal to the loss threshold, so as to obtain the speech emotion recognition model.

[0149] In one embodiment, after the processor implements training the feature learning module and the classifier learning module based on the target loss function to obtain the speech emotion recognition model, it is further used to implement:

[0150] Acquire the audio to be recognized, and convert the audio to be recognized into a target spectrogram;

[0151] Performing feature extraction on the target spectrogram based on the feature extractor to obtain initial audio features;

[0152] Based on the feature learning module in the speech emotion recognition model, the initial audio features are processed to obtain target audio features;

[0153] Emotion prediction is performed based on the classifier learning module in the speech emotion recognition model and the target audio features to obtain the emotion label of the audio to be recognized.

[0154] In one embodiment, when the processor performs emotion prediction based on the classifier learning module in the speech emotion recognition model and the target audio feature to obtain the emotion label of the audio to be recognized, it is used to implement:

[0155] Based on the classifier learning module, the target audio features are processed to obtain at least one emotion category of the audio to be identified and the probability corresponding to each emotion category, and the emotion category with the largest probability is used as the emotion label of the audio to be identified.

[0156] In one embodiment, the processor, when implementing a feature learning module based on a pre-trained model and processing the sample audio features to obtain a first loss function, is used to implement:

[0157] Based on the feature learning module, the audio sample features are processed to obtain target sample features;

[0158] Based on the emotion category of each audio sample in the audio sample set, searching for positive sample features and negative sample features of each sample anchor point from the target sample features;

[0159] Obtaining the first loss function based on a preset contrast loss function formula, the sample anchor point, the positive sample features, and the negative sample features;

[0160] Based on the classifier learning module, the audio sample features are processed to obtain predicted emotion labels of the audio samples and predicted probabilities of each predicted emotion label;

[0161] The second loss function is obtained based on a preset cross entropy loss function formula, the emotion category of the audio sample, the predicted emotion label, and the predicted probability.

[0162] A computer-readable storage medium is also provided in an embodiment of the present application, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and the processor executes the program instructions to implement any one of the speech emotion recognition model training methods provided in the embodiments of the present application.

[0163] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (Smart Medi a Card, SMC), a secure digital (Secure Digital, SD) card, a flash memory card (Flash Card), etc., equipped on the computer device.

[0164] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for training a speech emotion recognition model, characterized in that: include: Acquire an audio sample set, and perform feature extraction on audio samples in the audio sample set based on a preset feature extractor to obtain sample audio features; Based on the feature learning module and the classifier learning module of the pre-trained model respectively, the sample audio features are processed to obtain a first loss function and a second loss function; Performing a weighted combination of the first loss function and the second loss function according to a target weight coefficient to obtain a target loss function; Based on the target loss function, the feature learning module and the classifier learning module are trained to obtain a speech emotion recognition model.

2. The method for training a speech emotion recognition model according to claim 1, characterized in that: The step of performing a weighted combination of the first loss function and the second loss function according to a target weight coefficient to obtain a target loss function includes: Obtain target weight coefficients of the first loss function and the second loss function in each training round, and based on the target weight coefficients, obtain a loss function association formula for each training round; Based on the loss function association formula, the first loss function and the second loss function of each training round are combined to obtain the target loss function of each training round.

3. The training method of the speech emotion recognition model according to claim 2, characterized in that: The obtaining target weight coefficients of the first loss function and the second loss function in each training round includes: Get the current training round and the preset maximum training round; Based on the current training round and the maximum training round, calculate and obtain the target weight coefficient corresponding to the current training round; Among them, the target weight coefficient T is the current training round, T max is the maximum number of training rounds.

4. The method for training a speech emotion recognition model according to claim 1, characterized in that: The method of training the feature learning module and the classifier learning module based on the target loss function to obtain a speech emotion recognition model includes: Based on the target loss function, obtaining a target loss value for the current training round; When the target loss value is greater than a preset loss threshold, performing gradient calculations on each parameter in the feature learning module and the classifier learning module respectively to obtain a gradient value corresponding to each parameter; The parameters of the feature learning module and the classifier learning module are updated based on the gradient value, and the model training of the next training round is performed based on the sample audio features until the target loss value is less than or equal to the loss threshold, so as to obtain the speech emotion recognition model.

5. The method for training a speech emotion recognition model according to claim 1, characterized in that: After the feature learning module and the classifier learning module are trained based on the target loss function to obtain the speech emotion recognition model, the method further includes: Acquire the audio to be recognized, and convert the audio to be recognized into a target spectrogram; Performing feature extraction on the target spectrogram based on the feature extractor to obtain initial audio features; Based on the feature learning module in the speech emotion recognition model, the initial audio features are processed to obtain target audio features; Emotion prediction is performed based on the classifier learning module in the speech emotion recognition model and the target audio features to obtain the emotion label of the audio to be recognized.

6. The method for training a speech emotion recognition model according to claim 5, characterized in that: The performing of emotion prediction based on the classifier learning module in the speech emotion recognition model and the target audio features to obtain the emotion label of the audio to be recognized includes: Based on the classifier learning module, the target audio features are processed to obtain at least one emotion category of the audio to be identified and the probability corresponding to each emotion category, and the emotion category with the largest probability is used as the emotion label of the audio to be identified.

7. The method for training a speech emotion recognition model according to any one of claims 1 to 6, characterized in that: The feature learning module and the classifier learning module based on the pre-trained model respectively process the sample audio features to obtain the first loss function and the second loss function, including: Based on the feature learning module, the audio sample features are processed to obtain target sample features; Based on the emotion category of each audio sample in the audio sample set, searching for positive sample features and negative sample features of each sample anchor point from the target sample features; Obtaining the first loss function based on a preset contrast loss function formula, the sample anchor point, the positive sample features, and the negative sample features; Based on the classifier learning module, the audio sample features are processed to obtain predicted emotion labels of the audio samples and predicted probabilities of each predicted emotion label; The second loss function is obtained based on a preset cross entropy loss function formula, the emotion category of the audio sample, the predicted emotion label, and the predicted probability.

8. A training device for a speech emotion recognition model, characterized in that: include: A sample audio feature acquisition module, used to acquire an audio sample set, and perform feature extraction on the audio samples in the audio sample set based on a preset feature extractor to obtain sample audio features; A sample audio feature processing module, used to process the sample audio features based on the feature learning module and the classifier learning module of the pre-trained model, respectively, to obtain a first loss function and a second loss function; An objective loss function combination module, used for weighted combination of the first loss function and the second loss function according to an objective weight coefficient to obtain an objective loss function; The emotion recognition model acquisition module is used to train the feature learning module and the classifier learning module based on the target loss function to obtain a speech emotion recognition model.

9. A computer device, characterized in that: The computer device includes a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the training method of the speech emotion recognition model as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, enables the processor to implement the training method for a speech emotion recognition model as described in any one of claims 1 to 7.

Images