Methods, devices, equipment, media, and products for voice emotion type recognition

Abstract

This invention provides a method, apparatus, device, medium, and product for speech emotion type recognition, belonging to the field of emotion recognition technology. The method includes: acquiring an audio signal; performing multi-level feature extraction based on the audio signal; inputting the multi-level audio features into an emotion recognition model, outputting a first posterior probability of the audio signal containing an emotion, a probability threshold value for the audio signal containing an emotion, and second posterior probabilities corresponding to multiple emotion types; determining the emotion coefficient of the audio signal based on the first posterior probability and the probability threshold value; and determining the target emotion type result of the audio signal based on the emotion coefficient and multiple second posterior probabilities. The speech emotion type recognition method provided by this invention first reflects a preliminary emotion classification result through the emotion coefficient, and then, based on this emotion classification result, achieves a more granular emotion classification by combining the emotion coefficient and multiple second posterior probabilities, thereby improving the accuracy of speech emotion type recognition.

Description

Technical Field

[0001] This invention relates to the field of emotion recognition technology, and in particular to a method, apparatus, device, medium, and product for recognizing voice emotion types. Background Technology

[0002] Emotions play a crucial role in human communication, and speech is a vital carrier of human emotions. Effectively identifying emotional states in speech can improve the efficiency of human-computer interaction. For example, in call centers, analyzing a customer's historical emotional state allows for the assignment of an emotional label. Calls from irritable customers can be routed to experienced agents, and real-time monitoring of customer emotional states also helps improve service quality. Technically, emotions influence audio features, so analyzing these features can help identify the speaker's emotional state.

[0003] Traditional speech emotion recognition methods first extract acoustic features believed to contain human emotional information frame by frame from the original speech waveform. Then, they fuse the features from all frames of the entire speech segment using a specific method to form a feature representing the entire speech segment. Finally, the feature representing the entire speech segment is input into a classifier for emotion classification. Existing technologies often use a single traditional neural network model when designing classifiers, failing to consider the multi-granularity of emotion categories. This leads to confusion between fine-grained emotion categories and inaccurate recognition. Summary of the Invention

[0004] This invention provides a method, apparatus, device, medium, and product for recognizing voice emotion types, in order to solve the problem of low accuracy in voice emotion type recognition in the prior art.

[0005] In a first aspect, the present invention provides a method for recognizing speech emotion types, comprising:

[0006] Acquire audio signals;

[0007] Multi-level feature extraction is performed based on the audio signal to obtain multi-level audio features;

[0008] The multi-level audio features are input into an emotion recognition model to obtain a first posterior probability of the audio signal containing an emotion, a probability threshold value for the audio signal containing an emotion, and second posterior probabilities corresponding to multiple emotion types, respectively. The emotion recognition model includes an emotion detector, an emotion probability threshold calculator, and an emotion classifier. The first posterior probability is predicted by the emotion detector. The probability threshold value is predicted by the emotion probability threshold calculator. The second posterior probabilities corresponding to the multiple emotion types are predicted by the emotion classifier.

[0009] Based on the first posterior probability and the probability threshold, the emotion coefficient of the audio signal is determined; the emotion coefficient is used to reflect the confidence that the audio signal contains emotion.

[0010] Based on the emotion coefficient and multiple second posterior probabilities, the target emotion type of the audio signal is determined.

[0011] In one embodiment, determining the emotion coefficient of the audio signal based on the first posterior probability and the probability threshold includes:

[0012] The difference between the first posterior probability and the probability threshold value is calculated to obtain the difference.

[0013] The emotional coefficient of the audio signal is obtained by performing an exponential calculation based on the difference.

[0014] In one embodiment, determining the target emotion type result of the audio signal based on the emotion coefficient and multiple second posterior probabilities includes:

[0015] The multiple second posterior probabilities are multiplied by the emotion coefficient to obtain multiple products;

[0016] The largest product among the plurality of products is determined as the target posterior probability;

[0017] The emotion type corresponding to the target posterior probability is determined as the target emotion type result of the audio signal.

[0018] In one embodiment, the multi-level feature extraction based on the audio signal to obtain multi-level audio features includes:

[0019] The audio signal is preprocessed to obtain the preprocessed audio signal.

[0020] The preprocessed audio signal is input into an audio feature extraction model to obtain multi-level audio features output by the audio feature extraction model; the multi-level audio features include low-level audio features, mid-level audio features, and high-level audio features; the semantic meaning levels of the low-level audio features, mid-level audio features, and high-level audio features are different;

[0021] The audio feature extraction model is obtained by pre-training a model based on the Transformer algorithm.

[0022] In one embodiment, the emotion recognition model includes a dimensionality reduction linear layer, a feature concatenation layer, an attention linear layer, a feature stacking layer, a feature fusion layer, and an output linear layer; the step of inputting the multi-level audio features into the emotion recognition model to obtain a first posterior probability of the audio signal containing an emotion, a probability threshold value of the audio signal containing an emotion, and second posterior probabilities corresponding to multiple emotion types, respectively, includes:

[0023] The multi-level audio features are input into a dimension reduction linear layer to obtain the multi-level dimension reduction features output by the dimension reduction linear layer; the multi-level dimension reduction features include low-level dimension reduction features, mid-level dimension reduction features, and high-level dimension reduction features;

[0024] The low-level dimensionality reduction features, the mid-level dimensionality reduction features, and the high-level dimensionality reduction features are input into the feature stacking layer to obtain the feature matrix output by the feature stacking layer.

[0025] The low-level dimensionality reduction features, the mid-level dimensionality reduction features, and the high-level dimensionality reduction features are input into the feature concatenation layer to obtain the concatenated feature vector output by the feature concatenation layer.

[0026] The concatenated feature vector is input into the attention linear layer to obtain the attention vector output by the attention linear layer;

[0027] The feature matrix and the attention vector are input into the feature fusion layer to obtain the fusion vector output by the feature fusion layer;

[0028] The fusion vector is input to the output linear layer of the emotion detector to obtain the first posterior probability that the audio signal output by the output linear layer of the emotion detector has an emotion.

[0029] The fusion vector is input to the output linear layer of the emotion probability threshold calculator to obtain the probability threshold value of the audio signal having emotion output by the output linear layer of the emotion probability threshold calculator;

[0030] The fusion vector is input into the output linear layer of the emotion classifier to obtain the second posterior probabilities corresponding to the multiple emotion types output by the output linear layer of the emotion classifier.

[0031] In one embodiment, the emotion recognition model is trained in the following manner:

[0032] Obtain multi-level audio feature samples;

[0033] Emotion labeling is performed on the multi-level audio feature samples to obtain emotion labeling results; the emotion labeling results include whether an emotion exists or not.

[0034] The multi-level audio feature samples are labeled with emotion types to obtain emotion type labeling results; the emotion type labeling results include multiple emotion types.

[0035] The trained emotion detector is obtained by training based on the multi-level audio feature samples and the emotion annotation results; the output linear layer of the emotion detector is used to predict the first posterior probability that the multi-level audio feature samples contain emotions.

[0036] The trained emotion classifier is obtained by training based on the multi-level audio feature samples and the emotion type labeling results.

[0037] After the emotion detector and the emotion classifier have been trained, a trained emotion probability threshold calculator is obtained based on the multi-level audio feature samples and the emotion annotation results. The output linear layer of the emotion probability threshold calculator is used to predict the probability threshold value of the presence of emotion in the multi-level audio feature samples.

[0038] Secondly, the present invention also provides a voice emotion type recognition device, comprising:

[0039] The acquisition module is used to acquire audio signals;

[0040] The feature extraction module is used to perform multi-level feature extraction based on the audio signal to obtain multi-level audio features;

[0041] An emotion recognition module is used to input the multi-level audio features into an emotion recognition model to obtain a first posterior probability that the audio signal contains an emotion, a probability threshold value for the audio signal to contain an emotion, and second posterior probabilities corresponding to multiple emotion types, respectively, output by the emotion recognition model. The emotion recognition model includes an emotion detector, an emotion probability threshold calculator, and an emotion classifier. The first posterior probability is predicted by the emotion detector; the probability threshold value is predicted by the emotion probability threshold calculator; and the second posterior probabilities corresponding to the multiple emotion types are predicted by the emotion classifier.

[0042] The first determining module is used to determine the emotion coefficient of the audio signal based on the first posterior probability and the probability threshold value; the emotion coefficient is used to reflect the confidence that the audio signal contains emotion.

[0043] The second determining module is used to determine the target emotion type result of the audio signal based on the emotion coefficient and multiple second posterior probabilities.

[0044] Thirdly, the present invention provides an apparatus comprising an electronic device, the electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the steps of any of the above-described voice emotion type recognition methods.

[0045] Fourthly, the present invention also provides a medium comprising a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above-described speech emotion type recognition methods.

[0046] Fifthly, the present invention also provides a product comprising a computer program product, the computer program product comprising a computer program, the computer program being able to be stored on a non-transitory computer-readable storage medium, and the computer program, when executed by the processor, implementing the steps of any of the above-described voice emotion type recognition methods.

[0047] The speech emotion type recognition method, device, equipment, medium, and product provided by this invention extracts rich feature information from audio signals through multi-level feature extraction, obtaining multi-level audio features. These multi-level audio features are then input into an emotion recognition model, which can accurately predict the first posterior probability of an emotion in the audio signal, the probability threshold value of an emotion in the audio signal, and the second posterior probabilities corresponding to multiple emotion types. First, an emotion coefficient is determined based on the first posterior probability and the probability threshold value. The emotion coefficient reflects a preliminary emotion classification result. Then, based on this emotion classification result, a finer-grained emotion classification is achieved by combining the emotion coefficient and multiple second posterior probabilities, reducing the impact of confusion between multiple emotion types and thus improving the accuracy of speech emotion type recognition. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0049] Figure 1 This is a flowchart illustrating the speech emotion type recognition method provided by the present invention;

[0050] Figure 2 This is a schematic diagram of the structure of the emotion recognition model provided by the present invention;

[0051] Figure 3 This is a schematic diagram of the voice emotion type recognition device provided by the present invention;

[0052] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0054] The terms "first," "second," etc., used in this invention are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention can be implemented in orders other than those illustrated or described herein.

[0055] The following is combined with Figures 1-4 This invention describes the speech emotion type recognition method, apparatus, device, medium, and product provided by the present invention.

[0056] Combination Figure 1 , Figure 1 This is a flowchart illustrating the speech emotion type recognition method provided by the present invention. Figure 1 As shown, the speech emotion type recognition method provided by this invention includes, but is not limited to, the following steps:

[0057] Step 101: Acquire the audio signal;

[0058] Step 102: Perform multi-level feature extraction based on the audio signal to obtain multi-level audio features;

[0059] Step 103: Input the multi-level audio features into the emotion recognition model to obtain the first posterior probability of the audio signal having an emotion, the probability threshold value of the audio signal having an emotion, and the second posterior probabilities corresponding to multiple emotion types, respectively, as output by the emotion recognition model.

[0060] Step 104: Determine the emotion coefficient of the audio signal based on the first posterior probability and the probability threshold value;

[0061] Step 105: Based on the emotion coefficient and multiple second posterior probabilities, determine the target emotion type result of the audio signal.

[0062] It should be noted that the speech emotion type recognition method provided in this embodiment of the invention is implemented based on a speech emotion type recognition device. The method employs a two-stage approach to achieve accurate speech emotion type recognition. First, emotion detection is performed on the multi-level audio features extracted by the pre-trained model, i.e., emotion is classified as present or absent, yielding a preliminary emotion classification result. Then, based on this result, a more granular emotion classification is performed to obtain an accurate target emotion type result. Therefore, this embodiment of the invention uses a speech emotion type recognition device as the execution entity to describe the speech emotion type recognition method.

[0063] Specifically, the voice emotion type recognition device acquires audio signals. In one embodiment, the voice emotion type recognition method can be applied to scenarios such as telephone customer service and call centers to identify customer emotions. Therefore, the voice emotion type recognition device can acquire audio signals from the operator.

[0064] Furthermore, the voice emotion type recognition device performs multi-level feature extraction based on the audio signal to obtain multi-level audio features. Through multi-level feature extraction, rich feature information can be extracted from the audio signal for subsequent emotion detection and emotion type recognition, achieving accurate voice emotion type recognition.

[0065] Furthermore, the voice emotion type recognition device inputs multi-level audio features into the emotion recognition model to obtain the first posterior probability of the audio signal containing emotion, the probability threshold value of the audio signal containing emotion, and the second posterior probabilities corresponding to multiple emotion types. The emotion recognition model includes an emotion detector, an emotion probability threshold calculator, and an emotion classifier. Specifically, the first posterior probability is predicted by the emotion detector, the probability threshold value is predicted by the emotion probability threshold calculator, and the second posterior probabilities corresponding to multiple emotion types are predicted by the emotion classifier.

[0066] Therefore, it can be understood that the voice emotion type recognition device inputs multi-level audio features into an emotion detector to obtain a first posterior probability that the audio signal output by the emotion detector contains an emotion. Further, the voice emotion type recognition device inputs multi-level audio features into an emotion probability threshold calculator to obtain a probability threshold value for the audio signal output by the emotion probability threshold calculator to contain an emotion. Further, the voice emotion type recognition device inputs multi-level audio features into an emotion classifier to obtain second posterior probabilities corresponding to the multiple emotion types output by the emotion classifier.

[0067] It should be noted that the embodiments of the present invention use an emotion probability threshold calculator to automatically calculate the probability threshold value of the presence of emotion in the audio signal, assisting the emotion detector and emotion classifier in refining the emotion categories, reducing the impact of confusion between categories, and thus improving the accuracy of voice emotion type recognition.

[0068] Furthermore, the voice emotion type recognition device determines the emotion coefficient of the audio signal based on the first posterior probability and probability threshold value of the audio signal containing emotion, thereby realizing emotion detection of the audio signal. The emotion coefficient reflects the confidence level that the audio signal contains emotion. When the emotion coefficient is greater than a preset value, it indicates that the first posterior probability of the audio signal output by the emotion detector containing emotion is greater than the probability threshold value. In this case, the audio signal can be considered to contain emotion and classified as an emotional category. When the emotion coefficient is less than the preset value, it indicates that the first posterior probability of the audio signal output by the emotion detector containing emotion is less than the probability threshold value. In this case, the audio signal can be considered to not contain emotion and classified as a neutral category.

[0069] It should be noted that neutrality can be considered a state without emotion. Neutrality is characterized by emotional neutrality or emotional stability. In contrast, a state with emotions has obvious emotional expression and intensity. Examples of states with emotions include anger, happiness, and sadness.

[0070] Furthermore, the voice emotion type recognition device determines the target emotion type result of the audio signal based on the emotion coefficient of the audio signal containing emotion and the second posterior probability corresponding to multiple emotion types, thereby achieving more granular emotion type recognition.

[0071] The speech emotion type recognition method provided by this invention extracts rich feature information from audio signals through multi-level feature extraction, obtaining multi-level audio features. These multi-level audio features are then input into an emotion recognition model, which can accurately predict the first posterior probability of the presence of emotion in the audio signal, the probability threshold value for the presence of emotion, and the second posterior probabilities corresponding to multiple emotion types. First, an emotion coefficient is determined based on the first posterior probability and the probability threshold value. This emotion coefficient reflects a preliminary emotion classification result, i.e., the classification result of whether or not emotion is present. Then, based on this emotion classification result, a finer-grained emotion classification is achieved by combining the emotion coefficient and multiple second posterior probabilities, reducing the impact of confusion between multiple emotion types and thus improving the accuracy of speech emotion type recognition.

[0072] Further, based on step 102, the multi-level feature extraction based on the audio signal to obtain multi-level audio features includes:

[0073] The audio signal is preprocessed to obtain the preprocessed audio signal.

[0074] The preprocessed audio signal is input into an audio feature extraction model to obtain multi-level audio features output by the audio feature extraction model; the multi-level audio features include low-level audio features, mid-level audio features, and high-level audio features; the semantic meaning levels of the low-level audio features, mid-level audio features, and high-level audio features are different;

[0075] The audio feature extraction model is obtained by pre-training a model based on the Transformer algorithm.

[0076] Specifically, the voice emotion type recognition device performs signal preprocessing on the audio signal to obtain a preprocessed audio signal. The signal preprocessing may include noise removal, noise reduction, normalization, etc. The purpose of signal preprocessing is to improve the quality of the audio signal and reduce interference in subsequent processing.

[0077] Furthermore, the voice emotion type recognition device inputs the preprocessed audio signal into the audio feature extraction model to obtain low-level audio features, mid-level audio features, and high-level audio features output by the audio feature extraction model. The audio feature extraction model is obtained by pre-training a model based on the Transformer algorithm.

[0078] It should be noted that low-level, mid-level, and high-level audio features have different levels of abstraction and semantic meaning. Lower-level audio features contain more information but less semantic information. Low-level audio features are closer to the representation of the original audio signal and can capture the local structure and subtle changes of the sound. Mid-level audio features act as a bridge between low-level and high-level features in the representation of the audio signal, and can capture a certain degree of semantic and contextual information. High-level audio features usually correspond to more complex speech structures, speech units, or semantic information in the audio signal and have a higher semantic interpretation capability.

[0079] In one embodiment, the audio feature extraction model can employ the Hubert audio pre-training model, a Transformer-based pre-training model trained by performing K-means clustering on acoustic or Hubert features. During training, the model uses an iterative approach. The base model performs clustering on acoustic features in its first iteration, and subsequent iterations cluster on the intermediate layer features obtained from the previous iteration, ultimately resulting in a powerful pre-trained model. The 1024-dimensional vectors output from layers 18, 19, and 20 of the Hubert pre-training model can be extracted as low-level, mid-level, and high-level audio features, respectively.

[0080] This invention improves the quality of audio signals by performing signal preprocessing, and then inputs the preprocessed audio signals into an audio feature extraction model constructed by the Transformer algorithm to obtain multi-level audio features, namely low-level, mid-level, and high-level audio features, which have different levels of semantic expression. This feature method can capture information in the audio signal more comprehensively and express the semantic features of the audio content more effectively, thereby improving the accuracy and efficiency of audio signal processing tasks.

[0081] Further, based on step 103, the step of inputting the multi-level audio features into the emotion recognition model to obtain the first posterior probability of the audio signal containing emotion, the probability threshold value of the audio signal containing emotion, and the second posterior probabilities corresponding to multiple emotion types, as output by the emotion recognition model, includes:

[0082] The multi-level audio features are input into a dimension reduction linear layer to obtain the multi-level dimension reduction features output by the dimension reduction linear layer; the multi-level dimension reduction features include low-level dimension reduction features, mid-level dimension reduction features, and high-level dimension reduction features;

[0083] The low-level dimensionality reduction features, the mid-level dimensionality reduction features, and the high-level dimensionality reduction features are input into the feature stacking layer to obtain the feature matrix output by the feature stacking layer.

[0084] The low-level dimensionality reduction features, the mid-level dimensionality reduction features, and the high-level dimensionality reduction features are input into the feature concatenation layer to obtain the concatenated feature vector output by the feature concatenation layer.

[0085] The concatenated feature vector is input into the attention linear layer to obtain the attention vector output by the attention linear layer;

[0086] The feature matrix and the attention vector are input into the feature fusion layer to obtain the fusion vector output by the feature fusion layer;

[0087] The fusion vector is input to the output linear layer of the emotion detector to obtain the first posterior probability that the audio signal output by the output linear layer of the emotion detector has an emotion.

[0088] The fusion vector is input to the output linear layer of the emotion probability threshold calculator to obtain the probability threshold value of the audio signal having emotion output by the output linear layer of the emotion probability threshold calculator;

[0089] The fusion vector is input into the output linear layer of the emotion classifier to obtain the second posterior probabilities corresponding to the multiple emotion types output by the output linear layer of the emotion classifier.

[0090] It should be noted that the emotion detector, emotion probability threshold calculator, and emotion classifier in the emotion recognition model share the same model structure, except for their output linear layer. They are all composed of a multilayer perceptron with an attention mechanism. Therefore, the emotion recognition model includes a dimensionality reduction linear layer, a feature concatenation layer, an attention linear layer, a feature stacking layer, a feature fusion layer, and an output linear layer. It can be considered that the emotion detector, emotion probability threshold calculator, and emotion classifier all include these layers, but their output linear layers differ.

[0091] Specifically, in combination Figure 2 , Figure 2 This is a schematic diagram of the structure of the emotion recognition model provided by the present invention. The voice emotion type recognition device inputs multi-level audio features into the dimensionality reduction linear layer of the emotion detector to obtain multi-level dimensionality reduction features output by the dimensionality reduction linear layer of the emotion detector. The multi-level dimensionality reduction features include low-level dimensionality reduction features, mid-level dimensionality reduction features and high-level dimensionality reduction features.

[0092] It should be noted that the dimensionality reduction linear layer consists of two fully connected layers. The low-level, mid-level, and high-level dimensionality reduction features each have their own dedicated dimensionality reduction linear layer. That is, the low-level, mid-level, and high-level dimensionality reduction features will be input into the three dimensionality reduction linear layers respectively, and the parameters of the dimensionality reduction linear layers are not shared.

[0093] Furthermore, the voice emotion type recognition device inputs low-level, mid-level, and high-level dimensionality reduction features into the feature stacking layer of the emotion detector to obtain the feature matrix output by the feature stacking layer of the emotion detector.

[0094] Furthermore, the voice emotion type recognition device inputs low-level, mid-level, and high-level dimensionality reduction features into the feature splicing layer of the emotion detector to obtain the spliced ​​feature vector output by the feature splicing layer of the emotion detector.

[0095] It's important to note that stacking typically involves layering features or network layers sequentially to form a larger feature matrix, without dimensional concatenation. Concatenation, on the other hand, connects multiple features or feature vectors along a specific dimension to create a larger feature vector, resulting in a more dimensionally expanded outcome.

[0096] Furthermore, the voice emotion type recognition device inputs the concatenated feature vector into the attention linear layer of the emotion detector to obtain the attention vector output by the attention linear layer of the emotion detector.

[0097] It should be noted that the attention linear layer is a fully connected layer that outputs an attention vector with a dimension of 3. Each dimension corresponds to the attention weight of the low-level, mid-level, and high-level dimensionality reduction features, respectively. The larger the attention weight, the greater the proportion of the feature in the fusion process.

[0098] Furthermore, the speech emotion type recognition device inputs the feature matrix and attention vector into the feature fusion layer of the emotion detector to obtain the fusion vector output by the feature fusion layer of the emotion detector, wherein the fusion vector is obtained by multiplying the feature matrix and the attention vector.

[0099] It should be noted that the feature processing process of the emotion probability threshold calculator and emotion classifier, which involves dimensionality reduction linear layer, feature splicing layer, attention linear layer, feature stacking layer and feature fusion layer, is consistent with the feature processing process of the emotion detector, which involves dimensionality reduction linear layer, feature splicing layer, attention linear layer, feature stacking layer and feature fusion layer. Therefore, it will not be elaborated here.

[0100] Furthermore, the voice emotion type recognition device inputs the fusion vector output by the feature fusion layer of the emotion detector to the output linear layer of the emotion detector to obtain the first posterior probability that the audio signal output by the output linear layer of the emotion detector contains an emotion.

[0101] Furthermore, the voice emotion type recognition device inputs the fusion vector output by the feature fusion layer of the emotion probability threshold calculator to the output linear layer of the emotion probability threshold calculator to obtain the probability threshold value of the audio signal output by the output linear layer of the emotion probability threshold calculator containing emotion.

[0102] Furthermore, the voice emotion type recognition device inputs the fusion vector output by the feature fusion layer of the emotion classifier into the output linear layer of the emotion classifier to obtain the second posterior probabilities corresponding to the multiple emotion types output by the output linear layer of the emotion classifier.

[0103] It should be noted that the output linear layers of the emotion detector, emotion probability threshold calculator, and emotion classifier have different dimensions depending on their functions. The output dimension of the emotion detector is 1, representing the first posterior probability of the audio signal containing an emotion. The output dimension of the emotion probability threshold calculator is 1, representing the probability threshold value of the audio signal containing an emotion. The output dimension of the emotion classifier is the number of emotion types, representing the second posterior probability corresponding to each of the emotion types.

[0104] In one embodiment, the specific process of inputting multi-level audio features into the emotion recognition model for prediction can be as follows: First, the 1024-dimensional low-level, mid-level, and high-level audio features are each input into their own dedicated linear reduction layer, reducing the 1024-dimensional low-level, mid-level, and high-level audio features to 128 dimensions. Further, the 128-dimensional low-level, mid-level, and high-level dimensionality-reduced features are input into a feature stacking layer, stacking them into a 128*3 feature matrix, where the three columns correspond to the three layers of dimensionality-reduced features. Simultaneously, the 128-dimensional low-level, mid-level, and high-level dimensionality-reduced features are input into a feature concatenation layer, concatenating them into a 384-dimensional concatenated feature vector. Furthermore, the concatenated feature vector of 384 is input into the attention linear layer, outputting an attention vector of dimension 3. Next, the 128*3 feature matrix and the 3-dimensional attention vector are multiplied to obtain a fusion vector of dimension 128. Finally, the 128-dimensional fusion vector is passed through the output linear layer to obtain the classifier's output, which is the output of the emotion detector, emotion probability threshold calculator, or emotion classifier.

[0105] This invention effectively captures emotional information in audio signals by combining steps such as feature dimensionality reduction, feature stacking, feature splicing, attention mechanism, and feature fusion. It can accurately analyze the emotions in audio signals and output the first posterior probability of the presence of emotion in the audio signal, the probability threshold value of the presence of emotion in the audio signal, and the second posterior probability corresponding to multiple emotion types, thereby achieving a significant improvement in the performance of speech emotion type recognition tasks.

[0106] Further, based on step 104, determining the emotion coefficient of the audio signal based on the first posterior probability and the probability threshold value includes:

[0107] The difference between the first posterior probability and the probability threshold value is calculated to obtain the difference.

[0108] The emotional coefficient of the audio signal is obtained by performing an exponential calculation based on the difference.

[0109] Specifically, the voice emotion type recognition device calculates the difference between the first posterior probability and the probability threshold value to obtain the difference. Furthermore, the voice emotion type recognition device performs an exponential calculation based on the difference to obtain the emotion coefficient of the audio signal.

[0110] In one embodiment, the emotion coefficient is calculated as follows:

[0111] confidence = e (probs - gate)

[0112] Here, confidence represents the emotion coefficient, probs represents the first posterior probability, and gate represents the probability threshold.

[0113] As shown in the formula above, the emotion coefficient reflects the confidence level of an audio signal containing emotion. When the first posterior probability of the audio signal containing emotion output by the emotion detector is greater than the probability threshold value output by the emotion probability threshold calculator, the emotion coefficient is greater than 1; otherwise, the emotion coefficient is less than 1. Furthermore, the higher the first posterior probability, the greater the emotion coefficient. Therefore, when the emotion coefficient is greater than 1, the audio signal is judged to contain emotion, while when the emotion coefficient is less than 1, the audio signal is judged to lack emotion, i.e., neutral.

[0114] This invention effectively quantifies the intensity and salience of emotions in audio signals by calculating the difference between the first posterior probability and the probability threshold value and by performing exponential calculation. It can accurately classify the presence or absence of emotions, reflecting a preliminary emotion classification result. Furthermore, it can perform more granular emotion classification based on this emotion classification result, reducing the impact of no-emotion data on emotion categories and helping to improve the accuracy of emotion recognition.

[0115] Further, based on step 105, determining the target emotion type result of the audio signal based on the emotion coefficient and multiple second posterior probabilities includes:

[0116] The multiple second posterior probabilities are multiplied by the emotion coefficient to obtain multiple products;

[0117] The largest product among the plurality of products is determined as the target posterior probability;

[0118] The emotion type corresponding to the target posterior probability is determined as the target emotion type result of the audio signal.

[0119] Specifically, the voice emotion type recognition device multiplies multiple second posterior probabilities with emotion coefficients to obtain multiple products.

[0120] Furthermore, the voice emotion type recognition device compares the numerical values ​​of multiple products to obtain the numerical comparison result.

[0121] Furthermore, the voice emotion type recognition device determines the largest product among multiple products based on the numerical comparison results, and determines the largest product as the target posterior probability.

[0122] Furthermore, the voice emotion type recognition device determines the emotion type corresponding to the target posterior probability as the target emotion type result of the audio signal.

[0123] In one embodiment, taking a five-class emotion recognition task as an example, with neutral, angry, complaining, happy, and praise as examples, the emotion classifier outputs a second posterior probability in five dimensions, corresponding to neutral (probs-n), angry (probs-a), complaining (probs-c), happy (probs-h), and praise (probs-p), respectively. The emotion coefficient confidence is multiplied by probs-a, probs-c, probs-h, and probs-p to obtain a new five-class posterior probability. Then, a softmax layer is passed to obtain the final five-class posterior probability vector, and the emotion type with the highest posterior probability is selected as the recognition result.

[0124] Based on the preliminary emotion classification results, this invention fuses emotion coefficients and multiple second posterior probabilities to achieve finer-grained emotion classification, reducing the impact of confusion between multiple emotion types. By using the maximum target posterior probability, the main emotion type expressed by the audio signal is accurately identified, thus improving the accuracy of speech emotion type recognition.

[0125] Furthermore, the emotion recognition model is trained in the following manner:

[0126] Obtain multi-level audio feature samples;

[0127] Emotion labeling is performed on the multi-level audio feature samples to obtain emotion labeling results; the emotion labeling results include whether an emotion exists or not.

[0128] The multi-level audio feature samples are labeled with emotion types to obtain emotion type labeling results; the emotion type labeling results include multiple emotion types.

[0129] The trained emotion detector is obtained by training based on the multi-level audio feature samples and the emotion annotation results; the output linear layer of the emotion detector is used to predict the first posterior probability that the multi-level audio feature samples contain emotions.

[0130] The trained emotion classifier is obtained by training based on the multi-level audio feature samples and the emotion type labeling results.

[0131] After the emotion detector and the emotion classifier have been trained, a trained emotion probability threshold calculator is obtained based on the multi-level audio feature samples and the emotion annotation results. The output linear layer of the emotion probability threshold calculator is used to predict the probability threshold value of the presence of emotion in the multi-level audio feature samples.

[0132] It should be noted that during model training, the emotion detector and emotion classifier are trained separately first. The emotion detector performs binary classification to identify whether an emotion is present or absent, focusing on the difference between neutral and emotional samples. The training data uses two categories: emotional and emotionless. The emotion classifier performs multi-class emotion recognition, and the training data uses multi-class emotion type labels. The emotion probability threshold calculator is trained after the emotion detector and emotion classifier are trained. During training, the training parameters of the emotion detector and emotion classifier must be fixed, and only the emotion probability threshold calculator is trained.

[0133] Specifically, the voice emotion type recognition device acquires audio signal samples.

[0134] Furthermore, the voice emotion type recognition device performs signal preprocessing on the audio signal samples to obtain preprocessed audio signal samples.

[0135] Furthermore, the voice emotion type recognition device inputs the preprocessed audio signal samples into the audio feature extraction model to obtain multi-level audio feature samples output by the audio feature extraction model.

[0136] Furthermore, the voice emotion type recognition device performs emotion labeling on multi-level audio feature samples to obtain emotion labeling results, which include whether an emotion is present or not.

[0137] Furthermore, the voice emotion type recognition device performs emotion type labeling on multi-level audio feature samples to obtain emotion type labeling results, which include multiple emotion types.

[0138] Furthermore, the voice emotion type recognition device is trained based on multi-level audio feature samples and emotion annotation results to obtain a trained emotion detector.

[0139] Furthermore, the voice emotion type recognition device is trained based on multi-level audio feature samples and emotion type annotation results to obtain a trained emotion classifier.

[0140] Furthermore, after the emotion detector and emotion classifier have completed training, the voice emotion type recognition device is trained based on multi-level audio feature samples and emotion annotation results to obtain a trained emotion probability threshold calculator.

[0141] It should be noted that the output linear layers of the emotion detector, emotion classifier, and emotion probability threshold calculator are set differently. The output linear layer of the emotion detector is used to predict the first posterior probability of the presence of emotion in multi-level audio feature samples. The output linear layer of the emotion classifier is used to predict the second posterior probability corresponding to multiple emotion types. The output linear layer of the emotion probability threshold calculator is used to predict the probability threshold value of the presence of emotion in multi-level audio feature samples.

[0142] This invention utilizes multi-level audio feature samples and combines emotion annotation results and emotion type annotation results for training to obtain a trained emotion detector and emotion classifier. It can effectively predict the posterior probability of the presence of emotion in audio signal samples and the posterior probability of multiple emotion types. Furthermore, by introducing a threshold calculator to automatically calculate the probability threshold value of the presence of emotion, the accuracy of detecting the presence of emotion is improved. This can assist the emotion detector and emotion classifier in refining emotion categories, thereby helping to improve the accuracy of multi-class emotion recognition.

[0143] Furthermore, the present invention also provides a voice emotion type recognition device.

[0144] Reference Figure 3 , Figure 3 This is a schematic diagram of the voice emotion type recognition device provided by the present invention.

[0145] The voice emotion type recognition device is used for:

[0146] Acquisition module 310 is used to acquire audio signals;

[0147] Feature extraction module 320 is used to perform multi-level feature extraction based on the audio signal to obtain multi-level audio features;

[0148] The emotion recognition module 330 is used to input the multi-level audio features into the emotion recognition model to obtain a first posterior probability that the audio signal contains an emotion, a probability threshold value for the audio signal to contain an emotion, and second posterior probabilities corresponding to multiple emotion types, respectively, output by the emotion recognition model. The emotion recognition model includes an emotion detector, an emotion probability threshold calculator, and an emotion classifier. The first posterior probability is predicted by the emotion detector. The probability threshold value is predicted by the emotion probability threshold calculator. The second posterior probabilities corresponding to the multiple emotion types are predicted by the emotion classifier.

[0149] The first determining module 340 is used to determine the emotion coefficient of the audio signal based on the first posterior probability and the probability threshold value; the emotion coefficient is used to reflect the confidence that the audio signal contains emotion.

[0150] The second determining module 350 is used to determine the target emotion type result of the audio signal based on the emotion coefficient and multiple second posterior probabilities.

[0151] The speech emotion type recognition device provided by this invention extracts rich feature information from audio signals through multi-level feature extraction, obtaining multi-level audio features. These multi-level audio features are then input into an emotion recognition model, which can accurately predict the first posterior probability of an emotion in the audio signal, the probability threshold value of an emotion in the audio signal, and the second posterior probabilities corresponding to multiple emotion types. First, an emotion coefficient is determined based on the first posterior probability and the probability threshold value. The emotion coefficient reflects a preliminary emotion classification result. Then, based on this emotion classification result, a finer-grained emotion classification is achieved by combining the emotion coefficient and multiple second posterior probabilities, reducing the impact of confusion between multiple emotion types and thus improving the accuracy of speech emotion type recognition.

[0152] Furthermore, the feature extraction module 320 is also used for:

[0153] The audio signal is preprocessed to obtain the preprocessed audio signal.

[0154] The preprocessed audio signal is input into an audio feature extraction model to obtain multi-level audio features output by the audio feature extraction model; the multi-level audio features include low-level audio features, mid-level audio features, and high-level audio features; the semantic meaning levels of the low-level audio features, mid-level audio features, and high-level audio features are different;

[0155] The audio feature extraction model is obtained by pre-training a model based on the Transformer algorithm.

[0156] Furthermore, the emotion recognition module 330 is also used for:

[0157] The multi-level audio features are input into a dimension reduction linear layer to obtain the multi-level dimension reduction features output by the dimension reduction linear layer; the multi-level dimension reduction features include low-level dimension reduction features, mid-level dimension reduction features, and high-level dimension reduction features;

[0158] The low-level dimensionality reduction features, the mid-level dimensionality reduction features, and the high-level dimensionality reduction features are input into the feature stacking layer to obtain the feature matrix output by the feature stacking layer.

[0159] The low-level dimensionality reduction features, the mid-level dimensionality reduction features, and the high-level dimensionality reduction features are input into the feature concatenation layer to obtain the concatenated feature vector output by the feature concatenation layer.

[0160] The concatenated feature vector is input into the attention linear layer to obtain the attention vector output by the attention linear layer;

[0161] The feature matrix and the attention vector are input into the feature fusion layer to obtain the fusion vector output by the feature fusion layer;

[0162] The fusion vector is input to the output linear layer of the emotion detector to obtain the first posterior probability that the audio signal output by the output linear layer of the emotion detector has an emotion.

[0163] The fusion vector is input to the output linear layer of the emotion probability threshold calculator to obtain the probability threshold value of the audio signal having emotion output by the output linear layer of the emotion probability threshold calculator;

[0164] The fusion vector is input into the output linear layer of the emotion classifier to obtain the second posterior probabilities corresponding to the multiple emotion types output by the output linear layer of the emotion classifier.

[0165] Furthermore, the first determining module 340 is also used for:

[0166] The difference between the first posterior probability and the probability threshold value is calculated to obtain the difference.

[0167] The emotional coefficient of the audio signal is obtained by performing an exponential calculation based on the difference.

[0168] Furthermore, the second determining module 350 is also used for:

[0169] The multiple second posterior probabilities are multiplied by the emotion coefficient to obtain multiple products;

[0170] The largest product among the plurality of products is determined as the target posterior probability;

[0171] The emotion type corresponding to the target posterior probability is determined as the target emotion type result of the audio signal.

[0172] Furthermore, the voice emotion type recognition device is also used for:

[0173] Obtain multi-level audio feature samples;

[0174] Emotion labeling is performed on the multi-level audio feature samples to obtain emotion labeling results; the emotion labeling results include whether an emotion exists or not.

[0175] The multi-level audio feature samples are labeled with emotion types to obtain emotion type labeling results; the emotion type labeling results include multiple emotion types.

[0176] The trained emotion detector is obtained by training based on the multi-level audio feature samples and the emotion annotation results; the output linear layer of the emotion detector is used to predict the first posterior probability that the multi-level audio feature samples contain emotions.

[0177] The trained emotion classifier is obtained by training based on the multi-level audio feature samples and the emotion type labeling results.

[0178] After the emotion detector and the emotion classifier have been trained, a trained emotion probability threshold calculator is obtained based on the multi-level audio feature samples and the emotion annotation results. The output linear layer of the emotion probability threshold calculator is used to predict the probability threshold value of the presence of emotion in the multi-level audio feature samples.

[0179] It should be noted that the voice emotion type recognition device provided by the present invention can execute the voice emotion type recognition method described in any of the above embodiments during specific operation, which will not be elaborated in this embodiment.

[0180] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 4As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 communicate with each other through the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a speech emotion type recognition method. This method includes: acquiring an audio signal; performing multi-level feature extraction based on the audio signal to obtain multi-level audio features; inputting the multi-level audio features into an emotion recognition model to obtain a first posterior probability of the audio signal containing an emotion, a probability threshold value for the audio signal containing an emotion, and second posterior probabilities corresponding to multiple emotion types, all output by the emotion recognition model; the emotion recognition model includes an emotion detector, an emotion probability threshold calculator, and an emotion classifier; the first posterior probability is predicted by the emotion detector; the probability threshold value is predicted by the emotion probability threshold calculator; the second posterior probabilities corresponding to the multiple emotion types are predicted by the emotion classifier; determining the emotion coefficient of the audio signal based on the first posterior probability and the probability threshold value; the emotion coefficient reflects the confidence level that the audio signal contains an emotion; and determining the target emotion type result of the audio signal based on the emotion coefficient and multiple second posterior probabilities.

[0181] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0182] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, wherein when the program instructions are executed by a computer, the computer is able to execute the speech emotion type recognition method provided in the above embodiments, the method comprising: acquiring an audio signal; performing multi-level feature extraction based on the audio signal to obtain multi-level audio features; inputting the multi-level audio features into an emotion recognition model to obtain a first posterior probability of the audio signal having an emotion, a probability threshold value of the audio signal having an emotion, and multiple emotion type classifications output by the emotion recognition model. The emotion recognition model includes an emotion detector, an emotion probability threshold calculator, and an emotion classifier; the first posterior probability is obtained by prediction by the emotion detector; the probability threshold is obtained by prediction by the emotion probability threshold calculator; the second posterior probabilities corresponding to the multiple emotion types are obtained by prediction by the emotion classifier; based on the first posterior probability and the probability threshold, the emotion coefficient of the audio signal is determined; the emotion coefficient is used to reflect the confidence that the audio signal contains an emotion; based on the emotion coefficient and multiple second posterior probabilities, the target emotion type result of the audio signal is determined.

[0183] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the speech emotion type recognition method provided in the above embodiments. The method includes: acquiring an audio signal; performing multi-level feature extraction based on the audio signal to obtain multi-level audio features; inputting the multi-level audio features into an emotion recognition model to obtain a first posterior probability of the audio signal having an emotion, a probability threshold value of the audio signal having an emotion, and second posterior probabilities corresponding to multiple emotion types, all output by the emotion recognition model; the emotion recognition model includes an emotion detector, an emotion probability threshold calculator, and an emotion classifier; the first posterior probability is obtained by prediction by the emotion detector; the probability threshold value is obtained by prediction by the emotion probability threshold calculator; the second posterior probabilities corresponding to the multiple emotion types are obtained by prediction by the emotion classifier; determining an emotion coefficient of the audio signal based on the first posterior probability and the probability threshold value; the emotion coefficient is used to reflect the confidence level of the audio signal having an emotion; and determining the target emotion type result of the audio signal based on the emotion coefficient and multiple second posterior probabilities.

[0184] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0185] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0186] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for recognizing speech emotion types, characterized in that, The method comprises the following steps: obtaining an audio signal; performing multi-level feature extraction based on the audio signal to obtain multi-level audio features; inputting the multi-level audio features into an emotion recognition model to obtain a first posterior probability of the audio signal having an emotion, a probability threshold value of the audio signal having an emotion, and second posterior probabilities of a plurality of emotion types respectively corresponding to the emotion recognition model; the emotion recognition model comprises an emotion detector, an emotion probability threshold calculator, and an emotion classifier; the first posterior probability is obtained by the emotion detector; the probability threshold value is obtained by the emotion probability threshold calculator; and the second posterior probabilities of the plurality of emotion types respectively corresponding to the emotion classifier; determining an emotion coefficient of the audio signal based on the first posterior probability and the probability threshold value; the emotion coefficient is used to reflect the confidence of the audio signal having an emotion; determining a target emotion type result of the audio signal based on the emotion coefficient and the plurality of second posterior probabilities. 2.The voice emotion type recognition method of claim 1, wherein, The method comprises the following steps: performing difference calculation on the first posterior probability and the probability threshold value to obtain a difference value; performing exponential calculation based on the difference value to obtain the emotion coefficient of the audio signal. 3.The voice emotion type recognition method of claim 2, wherein, The method comprises the following steps: multiplying the plurality of second posterior probabilities respectively with the emotion coefficient to obtain a plurality of products; determining a maximum product in the plurality of products as a target posterior probability; determining an emotion type corresponding to the target posterior probability as the target emotion type result of the audio signal. 4.The voice emotion type recognition method of claim 1, wherein, The method comprises the following steps: performing signal preprocessing on the audio signal to obtain a signal-processed audio signal; inputting the signal-processed audio signal into an audio feature extraction model to obtain multi-level audio features output by the audio feature extraction model; the multi-level audio features comprise low-level audio features, middle-level audio features, and high-level audio features; the semantic meaning levels of the low-level audio features, the middle-level audio features, and the high-level audio features are different; wherein the audio feature extraction model is obtained by pre-training a model based on a Transformer algorithm. 5.The voice emotion type recognition method of claim 4, wherein, The emotion recognition model comprises a dimension reduction linear layer, a feature splicing layer, an attention linear layer, a feature stacking layer, a feature fusion layer, and an output linear layer; the method comprises the following steps: inputting the multi-level audio feature into a dimension reduction linear layer to obtain multi-level dimension reduction features output by the dimension reduction linear layer; the multi-level dimension reduction features include low-level dimension reduction features, middle-level dimension reduction features and high-level dimension reduction features; inputting the low-level dimension reduction features, the middle-level dimension reduction features and the high-level dimension reduction features into a feature stacking layer to obtain a feature matrix output by the feature stacking layer; inputting the low-level dimension reduction features, the middle-level dimension reduction features and the high-level dimension reduction features into a feature splicing layer to obtain a spliced feature vector output by the feature splicing layer; inputting the spliced feature vector into an attention linear layer to obtain an attention vector output by the attention linear layer; inputting the feature matrix and the attention vector into a feature fusion layer to obtain a fusion vector output by the feature fusion layer; inputting the fusion vector into an output linear layer of the emotion detector to obtain a first posterior probability that the audio signal exists emotion output by the output linear layer of the emotion detector; inputting the fusion vector into an output linear layer of the emotion probability threshold calculator to obtain a probability threshold value that the audio signal exists emotion output by the output linear layer of the emotion probability threshold calculator; inputting the fusion vector into an output linear layer of the emotion classifier to obtain a second posterior probability corresponding to each of a plurality of emotion types output by the output linear layer of the emotion classifier.

6. The voice emotion type recognition method according to any one of claims 1 to 5, characterized by, The emotion recognition model is obtained by the following method: obtaining multi-level audio feature samples; performing emotion labeling on the multi-level audio feature samples to obtain emotion labeling results; the emotion labeling results include existing emotion or non-existing emotion; performing emotion type labeling on the multi-level audio feature samples to obtain emotion type labeling results; the emotion type labeling results include a plurality of emotion types; training based on the multi-level audio feature samples and the emotion labeling results to obtain a trained emotion detector; an output linear layer of the emotion detector is used to predict a first posterior probability that the multi-level audio feature samples exist emotion; training based on the multi-level audio feature samples and the emotion type labeling results to obtain a trained emotion classifier; after the emotion detector and the emotion classifier are trained, training based on the multi-level audio feature samples and the emotion labeling results to obtain a trained emotion probability threshold calculator; an output linear layer of the emotion probability threshold calculator is used to predict a probability threshold value that the multi-level audio feature samples exist emotion.

7. A voice emotion type recognition apparatus characterized by comprising: comprise: an acquisition module configured to acquire an audio signal; a feature extraction module configured to perform multi-level feature extraction based on the audio signal to obtain multi-level audio features; The emotion recognition module is configured to input the multi-level audio features into an emotion recognition model to obtain a first posterior probability of the audio signal having an emotion, a probability threshold value of the audio signal having an emotion, and second posterior probabilities of a plurality of emotion types respectively corresponding to the emotion recognition model output; the emotion recognition model comprises an emotion detector, an emotion probability threshold calculator, and an emotion classifier; the first posterior probability is obtained by the emotion detector; the probability threshold value is obtained by the emotion probability threshold calculator; and the second posterior probabilities of the plurality of emotion types respectively corresponding to the emotion classifier. The first determination module is configured to determine an emotion coefficient of the audio signal based on the first posterior probability and the probability threshold value. The emotion coefficient is used to reflect a confidence degree of the audio signal having an emotion. The second determination module is configured to determine a target emotion type result of the audio signal based on the emotion coefficient and the second posterior probabilities.

8. An apparatus comprising an electronic device, the electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein, The processor executes the computer program to implement the steps of the speech emotion type recognition method according to any one of claims 1 to 6.

9. A medium comprising a non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the speech emotion type recognition method according to any one of claims 1 to 6.

10. A product comprising a computer program product comprising a computer program, characterized in that The computer program is executed by the processor to implement the steps of the speech emotion type recognition method according to any one of claims 1 to 6.

Images