Dimensional speech emotion recognition method, computer device, medium, product

Abstract

The present application discloses a dimensional speech emotion recognition method, computer device, medium, and product. The method includes: based on the speech to be recognized, obtaining the pronunciation features of the target speech; extracting the auditory modulation cochlear map features from the speech to be recognized based on a preset extraction strategy; analyzing the speech to be recognized based on a preset model and extracting the pre-trained model features; performing time-series encoding on the pronunciation features of the target speech, the auditory modulation cochlear map features, and the pre-trained model features to form three types of encoding features, and then performing weighted processing to obtain three types of weighted features; then performing multi-task recognition to obtain discrete emotion recognition information and dimensional emotion initial recognition information; processing these two types of information based on a preset loss calculation strategy to obtain dimensional speech emotion target recognition results. This method can effectively narrow the gap between the representation space and the actual emotion space in the prior art, and through comprehensive and detailed analysis, accurate and reliable emotion recognition results can be obtained.

Description

Technical Field

[0001] The present disclosure relates to the field of emotion recognition technology, and in particular to a dimensional speech emotion recognition method, computer device, medium, and product. Background Art

[0002] Speech emotion recognition technology, as a key branch in the field of artificial intelligence,

[0003] Aiming to analyze voice information through computers to understand and identify people's emotional states, this technology has been widely used in many fields such as intelligent interaction, security and financial services.

[0004] At present, speech emotion recognition is mainly based on two modeling methods: one is to extract and analyze acoustic features using tools such as openSMILE to extract multidimensional feature parameters. Although this method can extract rich speech features, these high-dimensional feature sets often contain a lot of redundant information and have limited ability to effectively represent emotions, resulting in a significant reduction in the emotion recognition rate on untrained data.

[0005] The second is to use deep learning technology and pre-trained models such as Wav2vec2.0 and Hubert to automatically learn the representation form of speech signals from massive speech data, and then apply them to emotion recognition tasks. Although the pre-trained model can learn deeper emotional information, its representation space is essentially different from the emotional space understandable to humans and lacks interpretability, resulting in inaccurate and unreliable emotion recognition results. Summary of the invention

[0006] In view of this, the embodiments of the present disclosure provide a dimensional speech emotion recognition method, computer device, medium, and product, which can solve the problems of poor accuracy and unreliability of speech dimensional emotion recognition results existing in the prior art.

[0007] In a first aspect, the present disclosure provides a method for dimensional speech emotion recognition, including the following scheme:

[0008] Based on the speech to be recognized, obtaining the pronunciation features of the target speech, wherein the pronunciation features of the target speech include target initial glottal wave characteristic parameters and target vocal tract filter characteristic parameters;

[0009] Extracting auditory modulation cochleogram features from the speech to be recognized based on a preset extraction strategy;

[0010] Analyze the speech to be recognized based on the preset model and extract the pre-trained model features;

[0011] The target speech pronunciation feature, the auditory modulation cochlear map feature, and the pre-trained model feature are respectively time-series encoded to form three types of encoding features;

[0012] Performing weighted processing on the three types of coding features to obtain three types of weighted features;

[0013] Perform multi-task recognition on the three types of weighted features to obtain discrete emotion recognition information and dimensional emotion initial recognition information;

[0014] The discrete emotion recognition information and the dimensional emotion initial recognition information are processed based on a preset loss calculation strategy to obtain a dimensional speech emotion target recognition result.

[0015] Optionally, obtaining the pronunciation features of the target speech based on the speech to be recognized includes:

[0016] Analyze the speech to be recognized to obtain initial glottal wave characteristic parameters and initial vocal tract filter characteristic parameters;

[0017] Based on the initial glottal wave characteristic parameters and the initial vocal tract filter characteristic parameters, Q synthesized speech is obtained, Q≥2;

[0018] Obtaining Q difference values ​​between the synthesized speech and the speech to be recognized;

[0019] Based on the Q difference values, obtaining a minimum mean square error;

[0020] Based on the minimum mean square error, determining the corresponding initial glottal wave characteristic parameters as target initial glottal wave characteristic parameters;

[0021] The initial vocal channel filter characteristic parameters are used as target vocal channel filter characteristic parameters.

[0022] Optionally, obtaining Q synthesized speech sounds based on the initial glottal wave characteristic parameters and the initial vocal tract filter characteristic parameters includes:

[0023] Processing the initial glottal wave characteristic parameters Q times based on a preset loop strategy to obtain Q glottal waveform signals;

[0024] Based on the initial vocal tract filter characteristic parameters, convolution operations are performed on the Q glottal waveform signals to obtain Q synthesized speech;

[0025] The qth synthesized speech is S2 q (n):

[0026] S2 q (n)= ;

[0027] 1≤q≤Q, 1≤ ≤ ;

[0028] in, The speech to be recognized is The sampling point at time, For the current moment, is the filter order, is the channel filter coefficient, is the coefficient for adjusting the amplitude of the glottal wave, u q (n) is the qth glottal waveform signal.

[0029] Optionally, the extracting the auditory modulation cochleogram feature from the speech to be recognized based on a preset extraction strategy includes:

[0030] Decompose the speech to be recognized into channel signal of acoustic frequencies, >1;

[0031] Extracting the time envelope of each of the channel signals;

[0032] Each of the time envelopes is modulated to generate The characteristic signal of the modulation frequency;

[0033] based on The characteristic signal generates modulation units;

[0034] right The modulation units perform convolution operations respectively to obtain Cochleogram features of auditory modulation.

[0035] Optionally, the modulated cochleogram feature is :

[0036] = ;

[0037] ;

[0038] , ;in, is the Hamming window, is the number of samples in each time window, It is a voice signal length of time, It is frame shift. For the The starting position of the time window is the j-1th frame shift position.

[0039] Optionally, analyzing the speech to be recognized based on a preset model and extracting pre-trained model features includes: analyzing the speech to be recognized based on a Wav2vec2.0 pre-trained model and extracting a preset number of speech depth features;

[0040] A preset number of the speech depth features constitute the pre-trained model features.

[0041] Optionally, the discrete emotion recognition information and the dimensional emotion initial recognition information are processed based on a preset loss calculation strategy to obtain a dimensional speech emotion target recognition result, including:

[0042] Processing the discrete emotion recognition information based on the emotion category prediction loss function to obtain a first error value;

[0043] The emotion category prediction loss function is: ; where x h is the predicted value corresponding to the discrete emotion recognition information of the hth category, y h is the true value corresponding to the h-th discrete emotion, is the corresponding emotion type in the discrete emotion recognition information;

[0044] Processing the dimensional emotion initial recognition information based on the dimensional emotion prediction loss function to obtain a second error value;

[0045] The dimensional sentiment prediction loss function is: ;

[0046] in, is the Pearson correlation coefficient, is the variance of the predicted values ​​corresponding to all samples of the same dimension in the initial recognition information of the dimensional emotion, is the variance of the true values ​​corresponding to all samples of the same dimension in the initial recognition information of the dimensional emotion, is the average of the predicted values ​​corresponding to all samples of the same dimension in the initial recognition information of the dimension emotion, is the average value of the true values ​​corresponding to all samples of the same dimension in the initial recognition information of the dimensional emotion;

[0047] Obtaining an overall error value based on the first error value and the second error value;

[0048] The overall error value is: ;in, is the weight used to adjust the dimension sentiment loss and sentiment category loss;

[0049] When the overall error value is less than a preset error, the dimensional emotion initial recognition information is used as the dimensional speech emotion target recognition result;

[0050] When the overall error value is not less than the preset error, the weighted features are updated by using a back propagation algorithm for discrete emotion and dimensional emotion multi-task recognition to obtain updated discrete emotion recognition information and dimensional emotion initial recognition information;

[0051] Based on the updated discrete emotion recognition information and dimensional emotion initial recognition information, an updated overall error value is obtained, and the updated dimensional emotion initial recognition information corresponding to the overall error value that meets the conditions is used as the dimensional speech emotion target recognition result.

[0052] In a second aspect, the present disclosure also provides a dimensional speech emotion recognition system, including:

[0053] A target speech pronunciation feature acquisition module, used for acquiring the target speech pronunciation features based on the speech to be recognized, wherein the target speech pronunciation features include target initial glottal wave feature parameters and target vocal tract filter feature parameters;

[0054] An auditory modulation cochlear map feature acquisition module, used for extracting auditory modulation cochlear map features from the speech to be recognized based on a preset extraction strategy;

[0055] A pre-trained model feature acquisition module is used to analyze the speech to be recognized based on a preset model and extract pre-trained model features;

[0056] An encoding module, used for performing time-series encoding on the target speech pronunciation feature, the auditory modulation cochlear map feature, and the pre-trained model feature to form three types of encoding features;

[0057] A weighting module, used for performing weighted processing on the three types of coding features to obtain three types of weighted features;

[0058] A recognition module, used for performing multi-task recognition on the three types of weighted features to obtain discrete emotion recognition information and dimensional emotion initial recognition information;

[0059] The analysis module is used to process the discrete emotion recognition information and the dimensional emotion initial recognition information based on a preset loss calculation strategy to obtain a dimensional speech emotion target recognition result.

[0060] In a third aspect, the embodiments of the present disclosure further provide a computer device, which adopts the following technical solution:

[0061] The computer device comprises:

[0062] at least one processor; and,

[0063] a memory communicatively connected to the at least one processor; wherein,

[0064] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any of the above-mentioned dimensional speech emotion recognition methods.

[0065] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute any of the above-mentioned dimensional speech emotion recognition methods.

[0066] In a fifth aspect, an embodiment of the present disclosure further provides a computer program product, including a computer program / instruction, which implements the steps of any of the above methods when executed by a processor.

[0067] The dimensional speech emotion recognition method disclosed in the present application can capture the physical pronunciation characteristics of speech by acquiring the target initial glottal wave characteristic parameters and the target vocal tract filter characteristic parameters. These characteristics provide detailed information about the speech production process and help to more accurately understand the emotions in the speech; based on a preset extraction strategy, auditory modulation cochlear map features are extracted from the speech to be recognized, simulating the processing method of the human auditory system, and can capture emotional information that is closer to human perception; based on a preset model, the speech to be recognized is analyzed and pre-trained model features are extracted, and the emotional information already learned by the deep learning model can be used to improve the understanding and recognition capabilities of emotions; the target speech pronunciation features, auditory modulation cochlear map features, and pre-trained model features are time-series encoded and weighted It can better integrate different types of feature information, reduce the interference of redundant information, and improve the accuracy and robustness of emotion recognition. Temporal coding helps to capture the dynamic changes of speech features and reflect the temporal changes of emotions. By performing multi-task recognition on weighted features, not only discrete emotion recognition information can be obtained, but also dimensional emotion initial recognition information can be obtained. This multi-task learning method can pay attention to the specific category of emotion and the dimensional information of emotion at the same time, making the recognition results more comprehensive and detailed. Based on the preset loss calculation strategy, discrete emotion recognition information and dimensional emotion initial recognition information are processed, which can effectively optimize the model parameters, greatly narrow the gap between the representation space and the actual emotion space in the existing technology, and ensure that the acquired emotion recognition results are accurate and reliable.

[0068] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the following preferred embodiments are specifically cited and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0070] Figure 1 A flowchart of a dimensional speech emotion recognition method provided in an embodiment of the present disclosure.

[0071] Figure 2 A flowchart of a method for acquiring pronunciation features of a target speech provided in an embodiment of the present disclosure.

[0072] Figure 3 A flowchart of a method for acquiring synthesized speech provided in an embodiment of the present disclosure.

[0073] Figure 4 A schematic flow chart of a method for extracting auditory modulation cochlear map features provided in an embodiment of the present disclosure.

[0074] Figure 5 A flowchart of a method for processing discrete emotion recognition information and dimensional emotion initial recognition information based on a preset loss calculation strategy provided in an embodiment of the present disclosure.

[0075] Figure 6 A schematic flow chart of a method for obtaining a first error value provided in an embodiment of the present disclosure.

[0076] Figure 7 A schematic flow chart of a method for obtaining a second error value provided in an embodiment of the present disclosure.

[0077] Figure 8 A schematic diagram of the structure of a computer device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0078] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0079] It should be clear that the following embodiments of the present disclosure are described by specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other in the absence of conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present disclosure.

[0080] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.

[0081] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The drawings only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0082] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects described may be practiced without these specific details.

[0083] Reference Figure 1 The first aspect of the present application discloses a dimensional speech emotion recognition method, which specifically comprises the following steps:

[0084] S100, based on the speech to be recognized, obtaining pronunciation features of a target speech, the pronunciation features of the target speech including target initial glottal wave feature parameters and target vocal tract filter feature parameters.

[0085] Extracting the pronunciation features of the target speech helps to understand the underlying acoustic properties of the speech; glottal waves and vocal tract filter features can provide detailed information about the speech pronunciation quality and speech production process, thus facilitating more accurate sentiment analysis.

[0086] S200, extracting auditory modulation cochleogram features from the speech to be recognized based on a preset extraction strategy.

[0087] In this step, the extracted auditory modulation cochleogram features can provide information about the auditory perception level of speech, help better understand the perceptual characteristics of speech signals, and make emotion recognition more accurate.

[0088] Based on the speech to be recognized, the pronunciation characteristics of the target speech and the auditory modulation cochlear map characteristics are obtained, which can mine the strong discriminative emotional feature representation related to emotions.

[0089] S300, analyzing the speech to be recognized based on a preset model and extracting pre-trained model features.

[0090] In this step, the preset model can learn rich speech features from large-scale corpus, that is, it can obtain representations that effectively express emotions, improve the accuracy and efficiency of feature extraction, and effectively improve the robustness and versatility of the emotion recognition system.

[0091] Specifically, the speech to be recognized is analyzed based on the Wav2vec2.0 pre-trained model, and a preset number of speech depth features are extracted; the preset number of speech depth features constitute the pre-trained model features.

[0092] Among them, the preset number is 768 dimensions.

[0093] Furthermore, the Wav2vec2.0 pre-trained model is first pre-trained on a large amount of unlabeled speech data, which does not require manual labeling, such as audio clips that have not been transcribed into text; the model learns useful representations (features) from the original audio through self-supervised learning. For example, it learns how to predict masked audio parts to understand the structure and characteristics of speech; after pre-training, the Wav2vec2.0 pre-trained model will be fine-tuned on a small amount of labeled speech data. These data are speech samples that have been transcribed into text, that is, audio data containing text labels; during the fine-tuning process, the model adjusts its weights according to the labeled text data to improve the performance of speech recognition tasks.

[0094] Semi-supervised learning methods use a small amount of labeled data and a large amount of unlabeled data for training. Although these methods also perform well in speech recognition tasks, Wav2vec 2.0 can achieve better performance with less labeled data through more effective self-supervised learning strategies and pre-training. The pre-training phase of Wav2vec 2.0 enables the model to learn richer speech representations, so that it can better utilize the small amount of labeled data in the subsequent fine-tuning phase. In tasks such as speech recognition, Wav2vec 2.0 has been found to outperform many existing semi-supervised learning methods, which means that it can provide higher accuracy and better results when dealing with actual speech recognition problems.

[0095] S400, performing temporal coding on the target speech pronunciation features, the auditory modulation cochlear map features, and the pre-trained model features respectively to form three types of coding features.

[0096] Specifically, a long short-term memory network (LSTM) is used to encode the target speech pronunciation features, auditory modulation cochlear map features, and pre-trained model features to form encoded features (i.e., output features); LSTM can learn temporal dependencies and contextual information by processing these sequence data, which means that LSTM can take into account the influence of the context in the emotion recognition task, thereby more accurately identifying the changes in emotion. The output of LSTM will contain the encoding of the contextual information in the input sequence, which helps capture the changing pattern of emotion in speech. These context-related features can be used for further emotion classification or other related tasks. By using LSTM to encode the input features in multi-task learning, the emotional changes in speech sequences can be effectively processed. LSTM can make full use of contextual information to help understand and identify the dynamic changes of emotion in time series. This method can improve the accuracy of emotion recognition and enable better capture of subtle changes in emotion in speech.

[0097] In this step, time series coding can capture the dynamic change characteristics in the speech signal, improve the modeling ability of time series data, enhance the understanding of the temporal dependencies in the speech signal, and help identify changes in emotional state.

[0098] S500, weighted processing is performed on the three types of coding features to obtain three types of weighted features.

[0099] Specifically, an attention mechanism can be introduced to take all time series output features (i.e., encoding features) as input features of the attention layer. The attention layer will weight these input features and calculate the importance score of each feature, so that different tasks can learn feature representations that are more suitable for the task and improve the performance of dimensional emotion recognition.

[0100] In this step, by weighting the features of different tasks, it is possible to emphasize important feature information, reduce the impact of noise on emotion recognition, and increase the sensitivity to key features, thereby enhancing the accuracy of emotion recognition.

[0101] S600, performing multi-task recognition on the three types of weighted features to obtain discrete emotion recognition information and dimensional emotion initial recognition information.

[0102] Specifically, a multi-task learning model is trained to simultaneously recognize discrete emotions and dimensional sentiment information, which can be achieved through a multi-task neural network.

[0103] In this embodiment, the discrete emotion recognition information may include one or more emotions of joy, anger, sadness, and happiness, or may include one or more emotions of anger, fear, happiness, neutrality, sadness, and surprise. The dimensional emotion initial recognition information preferably includes one or more of pleasure and excitement.

[0104] In this step, multi-task learning can share feature representations, improve the generalization ability and recognition performance of the model, and simultaneously identify sentiment information at different levels to provide more comprehensive sentiment analysis results.

[0105] S700, based on a preset loss calculation strategy, discrete emotion recognition information and dimensional emotion initial recognition information are processed to obtain a dimensional speech emotion target recognition result.

[0106] By processing discrete emotion recognition information and dimensional emotion initial recognition information using a preset loss calculation strategy, the accuracy of emotion recognition can be improved, ensuring that the final emotion recognition results have good reliability and accuracy.

[0107] The dimensional speech emotion recognition method disclosed in the present application can capture the physical pronunciation characteristics of speech by acquiring the target initial glottal wave characteristic parameters and the target vocal tract filter characteristic parameters. These characteristics provide detailed information about the speech production process and help to more accurately understand the emotions in the speech; based on a preset extraction strategy, auditory modulation cochlear map features are extracted from the speech to be recognized, simulating the processing method of the human auditory system, and can capture emotional information that is closer to human perception; based on a preset model, the speech to be recognized is analyzed and pre-trained model features are extracted, and the emotional information already learned by the deep learning model can be used to improve the understanding and recognition capabilities of emotions; the target speech pronunciation features, auditory modulation cochlear map features, and pre-trained model features are time-series encoded and weighted It can better integrate different types of feature information, reduce the interference of redundant information, and improve the accuracy and robustness of emotion recognition. Temporal coding helps to capture the dynamic changes of speech features and reflect the temporal changes of emotions. By performing multi-task recognition on weighted features, not only discrete emotion recognition information can be obtained, but also dimensional emotion initial recognition information can be obtained. This multi-task learning method can pay attention to the specific category of emotion and the dimensional information of emotion at the same time, making the recognition results more comprehensive and detailed. Based on the preset loss calculation strategy, discrete emotion recognition information and dimensional emotion initial recognition information are processed, which can effectively optimize the model parameters, greatly narrow the gap between the representation space and the actual emotion space in the existing technology, and ensure that the acquired emotion recognition results are accurate and reliable.

[0108] The acoustic feature extraction methods in the prior art have problems of redundant information and limited emotion representation capabilities. Although deep learning models can learn deep emotional information, they lack interpretability. The method of the present application integrates the target speech pronunciation features, auditory modulation cochlear map features, and pre-trained model features, and combines temporal coding, multi-task recognition and optimized loss calculation strategies to effectively improve the accuracy and reliability of speech emotion recognition, while overcoming the main defects in the prior art and providing more accurate and interpretable emotion recognition results.

[0109] Reference Figure 2 , a method for acquiring the pronunciation features of a target speech, specifically comprising the following steps:

[0110] S110, analyzing the speech to be recognized, and obtaining initial glottal wave characteristic parameters and initial vocal tract filter characteristic parameters.

[0111] Specifically, speech signal processing techniques (such as Fourier transform, wavelet transform, etc.) can be used to analyze the frequency components, energy distribution and other information of the speech to be recognized s1(n), and at the same time extract the initial glottal wave characteristic parameters and the initial vocal tract filter characteristic parameters. ; In this step, the important parameters of the speech signal are directly extracted through frequency domain analysis, providing basic data for subsequent speech synthesis and difference calculation.

[0112] The initial glottal wave characteristic parameters include the maximum glottal opening time T p , glottis instantaneous closing time T e , Glottal return time T a , glottis completely closed T c , the amplitude E at the instantaneous closing of the glottis e , the glottis opens and closes for one cycle T0.

[0113] Specifically, ; ; ; Most of the time, , , is the fundamental frequency.

[0114] S120, obtaining Q synthesized speech sounds based on the initial glottal wave characteristic parameters and the initial vocal tract filter characteristic parameters, where Q≥2.

[0115] These synthesized speech sounds simulate the sound characteristics of the speech to be recognized to a certain extent, but the parameter values ​​are changed to different degrees to simulate different pronunciations or vocal cord vibrations.

[0116] Further reference Figure 3 , a method for obtaining synthesized speech, specifically comprising the following steps:

[0117] S121, processing the initial glottal wave characteristic parameters Q times based on a preset loop strategy to obtain Q glottal waveform signals.

[0118] Specifically, starting from the second treatment, the maximum glottal opening time T p , glottis instantaneous closing time T e , Glottal return time T a , glottis completely closed T c , the amplitude E at the instantaneous closing of the glottis e , the six parameters of glottis opening and closing cycle T0 are adjusted as variables to obtain different waveform signals.

[0119] Specifically, these processes can be performed using speech synthesis tools or simulation software, such as the signal processing toolbox of MATLAB, or specialized speech synthesis software.

[0120] Through multiple processing, a series of different glottal waveform signals can be generated, which provides rich basic data for the subsequent synthesis process. This diverse signal can better simulate the changes in actual speech. At the same time, the generated multiple glottal waveform signals help the model capture more pronunciation changes and improve the matching degree and robustness of the synthesized speech with the real speech.

[0121] S122, performing convolution operations on the Q glottal waveform signals based on the initial vocal tract filter characteristic parameters to obtain Q synthesized speech sounds.

[0122] Specifically, the qth synthesized speech is S2 q (n):

[0123] S2 q (n)= ;

[0124] 1≤q≤Q, 1≤ ≤ .

[0125] in, The speech to be recognized is The sampling point at time, For the current moment, is the filter order, is the vocal tract filter coefficient (i.e., the initial vocal tract filter characteristic parameter), is the coefficient for adjusting the amplitude of the glottal wave, u q (n) is the qth glottal waveform signal.

[0126] The Q glottal waveform signals obtained in S121 are convolved using the initial vocal tract filter characteristic parameters. The convolution operation simulates the filtering effect of the vocal tract, and the glottal waveform signal is combined with the vocal tract filter characteristic parameters to generate synthetic speech. This process can be performed in the time domain or the frequency domain through a convolution operation.

[0127] By convolving the glottal waveform signal with the characteristic parameters of the vocal tract filter, high-fidelity synthetic speech can be generated. This processing method more accurately simulates the vocal tract characteristics of the actual pronunciation process. The convolution operation can effectively apply the characteristics of the vocal tract filter to the glottal waveform, and the generated synthetic speech is closer to the pronunciation characteristics of real speech, thereby improving the accuracy of emotion recognition.

[0128] The method for obtaining synthesized speech disclosed in this embodiment can generate various simulated speech by processing the initial glottis waveform multiple times and convolving it with the characteristic parameters of the vocal tract filter. This method provides diversity and details close to real speech, which helps to capture more accurate pronunciation features in speech analysis; the Q synthesized speech generated provides rich sample data for subsequent feature analysis and optimization, which helps to improve the accuracy and reliability of features in the modeling and recognition process; by accurately simulating the characteristics of the glottis and vocal tract, the synthesized speech can better reflect the real pronunciation of the speech, thereby improving the performance of the emotion recognition system in processing complex emotion patterns; multiple processing and convolution operations enhance the model's adaptability to different pronunciation variations, so that it can better handle changes in various actual speech data in practical applications.

[0129] In summary, steps S121 and S122 help generate more accurate and reliable synthesized speech, which is of great significance for improving the effect and stability of speech emotion recognition.

[0130] S130, obtaining difference values ​​between Q synthesized speech and the speech to be recognized.

[0131] Specifically, the difference value is , that is, the difference between the qth synthesized speech and the speech to be recognized is:

[0132] =s1(n)-s2(n)=s1(n) .

[0133] This difference value can reflect the degree of closeness between the synthesized speech and the original speech; by quantifying the difference between the two, we can intuitively understand the impact of different parameter combinations on the speech, which helps to screen out the parameter combination that is closest to the original speech.

[0134] S140: Obtain a minimum mean square error based on the Q difference values.

[0135] Specifically, from the Q generated synthesized speech, find and determine the mean square error with the smallest difference from the original speech to be recognized. This is the result obtained after comparing the Q difference values. The minimum mean square error can reflect the situation where the synthesized speech is closest to the original speech, providing a basis for the subsequent extraction of accurate target feature parameters.

[0136] S150, based on the minimum mean square error, determining the corresponding initial glottal wave characteristic parameters as target initial glottal wave characteristic parameters; and using the initial vocal tract filter characteristic parameters as target vocal tract filter characteristic parameters.

[0137] Based on the minimum mean square error determined above, the initial glottal wave feature parameters corresponding to the synthesized speech are selected as the target initial glottal wave feature parameters; the feature parameters obtained in this way are closest to the original speech to be recognized, thereby improving the accuracy of feature extraction in subsequent emotion recognition tasks.

[0138] The method for acquiring the target speech pronunciation features disclosed in this embodiment, i.e., the method disclosed in S110-S150, can reduce the emotion recognition error caused by inaccurate parameters through a refined feature parameter acquisition process; through the method of synthesizing speech, it can more comprehensively explore the influence of different parameters on speech features, thereby improving the accuracy of feature selection; the determination of the minimum mean square error provides a quantitative standard for the extraction of feature parameters, thereby enhancing the interpretability and reliability of the features; the precise and reliable feature parameters finally obtained help to improve the accuracy and robustness of speech emotion recognition.

[0139] Reference Figure 4 , a method for extracting auditory modulation cochleogram features, specifically comprising the following steps:

[0140] S210, decomposing the speech to be recognized into channel signals of n acoustic frequencies, where n>1.

[0141] Specifically, a cochlear filter can be used to simulate the basilar membrane of the cochlea to decompose the speech signal into n frequency band signals (i.e., channel signals) of different acoustic frequencies. Each frequency band signal represents the sound energy within a specific frequency range. These filter groups usually use Mel filter groups in wavelet transform, short-time Fourier transform (STFT) or Mel frequency cepstral coefficient (MFCC).

[0142] Through this step, the spectral characteristics of the speech signal can be captured, which is helpful for subsequent analysis. Signals in different frequency bands can reveal different auditory characteristics and represent the speech information more comprehensively.

[0143] S220, extracting the time envelope of each channel signal.

[0144] Specifically, envelope detection technology is applied to each frequency band signal, such as obtaining the time envelope through Hilbert transform, or smoothing the signal using a sliding average method to obtain the envelope.

[0145] The time envelope represents the main changing trend of the signal, which can reduce the interference of high-frequency noise and better capture the low-frequency characteristics and dynamic changes of the signal, which is very important for the recognition of speech signals.

[0146] S230, modulate each time envelope to generate characteristic signals of m modulation frequencies.

[0147] Specifically, a modulation filter can be used to modulate the time envelope signal with a set of sine waves of modulation frequency to generate characteristic signals at different modulation frequencies, which can be obtained by multiplying the time envelope signal. The modulation process can extract different frequency components in the time envelope, reflecting the modulation characteristics of the sound. The modulated signal can be more robust to different sound intensities and frequency changes.

[0148] S240, generating m×n modulation units based on the m×n characteristic signals.

[0149] Specifically, no. The sub-band modulation unit of the modulation unit is :

[0150] ;

[0151] , .

[0152] Among them, The modulation unit is The channel signal The modulation unit corresponding to the characteristic signal, is the Hamming window, is the number of samples in each time window, It is a voice signal length of time, It is frame shift. For the The starting position of the time window is the j-1th frame shift position.

[0153] Specifically, represents the period starting from the (j-1)th frame shift position and extending backward to t w The signal feature matrix of the original signal fragment of samples, that is, it is the signal feature matrix in the jth time window (position in the time series).

[0154] Further, represents the windowed signal in the jth time window, which is the result of multiplying the original signal by the Hamming window. =α−βcos(2π / (N−1)) where N is the length of the window function, α is usually 0.54, and β is 1- α. By adding a window, the spectrum leakage of the signal can be reduced and the accuracy of spectrum analysis can be improved.

[0155] S250, performing convolution operations on the m×n modulation units respectively to obtain m×n auditory modulation cochleogram features.

[0156] The modulated cochleogram features are :

[0157] = .

[0158] The method for extracting auditory modulation cochleogram features disclosed in this embodiment can more comprehensively capture important information in speech signals through multi-level feature extraction (frequency decomposition, time envelope, modulation, convolution); modulation and convolution operations can improve the robustness to noise and variation, making the recognition system more stable in different environments; fine-grained feature extraction and learning can improve the accuracy of speech recognition, allowing the system to recognize more complex speech patterns. This method simulates the auditory processing process of the human ear, can effectively extract useful information from complex audio signals, and improve the performance of the speech recognition system.

[0159] In a preferred embodiment, using , , =40ms, and finally a 288-dimensional auditory modulation cochleogram feature is obtained.

[0160] In the prior art, in terms of auditory features, commonly used auditory spectral features such as Mel-frequency cepstral coefficients (MFCC) and FBanks features only consider short-term time-frequency information and ignore the characteristics of emotion changing over time. To compensate for this deficiency, the present application adopts auditory modulation cochlear map features to capture the time-spectral changes of speech sequences and improve the ability of speech emotion recognition.

[0161] Reference Figure 5 , based on the preset loss calculation strategy, the discrete emotion recognition information and the dimensional emotion initial recognition information are processed to obtain the dimensional speech emotion target recognition result, that is, the method for obtaining the dimensional speech emotion target recognition result specifically includes the following steps:

[0162] S710, processing discrete emotion recognition information based on the emotion category prediction loss function to obtain a first error value.

[0163] Through this step, accurate results of analyzing discrete emotion recognition information can be obtained.

[0164] In this embodiment, refer to Figure 6 S710 further includes the following steps, that is, the method for obtaining the first error value specifically includes the following steps:

[0165] S711, based on the discrete emotion recognition information and a first preset discrete emotion database, obtaining discrete prediction values ​​corresponding to all discrete emotions in the discrete emotion recognition information.

[0166] Specifically, the first preset discrete emotion database contains mapping relationships between all different discrete emotions and preset prediction values. Therefore, the corresponding discrete prediction value can be found in the first preset discrete emotion database according to the discrete emotion recognition information.

[0167] S712, obtaining all discrete emotion categories in the discrete emotion recognition information, and based on the discrete emotion categories and a second preset discrete emotion database, obtaining discrete true values ​​corresponding to all discrete emotion categories.

[0168] Specifically, the second preset discrete emotion database contains mapping relationships between all different discrete emotions and preset true values. Therefore, the corresponding discrete true values ​​can be found in the second preset discrete emotion database according to the discrete emotion recognition information.

[0169] S713, obtaining a first error value based on all discrete predicted values, all discrete true values, and the sentiment category prediction loss function.

[0170] Specifically, the emotion category prediction loss function is: .

[0171] Among them, x h is the discrete prediction value corresponding to the h-th discrete emotion recognition information, y h is the discrete true value corresponding to the hth discrete emotion, is the corresponding emotion type in the discrete emotion recognition information, is the first error value.

[0172] The calculation of the first error value provides a quantitative indicator of the current performance of the model, which can intuitively display the accuracy of the model in discrete sentiment classification tasks, facilitating subsequent model optimization and parameter adjustment.

[0173] S720, processing the dimensional emotion initial recognition information based on the dimensional emotion prediction loss function to obtain a second error value.

[0174] Through this step, accurate results of analyzing the initial recognition information of dimensional emotions can be obtained.

[0175] Specifically, the dimensional sentiment prediction loss function is: .

[0176] in, is the Pearson correlation coefficient, is the variance of the predicted values ​​corresponding to all samples of the same dimension in the initial recognition information of dimensional emotion, is the variance of the true values ​​of all samples of the same dimension in the initial recognition information of dimensional emotion, is the average of the predicted values ​​corresponding to all samples of the same dimension in the initial recognition information of dimensional emotion, It is the average value of the true values ​​of all samples of the same dimension in the initial recognition information of dimensional emotion. is the error value corresponding to the f-th dimension (i.e., the second error value).

[0177] S730, obtaining an overall error value based on the first error value and the second error value.

[0178] The overall error value is : .

[0179] in, is the weight used to adjust the dimension sentiment loss and sentiment category loss.

[0180] By analyzing the discrete emotion recognition information and the dimensional emotion initial recognition information of all dimensions, that is, using discrete information for assistance and combining all dimensions, accurate multi-dimensional recognition can be achieved. That is, by comprehensively considering the errors of discrete emotions and dimensional emotions, a more comprehensive performance evaluation can be obtained, thereby improving the generalization of emotions.

[0181] S740, when the overall error value is less than the preset error, taking the dimensional emotion initial recognition information as the dimensional speech emotion target recognition result;

[0182] When the overall error value is not less than the preset error, the back propagation algorithm is used to update the weighted features for discrete emotion and dimensional emotion multi-task recognition to obtain updated discrete emotion recognition information and dimensional emotion initial recognition information.

[0183] Specifically, during the update process, the weights and biases are continuously adjusted to minimize the error function, that is, through continuous back propagation , update the LSTM network parameters until It is no longer significantly reduced, and the predicted valence-arousal value is obtained, realizing dimensional speech emotion recognition.

[0184] S750, based on the updated discrete emotion recognition information and the dimensional emotion initial recognition information, obtain the updated overall error value, and use the updated dimensional emotion initial recognition information corresponding to the overall error value that meets the conditions as the dimensional speech emotion target recognition result.

[0185] The method for obtaining dimensional speech emotion target recognition results disclosed in this embodiment can gradually reduce the prediction error through iterative optimization, thereby improving the accuracy of emotion recognition. The initial recognition result may have a large error. Through continuous updating and optimization, the system can better capture and recognize the emotion dimension in the speech; the multi-task learning method combining discrete emotion recognition and dimensional emotion recognition enables the model to handle these two emotion recognition tasks at the same time, thereby improving the overall performance. Multi-task learning can help the model share knowledge and features between different tasks and improve the comprehensive understanding of emotions; by setting a preset error value to control the overall error, the model can effectively avoid overfitting or underfitting during the training process. The calculation and control of the overall error value helps to ensure that the model can maintain high robustness and reliability in the recognition task; the back propagation algorithm is used to update the model parameters, so that the model can dynamically adapt to new data and task requirements. Through back propagation and weighted feature updates, the model can gradually optimize its recognition ability and better adapt to changes in actual application scenarios.

[0186] This method realizes an automated model optimization process without manual intervention. It only needs to set the preset error and optimization strategy. The automated optimization process can greatly improve efficiency and reduce the impact of human errors. In general, this processing strategy can dynamically optimize and adjust the model in the process of multi-task emotion recognition, improve recognition accuracy and robustness, and simplify the model training and optimization process.

[0187] The methods disclosed in the prior art mainly focus on the simple mapping of acoustic features and emotional dimension space, or only rely on data-driven pre-trained models. They fail to deeply explore the generation and auditory perception process of human speech emotions, and fail to combine this knowledge with the deep representation of pre-trained models.

[0188] The dimensional speech emotion recognition method proposed in this application aims to solve the main problems existing in the existing speech emotion recognition technology through multiple innovative steps. Specifically, in response to the problems of redundant information and feature validity, this application obtains the target initial glottal wave feature parameters and vocal tract filter feature parameters to extract more representative pronunciation features. This feature extraction method can effectively reduce redundant information, thereby improving the effectiveness of emotion representation. By focusing on acoustic features related to emotion recognition, it avoids the influence of a large amount of redundant information in high-dimensional feature sets and improves the emotion recognition rate on untrained data.

[0189] In response to the interpretability problem of feature learning, this application uses a preset extraction strategy to extract cochlear map features from the speech to be recognized. This feature extraction imitates the characteristics of the human auditory system, making the extracted features more interpretable at the physiological and psychological levels. By introducing features that are consistent with human auditory perception, the comprehensibility of the emotion recognition results output by the model is increased, which helps to build a more reliable emotion recognition system.

[0190] In response to the problem of the difference between the model representation space and the human emotion space, this application performs time-series encoding on the target speech pronunciation features, cochlear map features and pre-trained model features to form three types of coding features. Then, through weighted processing and multi-task recognition technology, a variety of feature information is integrated to perform emotion recognition. This comprehensive method can effectively make up for the defects of deep learning models in emotion representation, making the model closer to human understanding of emotions. At the same time, through multi-task learning, the model's performance ability in different emotion tasks is improved.

[0191] In response to the problem of insufficient emotion recognition rate, this application processes the discrete emotion recognition information and dimensional emotion initial recognition information based on a preset loss calculation strategy after multi-task recognition to obtain dimensional speech emotion target recognition results. By combining different loss calculation strategies, the model training process can be more effectively optimized and the accuracy and reliability of the final emotion recognition can be improved.

[0192] This application improves the accuracy of emotion recognition by integrating multiple feature sources (acoustic features, auditory features, pre-trained features), ensures the coordination between different features, can be flexibly applied to different scenarios and tasks, has good scalability, and can be optimized in a variety of applications. Through improvements in feature extraction and encoding methods, the decision-making process of the model is easier to understand, thereby enhancing user trust.

[0193] Through the above methods, the solution disclosed in this application effectively solves multiple key problems existing in the prior art and provides a more reliable solution for the application of speech emotion recognition.

[0194] In a second aspect, the present application discloses a dimensional speech emotion recognition system, comprising:

[0195] A target speech pronunciation feature acquisition module is used to acquire the target speech pronunciation features based on the speech to be recognized, wherein the target speech pronunciation features include target initial glottal wave feature parameters and target vocal tract filter feature parameters;

[0196] An auditory modulation cochlear map feature acquisition module, used to extract auditory modulation cochlear map features from the speech to be recognized based on a preset extraction strategy;

[0197] A pre-trained model feature acquisition module is used to analyze the speech to be recognized based on a preset model and extract pre-trained model features;

[0198] The encoding module is used to perform temporal encoding on the target speech pronunciation features, the auditory modulation cochlear map features, and the pre-trained model features to form three types of encoding features;

[0199] A weighting module is used to perform weighted processing on the three types of coding features to obtain three types of weighted features;

[0200] The recognition module is used to perform multi-task recognition on the three types of weighted features to obtain discrete emotion recognition information and dimensional emotion initial recognition information;

[0201] The analysis module is used to process discrete emotion recognition information and dimensional emotion initial recognition information based on a preset loss calculation strategy to obtain dimensional speech emotion target recognition results.

[0202] It should be noted that the solutions in the dimensional speech emotion recognition method disclosed in the first aspect of this application are all applicable to the dimensional speech emotion recognition system disclosed in the second aspect of this application, so they will not be described one by one here.

[0203] The computer device according to the embodiment of the present disclosure includes a memory and a processor. The memory is used to store non-temporary computer-readable instructions. Specifically, the memory may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc.

[0204] The processor may be a central processing unit (CPU) or other forms of processing units with data processing capabilities and / or instruction execution capabilities, and may control other components in the computer device to perform desired functions. In one embodiment of the present disclosure, the processor is used to run the computer-readable instructions stored in the memory, so that the computer device performs all or part of the steps of the dimensional speech emotion recognition method of each embodiment of the present disclosure.

[0205] Those skilled in the art should be able to understand that in order to solve the technical problem of how to obtain a good user experience, the present embodiment may also include well-known structures such as a communication bus and an interface, and these well-known structures should also be included in the protection scope of the present disclosure.

[0206] like Figure 8A schematic diagram of the structure of a computer device provided in an embodiment of the present disclosure is shown, which is a schematic diagram of the structure of a computer device suitable for implementing the embodiment of the present disclosure. Figure 8 The computer device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0207] like Figure 8 As shown, the computer device may include a processor (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage device into a random access memory (RAM). In the RAM, various programs and data required for the operation of the computer device are also stored. The processor, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0208] Typically, the following devices can be connected to the I / O interface: input devices such as sensors or visual information acquisition devices; output devices such as display screens; storage devices such as tapes, hard disks, etc.; and communication devices. The communication device can allow the computer device to communicate with other devices (such as edge computing devices) wirelessly or by wire to exchange data. Figure 8 A computer device having various devices is shown, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.

[0209] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processor, all or part of the steps of the dimensional speech emotion recognition method of the embodiment of the present disclosure are executed.

[0210] For detailed description of this embodiment, reference may be made to the corresponding descriptions in the aforementioned embodiments, which will not be repeated here.

[0211] According to the computer-readable storage medium of the embodiment of the present disclosure, non-transitory computer-readable instructions are stored thereon. When the non-transitory computer-readable instructions are executed by a processor, all or part of the steps of the dimensional speech emotion recognition method of each embodiment of the present disclosure are executed.

[0212] The above-mentioned computer-readable storage media include, but are not limited to: optical storage media (e.g., CD-ROM and DVD), magneto-optical storage media (e.g., MO), magnetic storage media (e.g., magnetic tape or mobile hard disk), media with built-in rewritable non-volatile memory (e.g., memory card) and media with built-in ROM (e.g., ROM box).

[0213] For detailed description of this embodiment, reference may be made to the corresponding descriptions in the aforementioned embodiments, which will not be repeated here.

[0214] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0215] In the present disclosure, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. The block diagrams of the devices, devices, equipment, and systems involved in the present disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.

[0216] Additionally, as used herein, "or" used in a list of items beginning with "at least one" indicates a separate list, so that, for example, a list of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not mean that the example described is preferred or better than other examples.

[0217] It should also be noted that in the system and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0218] Various changes, substitutions, and modifications of the techniques described herein may be made without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of the present disclosure is not limited to the specific aspects of the processes, machines, manufactures, compositions of events, means, methods, and actions described above. Currently existing or later to be developed processes, machines, manufactures, compositions of events, means, methods, or actions that perform substantially the same functions or achieve substantially the same results as the corresponding aspects described herein may be utilized. Thus, the appended claims include such processes, machines, manufactures, compositions of events, means, methods, or actions within their scope.

[0219] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0220] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A dimensional speech emotion recognition method, characterized in that: include: Based on the speech to be recognized, obtaining the pronunciation features of the target speech, wherein the pronunciation features of the target speech include target initial glottal wave characteristic parameters and target vocal tract filter characteristic parameters; Extracting auditory modulation cochleogram features from the speech to be recognized based on a preset extraction strategy; Analyze the speech to be recognized based on the preset model and extract the pre-trained model features; The target speech pronunciation feature, the auditory modulation cochlear map feature, and the pre-trained model feature are respectively time-series encoded to form three types of encoding features; Performing weighted processing on the three types of coding features to obtain three types of weighted features; Perform multi-task recognition on the three types of weighted features to obtain discrete emotion recognition information and dimensional emotion initial recognition information; The discrete emotion recognition information and the dimensional emotion initial recognition information are processed based on a preset loss calculation strategy to obtain a dimensional speech emotion target recognition result; The step of extracting auditory modulation cochleogram features from the speech to be recognized based on a preset extraction strategy includes: Decompose the speech to be recognized into channel signal of acoustic frequencies, >1; Extracting the time envelope of each of the channel signals; Each of the time envelopes is modulated to generate The characteristic signal of the modulation frequency; based on The characteristic signal generates modulation units; right The modulation units perform convolution operations respectively to obtain auditory modulation cochleogram features; The modulated cochleogram is characterized by : = ； ； , ;in, is the Hamming window, is the number of samples in each time window, It is a voice signal length of time, It is frame shift. For the The starting position of the time window is the j-1th frame shift position.

2. The dimensional speech emotion recognition method according to claim 1, characterized in that: The step of obtaining the pronunciation features of the target speech based on the speech to be recognized includes: Analyze the speech to be recognized to obtain initial glottal wave characteristic parameters and initial vocal tract filter characteristic parameters; Based on the initial glottal wave characteristic parameters and the initial vocal tract filter characteristic parameters, Q synthesized speech is obtained, Q≥2; Obtaining Q difference values ​​between the synthesized speech and the speech to be recognized; Based on the Q difference values, obtaining a minimum mean square error; Based on the minimum mean square error, determining the corresponding initial glottal wave characteristic parameters as target initial glottal wave characteristic parameters; The initial vocal channel filter characteristic parameters are used as target vocal channel filter characteristic parameters.

3. The dimensional speech emotion recognition method according to claim 2, characterized in that: The step of obtaining Q synthesized speech sounds based on the initial glottal wave characteristic parameters and the initial vocal tract filter characteristic parameters comprises: Processing the initial glottal wave characteristic parameters Q times based on a preset loop strategy to obtain Q glottal waveform signals; Based on the initial vocal tract filter characteristic parameters, convolution operations are performed on the Q glottal waveform signals to obtain Q synthesized speech; The qth synthesized speech is S2 q (n): S2 q (n)= ; 1≤q≤Q,1≤ ≤ ; in, The speech to be recognized is The sampling point at time, For the current moment, is the filter order, is the channel filter coefficient, is the coefficient for adjusting the amplitude of the glottal wave, u q (n) is the qth glottal waveform signal.

4. The dimensional speech emotion recognition method according to claim 1, characterized in that: The method of analyzing the speech to be recognized based on the preset model and extracting the pre-trained model features includes: analyzing the speech to be recognized based on the Wav2vec2.0 pre-trained model and extracting a preset number of speech depth features; A preset number of the speech depth features constitute the pre-trained model features.

5. The dimensional speech emotion recognition method according to claim 1, characterized in that: The discrete emotion recognition information and the dimensional emotion initial recognition information are processed based on a preset loss calculation strategy to obtain a dimensional speech emotion target recognition result, including: Processing the discrete emotion recognition information based on the emotion category prediction loss function to obtain a first error value; The emotion category prediction loss function is: ; where x h is the predicted value corresponding to the discrete emotion recognition information of the hth category, y h is the true value corresponding to the h-th discrete emotion, is the corresponding emotion type in the discrete emotion recognition information; Processing the dimensional emotion initial recognition information based on the dimensional emotion prediction loss function to obtain a second error value; The dimensional sentiment prediction loss function is: ; in, is the Pearson correlation coefficient, is the variance of the predicted values ​​corresponding to all samples of the same dimension in the initial recognition information of the dimensional emotion, is the variance of the true values ​​corresponding to all samples of the same dimension in the initial recognition information of the dimensional emotion, is the average of the predicted values ​​corresponding to all samples of the same dimension in the initial recognition information of the dimension emotion, is the average value of the true values ​​corresponding to all samples of the same dimension in the initial recognition information of the dimensional emotion; Obtaining an overall error value based on the first error value and the second error value; The overall error value is: ;in, is the weight used to adjust the dimension sentiment loss and sentiment category loss; When the overall error value is less than a preset error, the dimensional emotion initial recognition information is used as the dimensional speech emotion target recognition result; When the overall error value is not less than the preset error, the weighted features are updated by using a back propagation algorithm for discrete emotion and dimensional emotion multi-task recognition to obtain updated discrete emotion recognition information and dimensional emotion initial recognition information; Based on the updated discrete emotion recognition information and dimensional emotion initial recognition information, an updated overall error value is obtained, and the updated dimensional emotion initial recognition information corresponding to the overall error value that meets the conditions is used as the dimensional speech emotion target recognition result.

6. A computer device, characterized in that: The computer device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the dimensional speech emotion recognition method described in any one of claims 1-5.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, which are used to enable a computer to execute the dimensional speech emotion recognition method described in any one of claims 1-5.

8. A computer program product comprising computer instructions, characterized in that: When the computer instruction is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images