AI-based voice emotion recognition model training method

Abstract

The invention discloses a training method of a voice emotion recognition model based on AI. Comprising the following steps: S1, collecting a large amount of data information; s2, preprocessing the collected emotion speech recognition data set; s3, converting the acquired emotion speech recognition data set into a spectrogram; s4, extracting features in the spectrogram through a convolutional neural network; s5, storing the energy information of different emotional states; s6, performing evaluation processing on the model; according to the method, a spectrogram is established through a 2D matrix obtained through STFT and energy, amplitude values or relative positions obtained through calculation; extracting local features from the spectrogram through a convolutional neural network, and forming an energy information template library of an emotional state through an emotional speech recognition data set, a label, a feature set and a range value of an energy information value; and the accuracy rate, the recall rate and the F1 score of the matrix test are mixed, so that the model performance can be measured more comprehensively, and the identification model can be evaluated.

Description

Technical Field

[0001] The present invention belongs to the field of AI speech recognition technology, and more specifically, particularly relates to a training method for an AI-based speech emotion recognition model. Background Art

[0002] Speech emotion recognition is a technology that studies the emotional information contained in human speech. It is used to identify and analyze the emotional state of the speaker's voice, including happiness, sadness, anger, anxiety, etc. The speaker's emotional state can be judged by analyzing the tone, speaking speed, voice quality and other features in the speech, thereby helping people to better understand and communicate.

[0003] The development of speech emotion recognition can be traced back to the 1980s, when researchers began to try to use speech signal processing technology to identify the speaker's emotional state. With the continuous development of artificial intelligence and machine learning technology, speech emotion recognition technology has also made great progress. Now, speech emotion recognition has been widely used in voice assistants, customer service robots, sentiment analysis and other fields.

[0004] At present, speech emotion recognition technology can achieve high-accuracy emotion recognition, identify the speaker's emotional state, and make corresponding responses or processing accordingly. It has broad application prospects in business, medical care, education and other fields, and can help people better understand and communicate, and improve the efficiency and comfort of human-computer interaction.

[0005] With the continuous development of artificial intelligence and speech recognition technology, speech emotion recognition technology will also be further improved and applied.

[0006] When applying artificial intelligence to speech recognition, the speech recognition model needs to be trained so that the speech recognition model can quickly recognize speech and improve the accuracy of speech recognition.

[0007] However, there are some problems with the existing technology: when training, the existing speech recognition model uses a large number of samples for training, but does not process the speech information, resulting in slow model training and low recognition accuracy. Therefore, we propose a training method for an AI-based speech emotion recognition model. Summary of the invention

[0008] In view of the problems existing in the prior art, the purpose of the present invention is to provide a training method for a speech emotion recognition model based on AI, so that the emotional speech recognition data is obtained through a 2D matrix through STFT, and features are extracted from the emotional speech recognition data, and an energy information template library is built, thereby improving the training efficiency and accuracy of the recognition model.

[0009] To achieve the above object, the present invention provides the following technical solution: a training method for a speech emotion recognition model based on AI, comprising the following steps: S1. Collect a large amount of data information: collect a large amount of emotional speech recognition data sets, and divide the collected emotional speech recognition data sets into training sets, verification sets and test sets; S2. Preprocessing the collected emotional speech recognition data set: performing noise removal, mute removal, audio format conversion and data normalization processing on the collected emotional speech recognition data set; S3, converting the collected emotional speech recognition data set into a spectrogram: converting the emotional speech recognition data set into a 2D matrix obtained by STFT so that the emotional speech recognition data set displays energy distribution at different times and frequencies; S4. Extract features from the spectrogram using a convolutional neural network: Use a convolutional neural network to extract local features from the spectrogram, effectively capture the energy information of different emotional states, and use multiple rounds of feature extraction to form a set of energy information of all different emotional states. By calculating the average, maximum, and minimum values ​​of the energy information, the range of energy information values ​​of different emotional states is obtained. S5. Storing energy information of different emotional states: storing energy information of local features extracted from the convolutional neural network to form an energy information template library of emotional states; S6. Evaluate the model: input the validation set and test set into the model, use the model to predict the sentiment of the validation set and test set, label the prediction results of the validation set and test set, then compare the actual labels of the validation set and test set, generate a mixing matrix, obtain the predicted probability information, calculate the precision, recall and F1 score, and measure the model performance more comprehensively.

[0010] Specifically, the emotional speech recognition data set in S1 includes an equal number of samples of anger, disgust, fear, happiness, sadness, surprise and neutral emotions; The training set, validation set and test set in the emotional speech recognition dataset account for 70%-80% of the training set, and the validation set and test set account for 10%-15% each.

[0011] Specifically, the denoising in S2 uses an audio denoising algorithm to remove background noise; De-silencing is used to remove silent parts or useless leading and trailing blank segments in emotional speech recognition datasets; Audio format conversion is used to convert the emotional speech recognition dataset into WAV format; Data normalization is used to normalize the emotional speech recognition dataset.

[0012] Specifically, the denoising adopts a convolution filtering algorithm, and the calculation formula of the convolution filtering algorithm is as follows: , in, is the input signal; are the filter coefficients; is the output signal; The calculation formula for data normalization is as follows: , in, For the historical conversation record of input, is the minimum value in the data set, is the maximum value in the dataset, scaling the emotional speech recognition dataset to the range of [0, 1].

[0013] Specifically, the calculation formula of the STFT in S3 is as follows: ; in, It's in time and frequency STFT results under; is the original signal; It is a window function that limits the time range of the Fourier transform; is the core complex exponential of the Fourier transform; The result of STFT is a complex matrix, where each element represents the amplitude and phase information of a time point and frequency point; The 2D matrix obtained by STFT contains the following information: Each row represents a frequency component; Each column represents the spectrum information in a time period; The form of the matrix, Set the total length of the signal to N, the window length to W, and the length of the window overlap to O; The number of rows in the matrix corresponds to the resolution of the frequency axis, which is the window length W divided by 2, resulting in the number of frequency points W / 2; The number of columns of the matrix corresponds to the resolution of the time axis, which is determined by the step size of the sliding window. If the step size is S, the number of columns of the matrix is ; The final matrix size is: .

[0014] Specifically, the result of the STFT calculation is a complex matrix, each element It consists of amplitude and phase. The amplitude indicates the strength of the frequency component, and the phase indicates the relative position of the signal under the frequency. The amplitude is expressed as ; The phase is expressed as ; The amplitude spectrum of each time-frequency point is obtained by taking the modulus of the complex number, or the phase spectrum is obtained by the phase angle; The calculation formula of the amplitude spectrum is as follows: ; in, and are the real and imaginary parts of a complex number respectively; The calculation formula of phase spectrum is as follows: .

[0015] Specifically, the calculation of extracting local features from the spectrogram by the convolutional neural network in S4 is as follows: A local area of ​​the spectrogram is the value on the time and frequency axis and looks like this: , The horizontal axis represents different time points, and the vertical axis represents different frequency components; each element in the matrix Represents the energy, amplitude value or relative position under a certain combination of time and frequency; Convolution Kernel , 3×3 convolution kernel: ; Convolution operation: The convolution operation is the weighted sum of each local area: ; The output feature map Y is at position The values ​​are: ; in, is the first The value of the position indicates the result of the convolution operation; is the input spectrogram In Location The value on is the convolution kernel The index of is the convolution kernel In Location The weight value on ; is the convolution kernel size.

[0016] Specifically, the energy information in S4 is calculated to form a set as follows: That is, the energy information of anger, disgust, fear, happiness, sadness, surprise and neutral emotions are respectively formed into a set, and the form of the set is: ; The average is calculated as follows: ; And the maximum value is , the minimum value is ; Then the range of energy information values ​​of different emotional states is .

[0017] Specifically, the horizontal axis and vertical axis of the mixing matrix in S6 represent the predicted label and the actual label respectively; the predicted values ​​are filled into the mixing matrix according to a large number of model prediction results, and the diagonal of the mixing matrix is ​​set to the number of samples predicted to be positive and actually positive, indicating that the model correctly identifies the positive samples, and there is ; The predictions on the non-diagonal lines are negative, that is, the number of samples predicted as negative but actually positive, indicating the negative samples predicted incorrectly, and there is .

[0018] Specifically, the calculation formulas for the precision, recall and F1 score in S6 are as follows: The calculation formula of the accuracy is as follows: ; It indicates the proportion of correctly classified positive samples to all samples. A high precision rate means that among the samples predicted as positive, the prediction accuracy of the positive class is high. The calculation formula of the recall rate is as follows: ; It indicates the ratio of correctly classified positive samples to negative samples. A high recall rate means that all samples that are actually positive can be correctly predicted as positive. The calculation formula of the F1 score is as follows: ; It represents the harmonic mean of precision and recall. The F1 score is between 0 and 1. The larger the value, the more accurate the model's prediction results are.

[0019] Technical effects and advantages of the present invention: The present invention collects a large number of emotional speech recognition data sets and divides them into 70%-80% training sets, 10%-15% validation sets and test sets respectively, so as to facilitate the training and test processing of speech recognition models, improve the accuracy of a large number of emotional speech recognition data sets by preprocessing a large number of emotional speech recognition data sets, and eliminate white noise and blank segments in a large number of emotional speech recognition data sets; and convert the format of a large number of emotional speech recognition data sets, and reduce the speech dimension of a large number of emotional speech recognition data sets through normalization processing, and can also reduce the calculation pressure of subsequent calculations; The 2D matrix obtained by STFT enables the emotional speech recognition data set to display energy distribution at different times and frequencies, and the result of STFT is a complex matrix, in which each element represents the amplitude and phase information of the time point and the frequency point, and the energy, amplitude value or relative position of the emotional speech recognition data set is obtained, which is convenient for the subsequent recognition model to train the emotional speech recognition data set. By calculating the energy, amplitude value or relative position of the emotional speech recognition data set, the recognition model can realize the training and analysis of the emotional speech recognition data set, and establish a spectrogram in combination with the calculated energy, amplitude value or relative position, thereby improving the efficiency of calculation; And by extracting local features from the spectrogram through a convolutional neural network, and by calculating and processing between the convolution kernel and the spectrogram, feature extraction of the emotional speech recognition data set is realized, and a feature set is formed for the energy information of anger, disgust, fear, happiness, sadness, surprise and neutral emotions respectively, and the range values ​​of energy information values ​​of different emotional states are calculated by the average value, maximum value and minimum value, so as to facilitate the establishment of key-value pairs for storing the emotional speech recognition data set, label, feature set and range value of energy information value, and form an energy information template library of emotional state, so as to facilitate matching and recognition when predicting voice emotions in the future, and improve the accuracy and efficiency of recognition; By inputting the validation set and test set into the recognition model, recording the prediction results of the validation set and test set, generating a mixing matrix, obtaining the predicted probability information, calculating the precision, recall rate and F1 score, and more comprehensively measuring the model performance, the recognition model can be evaluated.

[0020] Further features and advantages of the present invention will become apparent from the following detailed description of exemplary embodiments of the present invention with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a schematic diagram of the step flow provided by the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0023] like Figure 1 As shown, the training method of the speech emotion recognition model based on AI provided by the embodiment of the present invention includes the following steps: S1. Collect a large amount of data information: collect a large amount of emotional speech recognition data sets, and divide the collected emotional speech recognition data sets into training sets, verification sets and test sets; S2. Preprocessing the collected emotional speech recognition data set: performing denoising, de-silencing, audio format conversion and data normalization processing on the collected emotional speech recognition data set; S3, converting the collected emotional speech recognition data set into a spectrogram: converting the emotional speech recognition data set into a 2D matrix obtained by STFT so that the emotional speech recognition data set displays energy distribution at different times and frequencies; S4. Extract features from the spectrogram using a convolutional neural network: Use a convolutional neural network to extract local features from the spectrogram, effectively capture the energy information of different emotional states, and use multiple rounds of feature extraction to form a set of energy information of all different emotional states. By calculating the average, maximum, and minimum values ​​of the energy information, the range of energy information values ​​of different emotional states is obtained. S5. Storing energy information of different emotional states: storing energy information of local features extracted from the convolutional neural network to form an energy information template library of emotional states; S6. Evaluate the model: input the validation set and test set into the model, use the model to predict the sentiment of the validation set and test set, label the prediction results of the validation set and test set, then compare the actual labels of the validation set and test set, generate a mixing matrix, obtain the predicted probability information, calculate the precision, recall and F1 score, and measure the model performance more comprehensively.

[0024] In this embodiment, preferably, the emotional speech recognition data set in S1 includes equal sample numbers of anger, disgust, fear, happiness, sadness, surprise and neutral emotions; The training set, validation set, and test set in the emotional speech recognition dataset account for 70%-80% of the training set, 10%-15% of the validation set, and 15% of the test set respectively; It should be noted that when collecting the emotional speech recognition data set, an equal number of samples of anger, disgust, fear, happiness, sadness, surprise and neutral emotions were selected to maintain the accuracy of training and prevent uneven number of samples from causing abnormalities in the training of the recognition model. In order to test the recognition model, the emotional speech recognition data set was distributed proportionally to facilitate training, verification and testing operations and improve the prediction accuracy of the recognition model.

[0025] In this embodiment, preferably, the denoising in S2 uses an audio denoising algorithm to remove background noise; De-silencing is used to remove silent parts or useless leading and trailing blank segments in emotional speech recognition datasets; Audio format conversion is used to convert the emotional speech recognition dataset into WAV format; Data normalization is used to normalize the emotional speech recognition dataset; It should be noted that processing the emotional speech recognition dataset can improve the clarity of the emotional speech recognition dataset, prevent erroneous analysis of the recognition model, and eliminate blank segments, which can reduce the size of the emotional speech recognition dataset. Combined with data normalization processing, the dimension of the emotional speech recognition dataset can be reduced to facilitate subsequent computational analysis and processing. In addition, audio format conversion can ensure the format uniformity of the emotional speech recognition dataset, which is convenient for subsequent training and processing of the recognition model.

[0026] In this embodiment, preferably, the denoising adopts a convolution filtering algorithm, and the calculation formula of the convolution filtering algorithm is as follows: , in, is the input signal; are the filter coefficients; is the output signal; The calculation formula for data normalization is as follows: , in, For the historical conversation record of input, is the minimum value in the data set, is the maximum value in the dataset, scaling the emotional speech recognition dataset to the range of [0, 1]; It should be noted that the convolution filtering algorithm is used to filter out the noise in the emotional speech recognition dataset, thereby improving the accuracy and clarity of the emotional speech recognition dataset. In addition, data normalization is used to reduce the dimension of the emotional speech recognition dataset, which facilitates the subsequent training of the recognition model and reduces the pressure and complexity of the calculation.

[0027] In this embodiment, preferably, the calculation formula of STFT in S3 is as follows: ; in, It's in time and frequency STFT results under; is the original signal; It is a window function that limits the time range of the Fourier transform; is the core complex exponential of the Fourier transform; The result of STFT is a complex matrix, where each element represents the amplitude and phase information of a time point and frequency point; The 2D matrix obtained by STFT contains the following information: Each row represents a frequency component; Each column represents the spectrum information in a time period; The form of the matrix, Set the total length of the signal to N, the window length to W, and the length of the window overlap to O; The number of rows in the matrix corresponds to the resolution of the frequency axis, which is the window length W divided by 2, resulting in the number of frequency points W / 2; The number of columns of the matrix corresponds to the resolution of the time axis, which is determined by the step size of the sliding window. If the step size is S, the number of columns of the matrix is ; The final matrix size is: ; It should be noted that the emotional speech recognition dataset is converted into a 2D spectrogram through STFT, the amplitude and phase information of each element representing the time point and frequency point is obtained, and the matrix size is set.

[0028] In this embodiment, preferably, the result of the STFT calculation is a complex matrix, each element It consists of amplitude and phase. The amplitude indicates the strength of the frequency component, and the phase indicates the relative position of the signal under the frequency. The amplitude is expressed as ; The phase is expressed as ; The amplitude spectrum of each time-frequency point is obtained by taking the modulus of the complex number, or the phase spectrum is obtained by the phase angle; The calculation formula of the amplitude spectrum is as follows: ; in, and are the real and imaginary parts of a complex number respectively; The calculation formula of phase spectrum is as follows: ; It should be noted that by performing calculation processing on the complex matrix, it is convenient to calculate the amplitude and phase information, obtain the amplitude spectrum and phase spectrum, and facilitate the subsequent extraction of amplitude and phase features.

[0029] In this embodiment, preferably, the calculation of extracting local features from the spectrogram by the convolutional neural network in S4 is as follows: A local area of ​​the spectrogram is the value on the time and frequency axis and looks like this: , The horizontal axis represents different time points, and the vertical axis represents different frequency components; each element in the matrix Represents the energy, amplitude value or relative position under a certain combination of time and frequency; Convolution Kernel , 3×3 convolution kernel: ; Convolution operation: The convolution operation is the weighted sum of each local area: ; The output feature map Y is at position The values ​​are: ; in, is the first The value of the position indicates the result of the convolution operation; is the input spectrogram In Location The value on is the convolution kernel The index of is the convolution kernel In Location The weight value on ; is the convolution kernel Size; It should be noted that, by performing computational processing on the convolution kernel and the spectrogram, characteristic information in the spectrogram is obtained, and the recognition model is trained using the obtained characteristic information, so as to analyze and process the emotional speech recognition data set and predict the emotional state.

[0030] In this embodiment, preferably, the energy information in S4 is calculated to form a set as follows: That is, the energy information of anger, disgust, fear, happiness, sadness, surprise and neutral emotions are respectively formed into a set, and the form of the set is: ; The average is calculated as follows: ; And the maximum value is , the minimum value is ; Then the range of energy information values ​​of different emotional states is ; It should be noted that the range of energy information values ​​for different emotional states is set by using the average value, maximum value and minimum value, which facilitates the subsequent matching processing when analyzing and processing the emotional speech recognition data set, thereby improving the recognition efficiency of the emotional state.

[0031] In this embodiment, preferably, the horizontal axis and the vertical axis in the mixing matrix in S6 represent the predicted label and the actual label respectively; the predicted values ​​are filled into the mixing matrix according to a large number of model prediction results, and the diagonal of the mixing matrix is ​​set to the number of samples predicted to be positive and actually positive, indicating that the model correctly identifies the positive samples, and there is ; The predictions on the non-diagonal lines are negative, that is, the number of samples predicted as negative but actually positive, indicating the number of negative samples predicted incorrectly, and there is ; It should be noted that the prediction results are divided through the mixing matrix to evaluate the performance of the classification model. By showing the comparison between the predicted values ​​and the true values, we can gain a deeper understanding of the prediction results of the classification model.

[0032] In this embodiment, preferably, the calculation formulas for the precision, recall and F1 score in S6 are as follows: The calculation formula of precision is as follows: ; It indicates the proportion of correctly classified positive samples to all samples. A high precision rate means that among the samples predicted as positive, the prediction accuracy of the positive class is high. The calculation formula for recall is as follows: ; It indicates the ratio of correctly classified positive samples to negative samples. A high recall rate means that all samples that are actually positive can be correctly predicted as positive. The calculation formula of F1 score is as follows: ; It represents the harmonic mean of precision and recall. The F1 score is between 0 and 1. The larger the value, the more accurate the model's prediction results are. It should be noted that a series of performance indicators such as precision, recall and F1 score are used to evaluate the effectiveness of the model, which facilitates in-depth understanding of the recognition model and the prediction results of the recognition model, and facilitates accurate evaluation of the recognition model.

[0033] The specific operation process of this application: Step 1: Collect a large amount of data information: collect a large amount of emotional speech recognition data sets, and divide the collected emotional speech recognition data sets into training sets, validation sets and test sets; The emotional speech recognition dataset includes equal number of samples of anger, disgust, fear, happiness, sadness, surprise, and neutral emotions; The training set, validation set, and test set for anger, disgust, fear, happiness, sadness, surprise, and neutral emotions account for 70%-80% of the training set, 10%-15% of the validation set, and 15% of the test set, respectively; Step 2: Preprocess the collected emotional speech recognition data set: remove noise, remove silence, convert audio format and normalize the collected emotional speech recognition data set; Denoising uses audio denoising algorithms to remove background noise; De-silencing is used to remove silent parts or useless leading and trailing blank segments in emotional speech recognition datasets; Audio format conversion is used to convert the emotional speech recognition dataset into WAV format; Data normalization is used to normalize the emotional speech recognition dataset; The denoising method uses the convolution filtering algorithm. The calculation formula of the convolution filtering algorithm is as follows: , in, is the input signal; are the filter coefficients; is the output signal; The calculation formula for data normalization is as follows: , in, For the historical conversation record of input, is the minimum value in the data set, is the maximum value in the dataset, scaling the emotional speech recognition dataset to the range of [0, 1]; Step 3: Convert the collected emotional speech recognition data set into a spectrogram: convert the emotional speech recognition data set into a 2D matrix obtained by STFT so that the emotional speech recognition data set displays energy distribution at different times and frequencies; The calculation formula of STFT is as follows: ; in, It's in time and frequency STFT results under; is the original signal; It is a window function that limits the time range of the Fourier transform; is the core complex exponential of the Fourier transform; The result of STFT is a complex matrix, where each element represents the amplitude and phase information of a time point and frequency point; The 2D matrix obtained by STFT contains the following information: Each row represents a frequency component; Each column represents the spectrum information in a time period; The form of the matrix, Set the total length of the signal to N, the window length to W, and the length of the window overlap to O; The number of rows in the matrix corresponds to the resolution of the frequency axis, which is the window length W divided by 2, resulting in the number of frequency points W / 2; The number of columns of the matrix corresponds to the resolution of the time axis, which is determined by the step size of the sliding window. If the step size is S, the number of columns of the matrix is ; The final matrix size is: ; The result of the STFT calculation is a complex matrix with each element It consists of amplitude and phase. The amplitude indicates the strength of the frequency component, and the phase indicates the relative position of the signal under the frequency. The amplitude is expressed as ; The phase is expressed as ; The amplitude spectrum of each time-frequency point is obtained by taking the modulus of the complex number, or the phase spectrum is obtained by the phase angle; The calculation formula of the amplitude spectrum is as follows: ; in, and are the real and imaginary parts of a complex number respectively; The calculation formula of phase spectrum is as follows: ; Step 4: Extract features from the spectrogram using a convolutional neural network: Use a convolutional neural network to extract local features from the spectrogram, effectively capture the energy information of different emotional states, and use multiple rounds of feature extraction to form a set of energy information of all different emotional states. By calculating the average, maximum, and minimum values ​​of the energy information, the range of energy information values ​​of different emotional states is obtained. The calculation of the convolutional neural network to extract local features from the spectrogram is as follows: A local area of ​​the spectrogram is the value on the time and frequency axis and looks like this: , The horizontal axis represents different time points, and the vertical axis represents different frequency components; each element in the matrix Represents the energy, amplitude value or relative position under a certain combination of time and frequency; Convolution Kernel , 3×3 convolution kernel: ; Convolution operation: The convolution operation is the weighted sum of each local area: ; The output feature map Y is at position The values ​​are: ; in, is the first The value of the position indicates the result of the convolution operation; is the input spectrogram In Location The value on is the convolution kernel The index of is the convolution kernel In Location The weight value on ; is the convolution kernel Size; The energy information is calculated to form a set as follows: That is, the energy information of anger, disgust, fear, happiness, sadness, surprise and neutral emotions are respectively formed into a set, and the form of the set is: ; The average is calculated as follows: ; And the maximum value is , the minimum value is ; Then the range of energy information values ​​of different emotional states is ; Step 5: Storing the energy information of different emotional states: storing the energy information of local features extracted from the convolutional neural network to form an energy information template library of emotional states; Step 6: Evaluate the model: Input the validation set and test set into the model, use the model to predict the sentiment of the validation set and test set, label the prediction results of the validation set and test set, then compare the actual labels of the validation set and test set, generate a mixing matrix, obtain the predicted probability information, calculate the precision, recall and F1 score, and measure the model performance more comprehensively. The horizontal axis and vertical axis in the mixing matrix represent the predicted label and the actual label respectively. The predicted values ​​are filled into the mixing matrix according to a large number of model prediction results, and the diagonal of the mixing matrix is ​​set to the number of samples predicted to be positive and actually positive, indicating that the model correctly identifies the positive samples. ; The predictions on the non-diagonal lines are negative, that is, the number of samples predicted as negative but actually positive, indicating the negative samples predicted incorrectly, and there is ; The calculation formulas for precision, recall, and F1 score are as follows: The calculation formula of precision is as follows: ; It indicates the proportion of correctly classified positive samples to all samples. A high precision rate means that among the samples predicted as positive, the prediction accuracy of the positive class is high. The calculation formula for recall is as follows: ; It indicates the ratio of correctly classified positive samples to negative samples. A high recall rate means that all samples that are actually positive can be correctly predicted as positive. The calculation formula of F1 score is as follows: ; It represents the harmonic mean of precision and recall. The F1 score is between 0 and 1. The larger the value, the more accurate the model's prediction results are.

[0034] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A training method for a speech emotion recognition model based on AI, characterized in that: The steps include: S1. Collect a large amount of data information: collect a large amount of emotional speech recognition data sets, and divide the collected emotional speech recognition data sets into training sets, verification sets and test sets; S2. Preprocessing the collected emotional speech recognition data set: performing noise removal, mute removal, audio format conversion and data normalization processing on the collected emotional speech recognition data set; S3, converting the collected emotional speech recognition data set into a spectrogram: converting the emotional speech recognition data set into a 2D matrix obtained by STFT so that the emotional speech recognition data set displays energy distribution at different times and frequencies; S4. Extract features from the spectrogram using a convolutional neural network: Use a convolutional neural network to extract local features from the spectrogram, effectively capture the energy information of different emotional states, and use multiple rounds of feature extraction to form a set of energy information of all different emotional states. By calculating the average, maximum, and minimum values ​​of the energy information, the range of energy information values ​​of different emotional states is obtained. S5. Storing energy information of different emotional states: storing energy information of local features extracted from the convolutional neural network to form an energy information template library of emotional states; S6. Evaluate the model: input the validation set and test set into the model, use the model to predict the sentiment of the validation set and test set, label the prediction results of the validation set and test set, then compare the actual labels of the validation set and test set, generate a mixing matrix, obtain the predicted probability information, calculate the precision, recall and F1 score, and measure the model performance more comprehensively.

2. The training method of the speech emotion recognition model based on AI according to claim 1, characterized in that: The emotional speech recognition data set in S1 includes an equal number of samples of anger, disgust, fear, happiness, sadness, surprise and neutral emotions; The training set, validation set and test set in the emotional speech recognition dataset account for 70%-80% of the training set, and the validation set and test set account for 10%-15% each.

3. The training method of the speech emotion recognition model based on AI according to claim 1, characterized in that: The denoising in S2 uses an audio denoising algorithm to remove background noise; De-silencing is used to remove silent parts or useless leading and trailing blank segments in emotional speech recognition datasets; Audio format conversion is used to convert the emotional speech recognition dataset into WAV format; Data normalization is used to normalize the emotional speech recognition dataset.

4. The training method of the AI-based speech emotion recognition model according to claim 3, characterized in that: The de-noising method uses a convolution filtering algorithm, and the calculation formula of the convolution filtering algorithm is as follows: ， in, is the input signal; are the filter coefficients; is the output signal; The calculation formula for data normalization is as follows: ， in, For the historical conversation record of input, is the minimum value in the data set, is the maximum value in the dataset, scaling the emotional speech recognition dataset to the range of [0, 1].

5. The training method of the speech emotion recognition model based on AI according to claim 1, characterized in that: The calculation formula of STFT in S3 is as follows: ； in, It's in time and frequency STFT results under; is the original signal; It is a window function that limits the time range of the Fourier transform; is the core complex exponential of the Fourier transform; The result of STFT is a complex matrix, where each element represents the amplitude and phase information of a time point and frequency point; The 2D matrix obtained by STFT contains the following information: Each row represents a frequency component; Each column represents the spectrum information in a time period; The form of the matrix, Set the total length of the signal to N, the window length to W, and the length of the window overlap to O; The number of rows in the matrix corresponds to the resolution of the frequency axis, which is the window length W divided by 2, resulting in the number of frequency points W / 2; The number of columns of the matrix corresponds to the resolution of the time axis, which is determined by the step size of the sliding window. If the step size is S, the number of columns of the matrix is ; The final matrix size is: .

6. The training method of the speech emotion recognition model based on AI according to claim 5, characterized in that: The result of the STFT calculation is a complex matrix with each element It consists of amplitude and phase. The amplitude indicates the strength of the frequency component, and the phase indicates the relative position of the signal under the frequency. The amplitude is expressed as ; The phase is expressed as ; The amplitude spectrum of each time-frequency point is obtained by taking the modulus of the complex number, or the phase spectrum is obtained by the phase angle; The calculation formula of the amplitude spectrum is as follows: ； in, and are the real and imaginary parts of a complex number respectively; The calculation formula of phase spectrum is as follows: 。 7. The training method of the speech emotion recognition model based on AI according to claim 6, characterized in that: The calculation of the local features extracted from the spectrogram by the convolutional neural network in S4 is as follows: A local area of ​​the spectrogram is the value on the time and frequency axis and looks like this: ， The horizontal axis represents different time points, and the vertical axis represents different frequency components; each element in the matrix Represents the energy, amplitude value or relative position under a certain combination of time and frequency; Convolution Kernel , 3×3 convolution kernel: ； Convolution operation: The convolution operation is the weighted sum of each local area: ； The output feature map Y is at position The values ​​are: ； in, is the first The value of the position indicates the result of the convolution operation; is the input spectrogram In Location The value on is the convolution kernel The index of is the convolution kernel In Location The weight value on ; is the convolution kernel size.

8. The training method of the speech emotion recognition model based on AI according to claim 1, characterized in that: The energy information in S4 is calculated to form a set as follows: That is, the energy information of anger, disgust, fear, happiness, sadness, surprise and neutral emotions are respectively formed into a set, and the form of the set is: ； The average is calculated as follows: ； And the maximum value is , the minimum value is ; Then the range of energy information values ​​of different emotional states is .

9. The training method of the speech emotion recognition model based on AI according to claim 8, characterized in that: The horizontal axis and vertical axis of the mixing matrix in S6 represent the predicted label and the actual label respectively; the predicted values ​​are filled into the mixing matrix according to a large number of model prediction results, and the diagonal of the mixing matrix is ​​set to the number of samples predicted to be positive and actually positive, indicating that the model correctly identifies the positive samples, and there is ; The predictions on the non-diagonal lines are negative, that is, the number of samples predicted as negative but actually positive, indicating the negative samples predicted incorrectly, and there is .

10. The training method of the speech emotion recognition model based on AI according to claim 9, characterized in that: The calculation formulas for the precision, recall and F1 score in S6 are as follows: The calculation formula of the accuracy is as follows: ； It indicates the proportion of correctly classified positive samples to all samples. A high precision rate means that among the samples predicted as positive, the prediction accuracy of the positive class is high. The calculation formula of the recall rate is as follows: ； It indicates the ratio of correctly classified positive samples to negative samples. A high recall rate means that all samples that are actually positive can be correctly predicted as positive. The calculation formula of the F1 score is as follows: ； It represents the harmonic mean of precision and recall. The F1 score is between 0 and 1. The larger the value, the more accurate the model's prediction results are.

Images