Emotion prediction method based on voice audio data acquisition and analysis

Abstract

The invention discloses an emotion prediction method based on voice and audio data acquisition and analysis, and the method comprises the following steps: S1, obtaining voice and audio data through a recording device, S2, carrying out the preprocessing of the obtained audio data, S3, carrying out the collection and marking of the audio data after the processing of the audio data, and S4, carrying out the emotion prediction of the audio data. The method has the beneficial effects that after the acquired voice audio data is subjected to related processing work such as acquisition and analysis, training can be carried out according to the feature data to obtain an emotion prediction model; therefore, the prediction trend and the prediction result can be displayed for people more visually through the emotion prediction model, upgrading and perfecting processing can be carried out in time through testing and perfecting of the emotion prediction model, then it can be guaranteed that the emotion prediction result is more accurate, and the prediction result can be displayed more visually.

Description

Technical Field

[0001] The present invention relates to the technical field of emotion prediction, and in particular to an emotion prediction method based on speech audio data collection and analysis. Background Art

[0002] Sound is a wave generated by the vibration of an object. When an object vibrates, the surrounding air is constantly compressed and relaxed and diffused to the surrounding. This is the sound wave. The frequency range of sound that people can hear is 20Hz~20kHz. The main formats of audio data are: WAVE, MOD, Layer-3, Real Audio, CD Audio, etc. There are three common methods for collecting audio data: directly obtaining existing audio, using audio processing software to capture and intercept sound, and recording sound with a microphone.

[0003] In the work of collecting and analyzing speech and audio data, the recorded speech and audio contain a lot of emotional emotions. Therefore, it is very important to achieve accurate emotion prediction results after completing the analysis and processing of the collected speech and audio data. In addition, the results obtained in the work of emotion prediction are difficult to display intuitively, which makes it difficult for people to better process the speech and audio data obtained after understanding the collection and analysis. Summary of the invention

[0004] The purpose of the present invention is to provide a method for emotion prediction based on speech audio data collection and analysis to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solution: an emotion prediction method based on speech audio data collection and analysis, comprising the following contents:

[0006] S1: Use recording equipment to obtain voice audio data;

[0007] S2: pre-processing the acquired audio data;

[0008] S3: After completing the audio data processing, the audio data is collected and labeled, and then significant features are extracted;

[0009] S4: Use machine learning technology or deep learning technology to train the sentiment prediction model;

[0010] S5: Input the speech audio data to be recognized into the emotion prediction model, and perform emotion prediction test through the model;

[0011] S6: After completing the preliminary test, evaluate the test results;

[0012] S7: When the result of the emotion prediction evaluation is high, the emotion prediction model can be used normally to perform emotion prediction work. If the result of the emotion prediction evaluation is low, the above steps are repeated and relevant data are supplemented.

[0013] Preferably, the step S1 includes the following:

[0014] S11: Use a microphone or other audio input device to record audio in real time;

[0015] S12: Directly import existing audio data through audio processing software or tools;

[0016] S13: Capture and intercept sound through audio processing software.

[0017] Preferably, step S2 includes the following:

[0018] S21: firstly remove the noise in the voice audio obtained in step S1, and improve the quality of the voice audio;

[0019] S22: then adjusting the voice audio signal to a unified standard to facilitate subsequent processing;

[0020] S23: subdividing the speech audio signal after signal unification into a plurality of short-time frame segments.

[0021] Preferably, step S3 includes the following:

[0022] S31: collecting and processing the processed audio data, and then marking the segments of the collected audio data;

[0023] S32: Marking features based on the MFCC and spectrum center;

[0024] S33: Feature marking can be performed based on the energy of the short-time frame and the short-time average zero-crossing rate, as well as the pitch, tone length and timbre of the audio data;

[0025] S34: After the feature labeling is completed, the labeled audio data is segmented and extracted using digital processing technology.

[0026] Preferably, step S4 includes the following:

[0027] S41: using machine learning technology or deep learning technology based on the extracted feature data;

[0028] S42: Use machine learning technology or deep learning technology to train the emotion prediction model;

[0029] S43: The emotion prediction model obtained through training is trained on the labeled samples to learn the emotion features, and is applied to the unlabeled speech data summary to identify and classify emotions.

[0030] Preferably, step S5 includes the following:

[0031] S51: Feature-marking the audio data of the speech to be recognized;

[0032] S52: using the trained emotion prediction model to input the marked audio data of the speech to be recognized;

[0033] S53: Display the input audio data of the marked speech to be recognized through the emotion prediction model, analyze and process the audio data of the marked speech to be recognized through the emotion prediction model, and output the corresponding emotion or mood category at the same time, so as to obtain the prediction test result.

[0034] Preferably, step S6 includes the following:

[0035] S61: Perform preliminary evaluation and identification based on the emotion prediction results obtained from the test;

[0036] S62: And use the historical sentiment prediction method to test the prediction work;

[0037] S63: Then compare and analyze the model test sentiment prediction results with the results predicted by the historical sentiment prediction method.

[0038] Preferably, the step S7 includes the following:

[0039] S71: Making a determination based on the comparison and analysis results;

[0040] S72: When the result of the judgment is higher than the result predicted by the emotion prediction model, it can be directly used to input the collected and analyzed speech audio data into the emotion prediction model to predict the emotion state;

[0041] S73: When the judgment result is lower than the result predicted by the emotion prediction model, repeat the above steps and supplement the defective data at the same time to train a more accurate emotion prediction model, and then test again.

[0042] Compared with the prior art, the beneficial effect of the present invention is that after the acquired speech audio data has been collected and analyzed and other related processing work has been carried out, an emotion prediction model can be trained according to the feature data, and then the emotion prediction model can be used to more intuitively display the prediction trend and prediction results to people. Moreover, through the testing and improvement of the emotion prediction model, it can be upgraded and improved in time, thereby ensuring that the results of the emotion prediction are more accurate and the prediction results can be displayed more intuitively. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a flow chart of the steps of the present invention. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] See also Figure 1 The present invention provides a technical solution: an emotion prediction method based on speech audio data collection and analysis, comprising the following contents:

[0046] S1: Use recording equipment to obtain voice audio data;

[0047] S2: pre-processing the acquired audio data;

[0048] S3: After completing the audio data processing, the audio data is collected and labeled, and then significant features are extracted;

[0049] S4: Use machine learning technology or deep learning technology to train the sentiment prediction model;

[0050] S5: Input the speech audio data to be recognized into the emotion prediction model, and perform emotion prediction test through the model;

[0051] S6: After completing the preliminary test, evaluate the test results;

[0052] S7: When the result of the emotion prediction evaluation is high, the emotion prediction model can be used normally to perform emotion prediction work. If the result of the emotion prediction evaluation is low, the above steps are repeated and relevant data are supplemented.

[0053] Wherein, the step S1 includes the following:

[0054] S11: Use a microphone or other audio input device to record audio in real time;

[0055] S12: Directly import existing audio data through audio processing software or tools;

[0056] S13: Capture and intercept sound through audio processing software;

[0057] Wherein, the step S2 includes the following:

[0058] S21: firstly remove the noise in the voice audio obtained in step S1, and improve the quality of the voice audio;

[0059] S22: then adjusting the voice audio signal to a unified standard to facilitate subsequent processing;

[0060] S23: subdividing the speech audio signal after signal unification into a plurality of short time frame segments;

[0061] Wherein, the step S3 includes the following:

[0062] S31: collecting and processing the processed audio data, and then marking the segments of the collected audio data;

[0063] S32: Marking features based on the MFCC and spectrum center;

[0064] S33: Feature marking can be performed based on the energy of the short-time frame and the short-time average zero-crossing rate, as well as the pitch, tone length and timbre of the audio data;

[0065] S34: After the feature labeling is completed, the labeled audio data is segmented and extracted by using a digital processing technology;

[0066] Wherein, the step S4 includes the following:

[0067] S41: using machine learning technology or deep learning technology based on the extracted feature data;

[0068] S42: Use machine learning technology or deep learning technology to train the emotion prediction model;

[0069] S43: training the labeled samples using the trained emotion prediction model and learning the emotion features, and applying the model to the unlabeled speech data to identify and classify the emotions;

[0070] Wherein, the step S5 includes the following:

[0071] S51: Feature-marking the audio data of the speech to be recognized;

[0072] S52: using the trained emotion prediction model to input the marked audio data of the speech to be recognized;

[0073] S53: displaying the input audio data of the marked speech to be recognized through the emotion prediction model, analyzing and processing the audio data of the marked speech to be recognized through the emotion prediction model, and outputting the corresponding emotion or mood category at the same time, thereby obtaining a prediction test result;

[0074] Wherein, the step S6 includes the following:

[0075] S61: Perform preliminary evaluation and identification based on the emotion prediction results obtained from the test;

[0076] S62: And use the historical sentiment prediction method to test the prediction work;

[0077] S63: then comparing and analyzing the model test emotion prediction results with the results predicted by the historical emotion prediction method;

[0078] Wherein, the step S7 includes the following:

[0079] S71: Making a determination based on the comparison and analysis results;

[0080] S72: When the result of the judgment is higher than the result predicted by the emotion prediction model, it can be directly used to input the collected and analyzed speech audio data into the emotion prediction model to predict the emotion state;

[0081] S73: When the judgment result is lower than the result predicted by the emotion prediction model, repeat the above steps and supplement the defective data at the same time to train a more accurate emotion prediction model, and then test again.

[0082] Specifically, when using the present invention, a microphone or other audio input device is used to record sound audio in real time, and existing audio data is directly imported through audio processing software or tools. The sound is captured and intercepted through the audio processing software. For the voice audio obtained in step S1, the noise in the audio is first removed, and the quality of the voice audio is improved. Then the signal of the voice audio is adjusted to a unified standard, so as to facilitate subsequent processing, and the voice audio signal with completed signal unification is subdivided into a number of short-time frame segments, and the processed audio data is collected and processed, and then the collected audio data is segmented and marked, and features are marked based on the MFCC and the center of the spectrum. Feature marking can also be performed according to the energy of the short-time frame and the short-time average zero-crossing rate, as well as the pitch, length and timbre of the audio data. After the feature marking work is completed, the marked audio data is segmented and extracted through digital processing technology, and machine learning technology or deep learning technology is used to cooperate based on the extracted feature data, and the emotion prediction model is trained using the machine learning technology or deep learning technology, and the labeled samples are trained by the emotion prediction model obtained by training, and the emotion prediction model is used to learn the emotion prediction model. The emotion features are collected and applied to the unlabeled speech data to identify and classify emotions. The audio data of the speech to be identified is feature-labeled, and the trained emotion prediction model is used to input the labeled audio data of the speech to be identified. The input labeled audio data of the speech to be identified is displayed through the emotion prediction model. The audio data of the labeled speech to be identified is analyzed and processed by the emotion prediction model, and the corresponding emotion or emotion category is output at the same time, so as to obtain the prediction test result. According to the emotion prediction result obtained by the test, a preliminary evaluation and recognition is performed, and the test prediction work is performed using the historical emotion prediction method. Then, the emotion prediction result of the model test is compared and analyzed with the result predicted by the historical emotion prediction method. According to the result of the comparison and analysis, the judgment work is performed. When the judgment result is higher than the result predicted by the emotion prediction model, it can be directly used to input the speech audio data after collection and analysis into the emotion prediction model to predict the emotional state. When the judgment result is lower than the result predicted by the emotion prediction model, the above steps are repeated, and the defective data is supplemented at the same time, so as to train a more accurate emotion prediction model, and then the test work is performed again.

[0083] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for emotion prediction based on speech audio data collection and analysis, characterized in that: Includes the following: S1: Use recording equipment to obtain voice audio data; S2: pre-processing the acquired audio data; S3: After completing the audio data processing, the audio data is collected and labeled, and then significant features are extracted; S4: Use machine learning technology or deep learning technology to train the sentiment prediction model; S5: Input the speech audio data to be recognized into the emotion prediction model, and perform emotion prediction test through the model; S6: After completing the preliminary test, evaluate the test results; S7: When the result of evaluating the emotion prediction is high, the emotion prediction model can be used normally to perform the emotion prediction work. If the result of evaluating the emotion prediction is low, the above steps are repeated and relevant data are supplemented.

2. The emotion prediction method based on speech audio data collection and analysis according to claim 1, characterized in that: The step S1 includes the following: S11: Use a microphone or other audio input device to record audio in real time; S12: Directly import existing audio data through audio processing software or tools; S13: Capture and intercept sound through audio processing software.

3. The emotion prediction method based on speech audio data collection and analysis according to claim 2 is characterized in that: The step S2 includes the following: S21: firstly remove the noise in the voice audio obtained in step S1, and improve the quality of the voice audio; S22: then adjusting the voice audio signal to a unified standard to facilitate subsequent processing; S23: subdividing the speech audio signal after signal unification into a plurality of short-time frame segments.

4. The emotion prediction method based on speech audio data collection and analysis according to claim 3 is characterized by: The step S3 includes the following: S31: collecting and processing the processed audio data, and then marking the segments of the collected audio data; S32: Marking features based on the MFCC and spectrum center; S33: Feature marking can be performed based on the energy of the short-time frame and the short-time average zero-crossing rate, as well as the pitch, tone length and timbre of the audio data; S34: After the feature labeling is completed, the labeled audio data is segmented and extracted using digital processing technology.

5. The emotion prediction method based on speech audio data collection and analysis according to claim 4 is characterized in that: The step S4 includes the following: S41: using machine learning technology or deep learning technology based on the extracted feature data; S42: Use machine learning technology or deep learning technology to train the emotion prediction model; S43: The emotion prediction model obtained through training is trained on the labeled samples to learn the emotion features, and is applied to the unlabeled speech data summary to identify and classify emotions.

6. The emotion prediction method based on speech audio data collection and analysis according to claim 5 is characterized by: The step S5 includes the following: S51: Feature-marking the audio data of the speech to be recognized; S52: using the trained emotion prediction model to input the marked audio data of the speech to be recognized; S53: Display the input audio data of the marked speech to be recognized through the emotion prediction model, analyze and process the audio data of the marked speech to be recognized through the emotion prediction model, and output the corresponding emotion or mood category at the same time, so as to obtain the prediction test result.

7. The emotion prediction method based on speech audio data collection and analysis according to claim 6 is characterized by: The step S6 includes the following: S61: Perform preliminary evaluation and identification based on the emotion prediction results obtained from the test; S62: And use the historical sentiment prediction method to test the prediction work; S63: Then compare and analyze the model test sentiment prediction results with the results predicted by the historical sentiment prediction method.

8. The emotion prediction method based on speech audio data collection and analysis according to claim 7 is characterized by: The step S7 includes the following: S71: Making a determination based on the comparison and analysis results; S72: When the result of the judgment is higher than the result predicted by the emotion prediction model, it can be directly used to input the collected and analyzed speech audio data into the emotion prediction model to predict the emotion state; S73: When the judgment result is lower than the result predicted by the emotion prediction model, repeat the above steps and supplement the defective data at the same time to train a more accurate emotion prediction model, and then test again.

Images