Speech emotion recognition method and system

Abstract

The invention relates to the technical field of speech recognition, and particularly discloses a speech emotion recognition method and system. According to the method, time domain feature information, frequency domain feature information and channel domain feature information are acquired, sound intensity, timbre feature values and voiceless rate are acquired according to the time domain feature information, the frequency domain feature information and the channel domain feature information, finally, comprehensive association value feature values are acquired according to the sound intensity, the timbre feature values and the voiceless rate, and finally, the sound quality is improved. The emotion timbre keyword is extracted according to the comprehensive association value feature value, and the emotion timbre keyword is compared with a preset emotion timbre definition library to obtain an emotion timbre definition, so that the keyword is mapped to a specific emotion category by comparing the preset emotion timbre definition library, and the emotion timbre definition is obtained. Therefore, emotion features of the voice in the appeal process can be obtained through emotion recognition, and the real intention of the voice content can be obtained through emotion tone definition.

Description

Technical Field

[0001] The present invention relates to the field of speech recognition technology, and in particular to a method and system for recognizing speech emotion. Background Art

[0002] With the rapid progress of computer network communication and multimedia technology, human-computer interaction technology has gradually become an important part of the field of artificial intelligence. Especially with the rise of digital entertainment, the popularization of smart home appliances and driverless taxis, and the increasing application of computers, the naturalness and intelligence of human-computer interaction have become particularly critical. In this context, the processing of emotional information has become an important research direction for improving human-computer interaction capabilities. Therefore, how to enhance the adaptability of speech emotion recognition technology to user emotional changes, so that computers can accurately recognize emotional speech information, thereby improving the naturalness and intelligence of human-computer interaction, has become increasingly urgent;

[0003] As car-free taxis become more and more popular, the existing car-free taxi speech recognition can usually only convert speech into text and recognize the meaning of language through text. However, speech has emotional characteristics in the process of narration, which results in that simple recognition of speech cannot express the true intention of the speech content. Therefore, a method and system for recognizing speech emotions are needed to solve the above problems. Summary of the invention

[0004] The purpose of the present invention is to provide a method and system for recognizing speech emotions to solve the technical problems raised in the above-mentioned background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A method for recognizing speech emotion, comprising:

[0007] Acquire the user's voice collection information, wherein the acquired voice collection information includes time domain feature information, frequency domain feature information and channel domain feature information;

[0008] Acquire sound amplitude information and sound zero-crossing rate according to the time domain feature information, and acquire sound intensity according to the sound amplitude information;

[0009] Acquire the sound resonance peak and frequency band according to the frequency domain feature information, perform envelope extraction on the frequency band to obtain a plurality of envelope instantaneous curve values, and oscillate the plurality of envelope instantaneous curve values ​​to calculate the timbre feature value within a preset time;

[0010] Acquire multiple sound transmission rates according to the channel domain characteristic information, acquire sound transmission rate characteristic information according to the multiple sound transmission rates, and acquire clear and voiced rate according to the sound transmission rate characteristic information and sound zero crossing rate;

[0011] Obtaining a comprehensive correlation value characteristic value according to the sound intensity, timbre characteristic value and voicing rate;

[0012] Emotional timbre keywords are extracted according to the comprehensive correlation value feature value, and the emotional timbre keywords are compared with a preset emotional timbre definition library to obtain an emotional timbre definition.

[0013] Preferably, the step of obtaining the sound amplitude information and the sound zero-crossing rate according to the time domain feature information comprises:

[0014] Acquire a plurality of first sound amplitudes corresponding to the time domain according to the time domain feature information;

[0015] Acquire a corresponding average first sound amplitude according to the multiple first sound amplitudes, and use information corresponding to the average first sound amplitude as sound amplitude information;

[0016] Acquire multiple time domain sampling points according to the time domain feature information, compare the multiple time domain sampling points in adjacent order, and obtain multiple comparison information, wherein the multiple time domain sampling points reflect the active deviation degree of the positive and negative reaction signals;

[0017] Determining in sequence whether there are adjacent positive and negative signals in the plurality of comparison information;

[0018] If so, the time domain sampling points corresponding to the multiple comparison information of adjacent positive and negative signals are counted as zero points;

[0019] The corresponding sampling times are obtained according to the time domain sampling points, and the zero point times are compared with the sampling times to obtain the zero point passing rate, and the zero point passing rate is used as the sound zero crossing rate.

[0020] Preferably, the step of calculating the timbre characteristic value within a preset time by oscillating the multiple envelope instantaneous curve values ​​comprises:

[0021] Get the start and end time of collection according to the preset time;

[0022] The average gas temperature is calculated according to the multiple envelope instantaneous curve values, the starting collection time and the ending collection time, wherein the calculation formula is:

[0023]

[0024] in, represents the average envelope instantaneous curve value, t1 represents the start acquisition time, t2 represents the end acquisition time, w(d) represents the envelope instantaneous curve value, n represents the number of acquisitions of the envelope instantaneous curve value, where n=1, 2, 3...n;

[0025] The standard envelope instantaneous curve value is calculated according to the plurality of envelope instantaneous curve values ​​and the average envelope instantaneous curve value, wherein the calculation formula is:

[0026]

[0027] Among them, j(z) represents the instantaneous curve value of the standard envelope, p(j) l Indicates the lth acquisition of the envelope instantaneous curve value, N indicates the number of acquisitions of the envelope instantaneous curve value, l indicates the sequence number of the envelope instantaneous curve value, Indicates the average envelope instantaneous curve value;

[0028] The envelope instantaneous coefficient is calculated according to the standard envelope instantaneous curve value and the average envelope instantaneous curve value, wherein the calculation formula is:

[0029]

[0030] Among them, X(s) represents the instantaneous coefficient of the envelope, represents the average envelope instantaneous curve value, j(z) represents the standard envelope instantaneous curve value;

[0031] The timbre initial transient value is acquired according to the envelope transient coefficient, and the timbre initial transient value is used as the timbre characteristic value.

[0032] Preferably, the step of obtaining the voicing rate according to the sound transmission rate characteristic information and the sound zero-crossing rate comprises:

[0033] Acquire a transmission rate spectrum according to the sound transmission rate characteristic information;

[0034] Acquire multiple transmission peaks according to the transmission rate spectrum, and extract the multiple transmission peaks based on a preset first appearance to obtain a first transmission peak;

[0035] Acquire a fallback time to a preset standard transmission value according to the first transmission peak value, and use the fallback time as a baseband period;

[0036] Acquire a corresponding fundamental frequency according to the fundamental frequency period;

[0037] The fundamental frequency is compared with the sound zero-crossing rate to obtain a fundamental frequency-zero-crossing ratio, and the fundamental frequency-zero-crossing ratio is used as the voicing rate.

[0038] Preferably, the step of obtaining a comprehensive correlation value characteristic value according to the sound intensity, timbre characteristic value and voicing rate comprises:

[0039] Acquire a corresponding first weight value according to the sound intensity;

[0040] The comprehensive correlation value characteristic value is calculated according to the first weight value, sound intensity, timbre characteristic value and voicing rate, wherein the calculation formula is:

[0041] z(h)=[P(Z)*a+P(q)*(1-a)]*P(H);

[0042] Among them, z(h) represents the comprehensive correlation value characteristic value, P(Z) represents the sound intensity, a represents the first weight value, P(q) represents the timbre characteristic value, and P(H) represents the voicing rate.

[0043] Preferably, the step of comparing the emotional timbre keyword with a preset emotional timbre definition library to obtain an emotional timbre definition comprises:

[0044] Encoding the emotional timbre keywords to obtain emotional timbre coding values;

[0045] Encode each word in the preset emotional timbre definition library in sequence to obtain a plurality of preset emotional timbre encoding values;

[0046] The emotional timbre coding value is traversed and matched with a plurality of assumed emotional timbre coding values ​​based on a similarity model to obtain a timbre similarity value, wherein the function of the similarity model is:

[0047]

[0048] in, represents the timbre similarity value, j(z) represents the emotional timbre encoding value, Y(S) k represents the set emotional timbre coding value, k represents the sequence number of the set emotional timbre coding value, k=1, 2, 3...k;

[0049] The corresponding emotional timbre coding value is obtained according to the timbre similarity value, and the corresponding emotional timbre definition is obtained according to the emotional timbre coding value.

[0050] The present application also provides a speech emotion recognition system, comprising:

[0051] A first acquisition module is used to acquire the user's voice collection information, wherein the acquired voice collection information includes time domain feature information, frequency domain feature information and channel domain feature information;

[0052] A second acquisition module, configured to acquire sound amplitude information and sound zero-crossing rate according to the time domain feature information, and acquire sound intensity according to the sound amplitude information;

[0053] A third acquisition module is used to obtain the sound resonance peak and frequency band according to the frequency domain feature information, and perform envelope extraction on the frequency band to obtain a plurality of envelope instantaneous curve values, and oscillate the plurality of envelope instantaneous curve values ​​to calculate the timbre feature value within a preset time;

[0054] A fourth acquisition module, configured to acquire a plurality of sound transmission rates according to the channel domain characteristic information, acquire sound transmission rate characteristic information according to the plurality of sound transmission rates, and acquire a clear and voiced rate according to the sound transmission rate characteristic information and a sound zero-crossing rate;

[0055] A fifth acquisition module, used for acquiring a comprehensive correlation value characteristic value according to the sound intensity, timbre characteristic value and voicing rate;

[0056] The first comparison module is used to extract emotional timbre keywords according to the comprehensive correlation value feature value, and compare the emotional timbre keywords with a preset emotional timbre definition library to obtain an emotional timbre definition.

[0057] Preferably, the second acquisition module includes:

[0058] A first acquisition unit, configured to acquire a plurality of first sound amplitudes in a corresponding time domain according to the time domain feature information;

[0059] A second acquisition unit, configured to acquire a corresponding average first sound amplitude according to the plurality of first sound amplitudes, and use information corresponding to the average first sound amplitude as sound amplitude information;

[0060] A third acquisition unit is used to acquire a plurality of time domain sampling points according to the time domain feature information, and compare the plurality of time domain sampling points in an adjacent order to obtain a plurality of comparison information, wherein the plurality of time domain sampling points are active deviation degrees of positive and negative reaction signals;

[0061] A first judging unit, used for sequentially judging whether there are adjacent positive and negative signals in the plurality of comparison information;

[0062] If so, the time domain sampling points corresponding to the multiple comparison information of adjacent positive and negative signals are counted as zero points;

[0063] The fourth acquisition unit is used to acquire the corresponding sampling times according to the time domain sampling points, and compare the zero point times with the sampling times to obtain the zero point passing rate, and use the zero point passing rate as the sound zero crossing rate.

[0064] The present application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0065] The present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are implemented.

[0066] The beneficial effects of the present application are as follows: the present invention obtains the user's voice collection information, wherein the acquired voice collection information includes time domain feature information, frequency domain feature information and channel domain feature information, then obtains the sound amplitude information and the sound zero-crossing rate according to the time domain feature information, and obtains the sound intensity according to the sound amplitude information, then obtains the sound resonance peak and frequency band according to the frequency domain feature information, and performs envelope extraction on the frequency band to obtain a plurality of envelope instantaneous curve values, and calculates the timbre feature value within a preset time by oscillating the plurality of envelope instantaneous curve values, and then obtains a plurality of sound transmission rates according to the channel domain feature information, and obtains the sound intensity according to the plurality of sound transmission rates. The sound transmission rate characteristic information is obtained, and the voicing rate is obtained according to the sound transmission rate characteristic information and the sound zero-crossing rate. Finally, the comprehensive correlation value characteristic value is obtained according to the sound intensity, timbre characteristic value and voicing rate. Finally, the emotional timbre keywords are extracted according to the comprehensive correlation value characteristic value, and the emotional timbre keywords are compared with the preset emotional timbre definition library to obtain the emotional timbre definition. In this way, by comparing the preset emotional timbre definition library, the keywords are mapped to specific emotional categories, such as happiness, sadness, anger, etc., so that emotion recognition can be realized to know the emotional characteristics of the voice in the process of narration, so that the true intention of the voice content can be obtained through the emotional timbre definition. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 The figure is a schematic diagram of a method flow of an embodiment of the present application.

[0068] Figure 2 A schematic diagram of the system structure of an embodiment of the present application.

[0069] Figure 3 A schematic diagram of the internal structure of a computer device according to an embodiment of the present application.

[0070] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0071] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0072] like Figure 1-3 As shown, the present application provides a method for recognizing speech emotion, comprising:

[0073] S1. Acquire the user's voice collection information, wherein the acquired voice collection information includes time domain feature information, frequency domain feature information, and channel domain feature information;

[0074] S2. Acquire sound amplitude information and sound zero-crossing rate according to the time domain feature information, and acquire sound intensity according to the sound amplitude information;

[0075] S3, obtaining the sound resonance peak and frequency band according to the frequency domain feature information, and performing envelope extraction on the frequency band to obtain a plurality of envelope instantaneous curve values, and calculating the timbre feature value within a preset time by oscillating the plurality of envelope instantaneous curve values;

[0076] S4. Acquire multiple sound transmission rates according to the channel domain characteristic information, acquire sound transmission rate characteristic information according to the multiple sound transmission rates, and acquire clear and voiced rate according to the sound transmission rate characteristic information and the sound zero-crossing rate;

[0077] S5, obtaining a comprehensive correlation value characteristic value according to the sound intensity, timbre characteristic value and turbidity rate;

[0078] S6. Extract emotional timbre keywords according to the comprehensive correlation value feature value, and compare the emotional timbre keywords with a preset emotional timbre definition library to obtain an emotional timbre definition.

[0079] As described in the above steps S1-S6, as car-free taxis become more and more popular, the existing car-free taxi speech recognition can usually only convert speech into text, and recognize the meaning of language through text. However, the speech has emotional characteristics in the process of narration, which results in that the simple recognition of speech cannot express the true intention of the speech content. Therefore, the present invention first obtains the user's speech collection information, wherein the acquired speech collection information includes time domain feature information, frequency domain feature information and channel domain feature information, which can ensure that the characteristics of the speech signal are obtained from multiple angles (time domain, frequency domain, channel domain), providing a rich data basis for subsequent feature extraction and emotion recognition, and at the same time, multi-dimensional data helps to improve the accuracy and robustness of feature extraction. Then, according to the time domain feature information, the sound amplitude information and the sound zero-crossing rate are obtained, and the sound intensity is obtained according to the sound amplitude information. In this way, the instantaneous amplitude of the sound is reflected by the amplitude information, which is used to evaluate the loudness of the sound. The zero-crossing rate reflects the activity of the sound and is used to distinguish between unvoiced and voiced sounds. Then, the sound intensity can reflect the comprehensive amplitude information and provide a quantitative sound loudness index, which is helpful to judge the user's emotional state (such as excitement, calmness, etc.). Then, according to the frequency domain feature information, the sound resonance peak and frequency band are obtained, and the sound resonance peak reflects the resonance frequency of the sound. The frequency band provides the energy distribution of the sound in different frequency ranges, which is helpful to analyze the complexity of the timbre, help to distinguish different timbres, and perform Envelope extraction is performed to obtain multiple envelope instantaneous curve values, which reflect the amplitude change of the sound in time, which is helpful to capture the dynamic characteristics of the timbre, and the timbre characteristic value within a preset time is calculated by oscillating the multiple envelope instantaneous curve values, so that the timbre characteristic value can reflect the comprehensive envelope instantaneous curve value, and provide a quantitative sound timbre index, which is helpful to judge the user's emotional state (such as happiness, sadness, etc.), and then multiple sound transmission rates are obtained according to the channel domain characteristic information, and sound transmission rate characteristic information is obtained according to the multiple sound transmission rates, and the turbidity rate is obtained according to the sound transmission rate characteristic information and the sound zero-crossing rate. In this way, a quantitative sound clarity and turbidity index is provided in combination with the transmission rate characteristic information and the zero-crossing rate, which is helpful to judge The user's emotional state (such as anger, calmness, etc.) is judged, and then the comprehensive correlation value feature value is obtained according to the sound intensity, timbre feature value and voicing rate. In this way, multiple features are combined to provide a comprehensive emotion recognition index, which is helpful to more accurately judge the user's emotional state. At the same time, the comprehensive correlation value feature value can also be defined as a speech emotion feature vector through the sound intensity, timbre feature value and voicing rate, and the text obtained by speech recognition is combined as a text feature vector, and then the speech emotion feature vector and the text feature vector are multimodally fused to obtain a comprehensive correlation value feature vector, and the comprehensive correlation value feature vector corresponds to the comprehensive correlation value feature value, and then the comprehensive correlation value feature value corresponds to the corresponding speech emotion label. Finally,According to the comprehensive correlation value feature value, the emotional timbre keywords are extracted, and the emotional timbre keywords are compared with the preset emotional timbre definition library to obtain the emotional timbre definition. By comparing the preset emotional timbre definition library, the keywords are mapped to specific emotional categories, such as happiness, sadness, anger, etc., so that the emotional recognition can know the emotional characteristics of the voice in the process of narration, so that the true intention of the voice content can be obtained through the emotional timbre definition.

[0080] In one embodiment, the step S2 of acquiring the sound amplitude information and the sound zero-crossing rate according to the time domain feature information includes:

[0081] S201, acquiring a plurality of first sound amplitudes corresponding to the time domain according to the time domain feature information;

[0082] S202, acquiring a corresponding average first sound amplitude according to the multiple first sound amplitudes, and using information corresponding to the average first sound amplitude as sound amplitude information;

[0083] S203, acquiring a plurality of time domain sampling points according to the time domain feature information, and comparing the plurality of time domain sampling points in adjacent order to obtain a plurality of comparison information, wherein the plurality of time domain sampling points reflect the active deviation degree of the positive and negative reaction signals;

[0084] S204, sequentially determining whether there are adjacent positive and negative signals in the plurality of comparison information;

[0085] If so, the time domain sampling points corresponding to the multiple comparison information of adjacent positive and negative signals are counted as zero points;

[0086] S205. Obtain the corresponding sampling times according to the time domain sampling points, and compare the zero-point times with the sampling times to obtain a zero-point passing rate, and use the zero-point passing rate as the sound zero-crossing rate.

[0087] As described in the above steps S201-S205, the present invention first obtains a plurality of first sound amplitudes of the corresponding time domain according to the time domain feature information, and then obtains the corresponding average first sound amplitude according to the plurality of first sound amplitudes, and uses the information corresponding to the average first sound amplitude as the sound amplitude information. In this way, short-term fluctuations can be smoothed out by averaging the first sound amplitude to obtain a more stable amplitude index. At the same time, the sound amplitude information can reflect the overall loudness of the sound, which is helpful for evaluating the intensity of the sound. Then, a plurality of time domain sampling points are obtained according to the time domain feature information, and the plurality of time domain sampling points are compared in adjacent order. Yes, multiple comparison information is obtained, wherein multiple time domain sampling points react to the active deviation degree of the positive and negative signals through positive and negative reactions, and then it is judged in turn whether there are adjacent positive and negative signals in the multiple comparison information. If so, the time domain sampling points corresponding to the multiple comparison information with adjacent positive and negative signals are counted as zero point times, and then the corresponding sampling times are obtained according to the time domain sampling points, and the zero point times are compared with the sampling times to obtain the zero point pass rate, and the zero point pass rate is used as the sound zero-crossing rate. The zero point times reflect the activity level of the signal. High-frequency signals usually have more zero point times, and can also provide a basis for subsequent turbidity rate calculations.

[0088] In one embodiment, the step S3 of calculating the timbre characteristic value within a preset time by oscillating the multiple envelope instantaneous curve values ​​includes:

[0089] S301, obtaining the start collection time and the end collection time according to the preset time;

[0090] S302, calculating the average gas temperature according to the multiple envelope instantaneous curve values, the start collection time and the end collection time, wherein the calculation formula is:

[0091]

[0092] in, represents the average envelope instantaneous curve value, t1 represents the start acquisition time, t2 represents the end acquisition time, w(d) represents the envelope instantaneous curve value, n represents the number of acquisitions of the envelope instantaneous curve value, where n=1, 2, 3...n;

[0093] S303, calculating a standard envelope instantaneous curve value according to the plurality of envelope instantaneous curve values ​​and the average envelope instantaneous curve value, wherein the calculation formula is:

[0094]

[0095] Among them, j(z) represents the instantaneous curve value of the standard envelope, p(j) l Indicates the lth acquisition of the envelope instantaneous curve value, N indicates the number of acquisitions of the envelope instantaneous curve value, l indicates the sequence number of the envelope instantaneous curve value, Indicates the average envelope instantaneous curve value;

[0096] S304, calculating the envelope instantaneous coefficient according to the standard envelope instantaneous curve value and the average envelope instantaneous curve value, wherein the calculation formula is:

[0097]

[0098] Among them, X(s) represents the instantaneous coefficient of the envelope, represents the average envelope instantaneous curve value, j(z) represents the standard envelope instantaneous curve value;

[0099] S305 , obtaining a timbre start transient value according to the envelope transient coefficient, and using the timbre start transient value as a timbre feature value.

[0100] As described in the above steps S301-S305, the present invention first obtains the start collection time and the end collection time according to the preset time, and then calculates the average gas temperature according to the multiple envelope instantaneous curve values, the start collection time and the end collection time, and then calculates the standard envelope instantaneous curve value according to the multiple envelope instantaneous curve values ​​and the average envelope instantaneous curve value, so that the discrete degree or volatility of the data is reflected by the standard envelope instantaneous curve value, and then calculates the envelope instantaneous coefficient according to the standard envelope instantaneous curve value and the average envelope instantaneous curve value, so that the envelope instantaneous coefficient reflects the relative volatility of the data, and finally, obtains the timbre starting transient value according to the envelope instantaneous coefficient, and uses the timbre starting transient value as the timbre characteristic value, so that the timbre starting transient value is derived through the envelope instantaneous coefficient, and it is used as the timbre characteristic value for further timbre analysis or recognition, and provides data support for the calculation of the comprehensive correlation value characteristic value.

[0101] In one embodiment, the step S4 of acquiring the voicing rate according to the sound transmission rate characteristic information and the sound zero-crossing rate comprises:

[0102] S401, acquiring a transmission rate spectrum according to the sound transmission rate characteristic information;

[0103] S402, obtaining a plurality of transmission peaks according to the transmission rate spectrum, and extracting the plurality of transmission peaks based on a preset first occurrence to obtain a first transmission peak;

[0104] S403, acquiring a fallback time to a preset standard transmission value according to the first transmission peak value, and using the fallback time as a baseband period;

[0105] S404, acquiring a corresponding fundamental frequency according to the fundamental frequency period;

[0106] S405 , comparing the fundamental frequency with the sound zero-crossing rate to obtain a fundamental frequency-zero-crossing ratio, and using the fundamental frequency-zero-crossing ratio as the voiced-unvoiced ratio.

[0107] As described in the above steps S401-S405, the present invention first obtains a transmission rate spectrum according to the sound transmission rate characteristic information, so that the transmission rate in the time domain is converted into a frequency domain representation, and the distribution of different frequency components can be more intuitively seen. Then, multiple transmission peaks are obtained according to the transmission rate spectrum, and the multiple transmission peaks are extracted based on the preset first appearance to obtain the first transmission peak. In this way, the first-appearing peak is selected to avoid interference from subsequent peaks and ensure that the most significant periodic component is extracted. Then, the fallback time to the preset standard transmission value is obtained according to the first transmission peak. In this way, the fallback time reflects the time when the transmission rate falls from the peak to the standard value. This time interval usually corresponds to the period of the fundamental frequency. The fallback time is used as the fundamental frequency period, and then the corresponding fundamental frequency is obtained according to the fundamental frequency period. Finally, the fundamental frequency is compared with the sound zero-crossing rate to obtain the fundamental frequency-zero-crossing ratio, and the fundamental frequency-zero-crossing ratio is used as the clearness and voicing rate. In this way, the ratio of the fundamental frequency to the zero-crossing rate can reflect the periodic characteristics and activity of the sound, which is helpful to distinguish between clear and voiced sounds. At the same time, the clearness and voicing rate is a comprehensive indicator used to judge the clarity and voicing of the sound. Generally, a signal with a higher fundamental frequency and a lower zero-crossing rate is considered to be a voiced sound, and vice versa. Therefore, the voice-unvoiced ratio is of great significance in applications such as speech recognition, sentiment analysis, and speech synthesis.

[0108] In one embodiment, the step S5 of obtaining the comprehensive correlation value characteristic value according to the sound intensity, timbre characteristic value and voicing rate includes:

[0109] S501, obtaining a corresponding first weight value according to the sound intensity;

[0110] S502: Calculate a comprehensive correlation value characteristic value according to the first weight value, sound intensity, timbre characteristic value and voicing rate, wherein the calculation formula is:

[0111] z(h)=[P(Z)*a+P(q)*(1-a)]*P(H);

[0112] Among them, z(h) represents the comprehensive correlation value characteristic value, P(Z) represents the sound intensity, a represents the first weight value, P(q) represents the timbre characteristic value, and P(H) represents the voicing rate.

[0113] As described in the above steps S501-S502, the present invention first obtains the corresponding first weight value according to the sound intensity, and then calculates the comprehensive correlation value characteristic value according to the first weight value, sound intensity, timbre characteristic value and turbidity rate. In this way, the comprehensive correlation value characteristic value can improve the richness and expressiveness of the features, help improve the accuracy and robustness of emotion recognition, and provide a basis for the definition and judgment of emotional timbre.

[0114] In one embodiment, the step S6 of comparing the emotional timbre keyword with a preset emotional timbre definition library to obtain an emotional timbre definition includes:

[0115] S601, encoding the emotional timbre keyword to obtain an emotional timbre encoding value;

[0116] S602, encoding each word in the preset emotional timbre definition library in sequence to obtain a plurality of preset emotional timbre encoding values;

[0117] S603, traversing and matching the emotional timbre coding value with a plurality of assumed emotional timbre coding values ​​based on a similarity model to obtain a timbre similarity value, wherein the function of the similarity model is:

[0118]

[0119] in, represents the timbre similarity value, j(z) represents the emotional timbre encoding value, Y(S) k represents the set emotional timbre coding value, k represents the sequence number of the set emotional timbre coding value, k=1, 2, 3...k;

[0120] S604: Obtain a corresponding emotional timbre coding value according to the timbre similarity value, and obtain a corresponding emotional timbre definition according to the emotional timbre coding value.

[0121] As described in the above steps S601-S604, the present invention first encodes the emotional timbre keywords to obtain emotional timbre coding values, and then encodes each word of the preset emotional timbre definition library in turn to obtain multiple emotional timbre coding values, so that the emotional timbre keywords and the preset emotional timbre definition library can be standardized, and then the emotional timbre coding values ​​are traversed and matched with multiple emotional timbre coding values ​​based on a similarity model to obtain timbre similarity values, and finally, the corresponding emotional timbre coding values ​​are obtained according to the timbre similarity values, and the corresponding emotional timbre definitions are obtained according to the emotional timbre coding values, so that by comparing the preset emotional timbre definition library, the keywords are mapped to specific emotional categories, such as happiness, sadness, anger, etc., so that emotion recognition can obtain the emotional characteristics of the voice in the narration process, so that the true intention of the voice content can be obtained through the emotional timbre definition.

[0122] The present application also provides a speech emotion recognition system, comprising:

[0123] The first acquisition module 1 is used to acquire the user's voice collection information, wherein the acquired voice collection information includes time domain feature information, frequency domain feature information and channel domain feature information;

[0124] A second acquisition module 2, used to acquire sound amplitude information and sound zero-crossing rate according to the time domain feature information, and acquire sound intensity according to the sound amplitude information;

[0125] The third acquisition module 3 is used to obtain the sound resonance peak and frequency band according to the frequency domain feature information, and perform envelope extraction on the frequency band to obtain multiple envelope instantaneous curve values, and oscillate the multiple envelope instantaneous curve values ​​to calculate the timbre feature value within a preset time;

[0126] A fourth acquisition module 4 is used to acquire multiple sound transmission rates according to the channel domain characteristic information, acquire sound transmission rate characteristic information according to the multiple sound transmission rates, and acquire clear and voiced rate according to the sound transmission rate characteristic information and the sound zero crossing rate;

[0127] A fifth acquisition module 5, used for acquiring a comprehensive correlation value characteristic value according to the sound intensity, timbre characteristic value and voicing rate;

[0128] The first comparison module 6 is used to extract emotional timbre keywords according to the comprehensive correlation value feature value, and compare the emotional timbre keywords with a preset emotional timbre definition library to obtain an emotional timbre definition.

[0129] In one embodiment, the second acquisition module includes:

[0130] A first acquisition unit, configured to acquire a plurality of first sound amplitudes in a corresponding time domain according to the time domain feature information;

[0131] A second acquisition unit, configured to acquire a corresponding average first sound amplitude according to the plurality of first sound amplitudes, and use information corresponding to the average first sound amplitude as sound amplitude information;

[0132] A third acquisition unit is used to acquire a plurality of time domain sampling points according to the time domain feature information, and compare the plurality of time domain sampling points in an adjacent order to obtain a plurality of comparison information, wherein the plurality of time domain sampling points are active deviation degrees of positive and negative reaction signals;

[0133] A first judging unit, used for sequentially judging whether there are adjacent positive and negative signals in the plurality of comparison information;

[0134] If so, the time domain sampling points corresponding to the multiple comparison information of adjacent positive and negative signals are counted as zero points;

[0135] The fourth acquisition unit is used to acquire the corresponding sampling times according to the time domain sampling points, and compare the zero point times with the sampling times to obtain the zero point passing rate, and use the zero point passing rate as the sound zero crossing rate.

[0136] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned method for recognizing speech emotions are implemented.

[0137] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing related hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided in this application and used in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0138] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0139] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for recognizing speech emotion, characterized in that: include: Acquire the user's voice collection information, wherein the acquired voice collection information includes time domain feature information, frequency domain feature information and channel domain feature information; Acquire sound amplitude information and sound zero-crossing rate according to the time domain feature information, and acquire sound intensity according to the sound amplitude information; Acquire the sound resonance peak and frequency band according to the frequency domain feature information, perform envelope extraction on the frequency band to obtain a plurality of envelope instantaneous curve values, and oscillate the plurality of envelope instantaneous curve values ​​to calculate the timbre feature value within a preset time; Acquire multiple sound transmission rates according to the channel domain characteristic information, acquire sound transmission rate characteristic information according to the multiple sound transmission rates, and acquire clear and voiced rate according to the sound transmission rate characteristic information and sound zero crossing rate; Obtaining a comprehensive correlation value characteristic value according to the sound intensity, timbre characteristic value and voicing rate; Emotional timbre keywords are extracted according to the comprehensive correlation value feature value, and the emotional timbre keywords are compared with a preset emotional timbre definition library to obtain an emotional timbre definition.

2. The method for recognizing speech emotion according to claim 1, characterized in that: The step of obtaining the sound amplitude information and the sound zero-crossing rate according to the time domain feature information includes: Acquire a plurality of first sound amplitudes corresponding to the time domain according to the time domain feature information; Acquire a corresponding average first sound amplitude according to the multiple first sound amplitudes, and use information corresponding to the average first sound amplitude as sound amplitude information; Acquire multiple time domain sampling points according to the time domain feature information, compare the multiple time domain sampling points in adjacent order, and obtain multiple comparison information, wherein the multiple time domain sampling points reflect the active deviation degree of the positive and negative reaction signals; Determining in sequence whether there are adjacent positive and negative signals in the plurality of comparison information; If so, the time domain sampling points corresponding to the multiple comparison information of adjacent positive and negative signals are counted as zero points; The corresponding sampling times are obtained according to the time domain sampling points, and the zero point times are compared with the sampling times to obtain the zero point passing rate, and the zero point passing rate is used as the sound zero crossing rate.

3. The method for recognizing speech emotion according to claim 1, characterized in that: The step of calculating the timbre characteristic value within a preset time by oscillating the multiple envelope instantaneous curve values ​​comprises: Get the start and end time of collection according to the preset time; The average gas temperature is calculated according to the multiple envelope instantaneous curve values, the starting collection time and the ending collection time, wherein the calculation formula is: in, represents the average envelope instantaneous curve value, t1 represents the start acquisition time, t2 represents the end acquisition time, w(d) represents the envelope instantaneous curve value, n represents the number of acquisitions of the envelope instantaneous curve value, where n=1, 2, 3...n; The standard envelope instantaneous curve value is calculated according to the plurality of envelope instantaneous curve values ​​and the average envelope instantaneous curve value, wherein the calculation formula is: Among them, j(z) represents the instantaneous curve value of the standard envelope, p(j) l Indicates the lth acquisition of the envelope instantaneous curve value, N indicates the number of acquisitions of the envelope instantaneous curve value, l indicates the sequence number of the envelope instantaneous curve value, Indicates the average envelope instantaneous curve value; The envelope instantaneous coefficient is calculated according to the standard envelope instantaneous curve value and the average envelope instantaneous curve value, wherein the calculation formula is: Among them, X(s) represents the instantaneous coefficient of the envelope, represents the average envelope instantaneous curve value, j(z) represents the standard envelope instantaneous curve value; The timbre initial transient value is acquired according to the envelope transient coefficient, and the timbre initial transient value is used as the timbre characteristic value.

4. The method for recognizing speech emotion according to claim 1, characterized in that: The step of obtaining the clarity and voicing rate according to the sound transmission rate characteristic information and the sound zero-crossing rate comprises: Acquire a transmission rate spectrum according to the sound transmission rate characteristic information; Acquire multiple transmission peaks according to the transmission rate spectrum, and extract the multiple transmission peaks based on a preset first appearance to obtain a first transmission peak; Acquire a fallback time to a preset standard transmission value according to the first transmission peak value, and use the fallback time as a baseband period; Acquire a corresponding fundamental frequency according to the fundamental frequency period; The fundamental frequency is compared with the sound zero-crossing rate to obtain a fundamental frequency-zero-crossing ratio, and the fundamental frequency-zero-crossing ratio is used as the voicing rate.

5. The method for recognizing speech emotion according to claim 1, characterized in that: The step of obtaining a comprehensive correlation value characteristic value according to the sound intensity, timbre characteristic value and voicing rate comprises: Acquire a corresponding first weight value according to the sound intensity; The comprehensive correlation value characteristic value is calculated according to the first weight value, sound intensity, timbre characteristic value and voicing rate, wherein the calculation formula is: z(h)=[P(Z)*a+P(q)*(1-a)]*P(H); Among them, z(h) represents the comprehensive correlation value characteristic value, P(Z) represents the sound intensity, a represents the first weight value, P(q) represents the timbre characteristic value, and P(H) represents the voicing rate.

6. The method for recognizing speech emotion according to claim 1, characterized in that: The step of comparing the emotional timbre keyword with a preset emotional timbre definition library to obtain an emotional timbre definition includes: Encoding the emotional timbre keywords to obtain emotional timbre coding values; Encode each word in the preset emotional timbre definition library in sequence to obtain a plurality of preset emotional timbre encoding values; The emotional timbre coding value is traversed and matched with a plurality of assumed emotional timbre coding values ​​based on a similarity model to obtain a timbre similarity value, wherein the function of the similarity model is: in, represents the timbre similarity value, j(z) represents the emotional timbre encoding value, Y(S) k represents the set emotional timbre coding value, k represents the sequence number of the set emotional timbre coding value, k=1, 2, 3...k; The corresponding emotional timbre coding value is obtained according to the timbre similarity value, and the corresponding emotional timbre definition is obtained according to the emotional timbre coding value.

7. A speech emotion recognition system, characterized in that: include: A first acquisition module is used to acquire the user's voice collection information, wherein the acquired voice collection information includes time domain feature information, frequency domain feature information and channel domain feature information; A second acquisition module, used to acquire sound amplitude information and sound zero-crossing rate according to the time domain feature information, and acquire sound intensity according to the sound amplitude information; A third acquisition module is used to obtain the sound resonance peak and frequency band according to the frequency domain feature information, and perform envelope extraction on the frequency band to obtain a plurality of envelope instantaneous curve values, and oscillate the plurality of envelope instantaneous curve values ​​to calculate the timbre feature value within a preset time; A fourth acquisition module, configured to acquire a plurality of sound transmission rates according to the channel domain characteristic information, acquire sound transmission rate characteristic information according to the plurality of sound transmission rates, and acquire a clear and voiced rate according to the sound transmission rate characteristic information and a sound zero-crossing rate; A fifth acquisition module, used for acquiring a comprehensive correlation value characteristic value according to the sound intensity, timbre characteristic value and voicing rate; The first comparison module is used to extract emotional timbre keywords according to the comprehensive correlation value feature value, and compare the emotional timbre keywords with a preset emotional timbre definition library to obtain an emotional timbre definition.

8. The speech emotion recognition system according to claim 7, characterized in that: The second acquisition module includes: A first acquisition unit, configured to acquire a plurality of first sound amplitudes in a corresponding time domain according to the time domain feature information; A second acquisition unit, configured to acquire a corresponding average first sound amplitude according to the plurality of first sound amplitudes, and use information corresponding to the average first sound amplitude as sound amplitude information; A third acquisition unit is used to acquire a plurality of time domain sampling points according to the time domain feature information, and compare the plurality of time domain sampling points in an adjacent order to obtain a plurality of comparison information, wherein the plurality of time domain sampling points are active deviation degrees of positive and negative reaction signals; A first judging unit, used for sequentially judging whether there are adjacent positive and negative signals in the plurality of comparison information; If so, the time domain sampling points corresponding to the multiple comparison information of adjacent positive and negative signals are counted as zero points; The fourth acquisition unit is used to acquire the corresponding sampling times according to the time domain sampling points, and compare the zero point times with the sampling times to obtain the zero point passing rate, and use the zero point passing rate as the sound zero crossing rate.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images