A method for AI to analyze speech and recognize emotional states

Abstract

A method for AI-powered speech analysis and emotional state recognition involves AI speech analysis technology. It accurately analyzes a person's speech, identifies emotional fluctuations, and determines risk levels, providing timely and effective feedback. Leveraging a third-party database, it obtains a rich set of emotional reference samples. Using semantic recognition and speech recognition systems, it compares these samples with the newly collected speech, obtaining all required parameter values. The corresponding weights are then calculated from a list to determine the difference. Finally, the difference is fed back in real time, and the results are returned to the list, making them more accurate and reasonable, increasing reference value, and enriching and diversifying the content in the database.

Description

Technical Field

[0001] The present invention relates to the field of AI speech analysis technology, and in particular to a method for AI speech analysis to identify emotional states. Background Art

[0002] Speech is a primary means of conveying information and a crucial behavioral signal that reflects human emotions. The different tones of individual words can generate numerous semantic meanings. Current AI technologies focus on semantic recognition, analyzing the message conveyed by language based on the meaning of text, but overlook the role of speech in conveying human emotions. The six most common human emotions are happiness, sadness, anger, disgust, fear, and surprise. These basic emotions can be conveyed through both language and speech during conversations, with emotional information conveyed through the intonation, duration, and stress of speech. Therefore, building intelligent emotion recognition systems requires combining semantic and speech information for comprehensive analysis to more accurately grasp the true intentions and emotional state of the person involved. Summary of the Invention

[0003] In order to overcome the shortcomings of the above technologies, the present invention provides a method for analyzing speech using AI that can accurately assess the emotions of the parties.

[0004] The technical solution adopted by the present invention to overcome the technical problems is:

[0005] A method for AI-based speech analysis and emotional recognition includes the following steps:

[0006] a) Building an information database RDB1, which is used to store the collected audio file information;

[0007] b) connecting the information database RDB1 to a third-party speech and semantic recognition library via the Internet, converting the audio files in the information database RDB1 into phrases through the API interface of the third-party speech and semantic recognition library, and placing the converted phrases into a list L1;

[0008] c) Submit the vocabulary in list L1 to Baidu's semantic analysis engine AIS1 through API callback. Use the conversation emotion recognition module of Baidu's semantic analysis engine AIS1 to identify the emotional information in list L1. The emotional information categories are: negative emotion, neutral emotion, and positive emotion. Negative emotion is -1, neutral emotion is 0, and positive emotion is 1. The emotional information value is filled in the emotion index variable x attitudeIn the example, the text information extraction module of Baidu semantic analysis engine AIS1 is used to identify the semantic information in list L1. The semantic information categories are: disagree, agree, and wait and see. Disagree is -1, wait and see is 0, and agree is 1. The value of the semantic information is filled in the attitude bias variable x aim middle;

[0009] d) Connect the information database RDB1 to the CASIA Chinese Emotion Corpus via the Internet. The CASIA Chinese Emotion Corpus analyzes the audio files in the information database RDB1 to obtain emotional information. The emotional information categories are: angry, happy, afraid, sad, surprised, and neutral. The angry emotional information is extracted and stored in the variable x. anger Extract the fear emotion information and store it in variable x fear Extract the sad emotional information and store it in variable x sad middle;

[0010] e) Set the sentiment index variable x attitude , attitude bias variable x aim ,variable x anger ,variable x fear ,variable x sad Input into the emotion recognition AIS3 process, the AIS3 process creates a weight list PList, and the weight list PList records the emotion index variable x attitude The weight variable W attitude , attitude bias variable x aim The weight variable W aim ,variable x anger The weight variable W anger ,variable x fear The weight variable W fear ,variable x sad The weight variable W sad and the weight variable W attitude The initial value V0, weight variable W aim The initial value V0, weight variable W anger The initial value V0, weight variable W fear The initial value V0, weight variable W sad The initial value V0;

[0011] f) Calculate the sentiment information weight W r ;

[0012] g) Through the formula Calculate the overall sentiment index α out , where e is a natural constant, α out When it is less than 0, it means disagreement. out When it is greater than 0, it means agreement, αout When it is equal to 0, it means wait-and-see and neutral.

[0013] Preferably, the information database RDB1 in step a) adopts a relational database management system.

[0014] Preferably, in step a), audio file information is collected by using an audio collection device, and the collected audio file information is stored in the information database RDB1 in WAV format.

[0015] Preferably, in step b), the third-party speech semantic recognition library is Baidu speech engine or iFLYTEK speech engine, and the audio files in the information database RDB1 are converted into UTF-8 encoded phrases through the API interface of the third-party speech semantic recognition library.

[0016] Furthermore, the weight variable W in step e) attitude , weight variable W aim , weight variable W anger , weight variable W fear , weight variable W sad The initial value V0 is a random decimal between 0 and 1.

[0017] Furthermore, in step f), the formula W r =(W aim +W anger *x anger +W fear *x fear +W sad *x sad )*x aim Calculate the emotional information weight W r .

[0018] Furthermore, the method further includes performing the following steps after step g):

[0019] h) Calculate the overall sentiment index α out At the same time, the meaning expressed by the speaker in the audio file information is manually collected α real , when the speaker expresses his disagreement, α real The value is -1, when the speaker means to agree, α real When the value is 1, the speaker expresses a wait-and-see attitude. real The value is 0;

[0020] i) Through the formula The difference Δα is obtained by calculation;

[0021] j) Through the formula W′ anger =W anger +W anger *(Δα / αout ) Calculate the new weight variable W′ anger , through the formula W′ fear =W fear +W fear *(Δα / α out ) Calculate the new weight variable W′ fear , through the formula W′ sad =W sad +W sad *(Δα / α out ) Calculate the new weight variable W′ sad , through the formula W′ aim =W aim +W aim *(Δα / α out ) Calculate the new weight variable W′ aim , through the formula W′ attitude =W attitude +W attitude *(Δα / α out ) Calculate the new weight variable W′ attitude ;

[0022] k) The new weight variable W′ anger Put it into the weight list PList and replace the weight variable W anger , the new weight variable W′ fear Put it into the weight list PList and replace the weight variable W fear , the new weight variable W′ sad Put it into the weight list PList and replace the weight variable W sad , the new weight variable W′ aim Put it into the weight list PList and replace the weight variable W aim , the new weight variable W′ attitude Put it into the weight list PList and replace the weight variable W attitude ;

[0023] l) Repeat steps a) to k) N times and output the final overall sentiment index α out Preferably, in step k), the value of N is 1000.

[0024] The beneficial effects of the present invention are: effectively solving the problem of analyzing multiple data sources, accurately analyzing the voice of the parties involved, identifying emotional fluctuations, analyzing risk levels, and providing timely and effective feedback. Leveraging a third-party database, a rich set of emotional reference samples is obtained. Using semantic recognition and speech recognition systems, these samples are matched and compared with the newly collected speech to obtain all required parameter values. The corresponding weights are then calculated from a list to obtain the difference. Finally, the difference can be fed back in real time, and the results are returned to the list, making them more accurate and reasonable, increasing reference value, and enriching and diversifying the content in the database. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 Flow chart of the method of the present invention. DETAILED DESCRIPTION

[0026] The following is combined with Figure 1 The present invention is further described.

[0027] A method for AI-based speech analysis and emotional recognition includes the following steps:

[0028] a) Build an information database RDB1, which is used to store the collected audio file information for later data retrieval.

[0029] b) Connect the information database RDB1 to a third-party speech and semantic recognition library via the Internet. Use the API of the third-party speech and semantic recognition library to convert the audio files in the information database RDB1 into phrases, and place the converted phrases into a list L1. For example: L1 = {"definitely", "will", "on time", "pay"}.

[0030] c) Submit the vocabulary in list L1 to Baidu semantic analysis engine AIS1 through API callback. Baidu semantic analysis engine AIS1 lexical analysis provides users with three major functions: word segmentation, part-of-speech tagging, and named entity recognition. It can identify basic vocabulary in the text, reorganize these vocabulary, mark the part of speech of the combined vocabulary, and further identify named entities. The present invention calls the functions of the "dialogue emotion recognition" and "text information extraction" modules. Specifically, the dialogue emotion recognition module of Baidu semantic analysis engine AIS1 is used to identify the emotional information in list L1. The emotional information categories are: negative emotion, neutral emotion, and positive emotion. The negative emotion value is -1, the neutral emotion value is 0, and the positive emotion value is 1. The value of the emotional information is filled in the emotional index variable x attitudeIn the example, the text information extraction module of Baidu semantic analysis engine AIS1 is used to identify the semantic information in list L1. The semantic information categories are: disagree, agree, and wait and see. Disagree is -1, wait and see is 0, and agree is 1. The value of the semantic information is filled in the attitude bias variable x aim Save it separately for later use.

[0031] d) Connect the information database RDB1 to the CASIA Chinese Emotion Corpus via the Internet. The CASIA Chinese Emotion Corpus analyzes the audio files in the information database RDB1 to obtain emotional information. The emotional information categories are: angry, happy, afraid, sad, surprised, and neutral. The angry emotional information is extracted and stored in the variable x. anger Extract the fear emotion information and store it in variable x fear Extract the sad emotional information and store it in variable x sad middle.

[0032] e) Set the sentiment index variable x attitude , attitude bias variable x aim ,variable x anger ,variable x fear ,variable x sad Input into the emotion recognition AIS3 process, the AIS3 process creates a weight list PList, and the weight list PList records the emotion index variable x attitude The weight variable W attitude , attitude bias variable x aim The weight variable W aim ,variable x anger The weight variable W anger ,variable x fear The weight variable W fear ,variable x sad The weight variable W sad and the weight variable W attitude The initial value V0, weight variable W aim The initial value V0, weight variable W anger The initial value V0, weight variable W fear The initial value V0, weight variable W sad The initial value V0 is shown in the following table:

[0033]

[0034] f) Calculate the sentiment information weight W r .

[0035] g) Through the formula Calculate the overall sentiment index α out, where e is a natural constant, α out When it is less than 0, it means disagreement. out When it is greater than 0, it means agreement, α out When it is equal to 0, it means wait-and-see and neutral.

[0036] By collecting voice and audio streams in real time, the emotional stability of the parties involved is analyzed from the two dimensions of semantics and intonation. Deep learning analysis is then performed using AI models to ultimately establish a model for accurately assessing the parties' emotions. While effectively analyzing multiple data sources, the parties' voices are accurately analyzed, emotional fluctuations are identified, and risk levels are determined, providing timely and effective feedback. This method first utilizes a third-party database to obtain a rich set of emotional reference samples. Using semantic and voice recognition systems, these samples are then compared with the newly collected speech to obtain all required parameter values. The corresponding weights are then calculated from a list to determine the difference. Finally, the difference is fed back in real time, and the results are returned to the list, making them more accurate and reasonable, increasing the reference value and enriching the content and diversity of the database.

[0037] In one embodiment of the present invention, the information database RDB1 in step a) utilizes a relational database management system, and its contents are accessible to the semantic recognition system AIS1. In step a), audio file information is captured using an audio capture device, and the captured audio file information is stored in the information database RDB1 in WAV format.

[0038] In one embodiment of the present invention, in step b), the third-party speech semantic recognition library is Baidu speech engine or iFLYTEK speech engine, and the audio files in the information database RDB1 are converted into UTF-8 encoded phrases through the API interface of the third-party speech semantic recognition library.

[0039] In one embodiment of the present invention, the weight variable W in step e) attitude , weight variable W aim , weight variable W anger , weight variable W fear , weight variable W sad The initial value V0 is a random decimal between 0 and 1.

[0040] In one embodiment of the present invention, in step f), the formula W r =(W aim +W anger *x anger +W fear *x fear +W sad *x sad )*x aimCalculate the emotional information weight W r .

[0041] In one embodiment of the present invention, the following steps are further performed after step g):

[0042] h) Calculate the overall sentiment index α out At the same time, the meaning expressed by the speaker in the audio file information is manually collected α real , when the speaker expresses his disagreement, α real The value is -1, when the speaker means to agree, α real When the value is 1, the speaker expresses a wait-and-see attitude. real The value is 0;

[0043] i) Through the formula The difference Δα is obtained by calculation;

[0044] j) Through the formula W′ anger =W anger +W anger *(Δα / α out ) Calculate the new weight variable W′ anger , through the formula W′ fear =W fear +W fear *(Δα / α out ) Calculate the new weight variable W′ fear , through the formula W′ sad =W sad +W sad *(Δα / α out ) Calculate the new weight variable W′ sad , through the formula W′ aim =W aim +W aim *(Δα / α out ) Calculate the new weight variable W′ aim , through the formula W′ attitude =W attitude +W attitude *(Δα / α out ) Calculate the new weight variable W′ attitude ;

[0045] k) The new weight variable W′ anger Put it into the weight list PList and replace the weight variable W anger , the new weight variable W′ fear Put it into the weight list PList and replace the weight variable W fear , the new weight variable W′ sadPut it into the weight list PList and replace the weight variable W sad , the new weight variable W′ aim Put it into the weight list PList and replace the weight variable W aim , the new weight variable W′ attitude Put it into the weight list PList and replace the weight variable W attitude ;

[0046] l) Repeat steps a) to k) N times and output the final overall sentiment index α out In this embodiment, preferably, in step k), the value of N is 1000.

[0047] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art will be able to modify the technical solutions described in the aforementioned embodiments or substitute equivalents for some of the technical features. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A method for AI analysis of speech recognition emotional state, characterized in that: The steps include: a) Building an information database RDB1, which is used to store the collected audio file information; b) connecting the information database RDB1 to a third-party speech and semantic recognition library via the Internet, converting the audio files in the information database RDB1 into phrases through the API interface of the third-party speech and semantic recognition library, and placing the converted phrases into a list L1; c) Submit the vocabulary in list L1 to Baidu's semantic analysis engine AIS1 through API callback. Use the conversation emotion recognition module of Baidu's semantic analysis engine AIS1 to identify the emotional information in list L1. The emotional information categories are: negative emotion, neutral emotion, and positive emotion. Negative emotion is -1, neutral emotion is 0, and positive emotion is 1. The emotional information value is filled in the emotion index variable x attitude In the example, the text information extraction module of Baidu semantic analysis engine AIS1 is used to identify the semantic information in list L1. The semantic information categories are: disagree, agree, and wait and see. Disagree is -1, wait and see is 0, and agree is 1. The value of the semantic information is filled in the attitude bias variable x aim middle; d) Connect the information database RDB1 to the CASIA Chinese Emotion Corpus via the Internet. The CASIA Chinese Emotion Corpus analyzes the audio files in the information database RDB1 to obtain emotional information. The emotional information categories are: angry, happy, afraid, sad, surprised, and neutral. The angry emotional information is extracted and stored in the variable x. anger Extract the fear emotion information and store it in variable x fear Extract the sad emotional information and store it in variable x sad middle; e) Set the sentiment index variable x attitude , attitude bias variable x aim ,variable x anger ,variable x fear ,variable x sad Input into the emotion recognition AIS3 process, the AIS3 process creates a weight list PList, and the weight list PList records the emotion index variable x attitude The weight variable W attitude , attitude bias variable x aim The weight variable W aim ,variable x anger The weight variable W anger ,variable x fear The weight variable W fear ,variable x sad The weight variable W sad and the weight variable W attitude The initial value V0, weight variable W aim The initial value V0, weight variable W anger The initial value V0, weight variable W fear The initial value V0, weight variable W sad The initial value V0; f) Calculate the sentiment information weight W r ; g) Through the formula Calculate the overall sentiment index α out , where e is a natural constant, α out When it is less than 0, it means disagreement. out When it is greater than 0, it means agreement, α out When it is equal to 0, it means wait-and-see and neutral.

2. The AI method for analyzing speech and identifying emotional states according to claim 1, characterized in that: In step a), the information database RDB1 adopts a relational database management system.

3. The AI method for analyzing speech recognition emotion states according to claim 1, characterized in that: In step a), audio file information is collected by using an audio collection device, and the collected audio file information is stored in the information database RDB1 in WAV format.

4. The AI method for analyzing speech recognition emotion states according to claim 1, characterized in that: Step b) The third-party speech semantic recognition library is Baidu speech engine or iFlytek speech engine, and the audio files in the information database RDB1 are converted into UTF-8 encoded phrases through the API interface of the third-party speech semantic recognition library.

5. The AI method for analyzing speech and identifying emotional states according to claim 1, characterized in that: The weight variable W in step e) attitude , weight variable W aim , weight variable W anger , weight variable W fear , weight variable W sad The initial value V0 is a random decimal between 0 and 1.

6. The AI method for analyzing speech and identifying emotional states according to claim 1, characterized in that: In step f), the formula W r = (W aim + W anger * x anger + W fear * x fear + W sad * x sad ) * x aim calculation Calculate the emotional information weight W r .

7. The method of AI analysis of speech recognition emotional state according to claim 1, characterized in that: The method further includes performing the following steps after step g): h) Calculate the overall sentiment index α out At the same time, the meaning of the speaker in the audio file information is manually collected α real , when the speaker expresses his disagreement, α real The value is -1, when the speaker means to agree, α real When the value is 1, the speaker expresses a wait-and-see attitude. real The value is 0; i) Through the formula The difference Δα is obtained by calculation; j) Through the formula W a ' nger =W anger +W anger *(Δα / α out ) Calculate the new weight variable W a ' nger , through the formula W f ' ear =W fear +W fear *(Δα / α out ) Calculate the new weight variable W f ' ear , through the formula W s ' ad =W sad +W sad *(Δα / α out ) Calculate the new weight variable W s ' ad , through the formula W a ' im =W aim +W aim *(Δα / α out ) Calculate the new weight variable W a ' im , through the formula W a ' ttitude =W attitude +W attitude *(Δα / α out ) Calculate the new weight variable W a ' ttitude ; k) The new weight variable W a ' nger Put it into the weight list PList and replace the weight variable W anger , the new weight variable W f ' ear Put it into the weight list PList and replace the weight variable W fear , the new weight variable W s ' ad Put it into the weight list PList and replace the weight variable W sad , the new weight variable W a ' im Put it into the weight list PList and replace the weight variable W aim , the new weight variable W a ' ttitude Put it into the weight list PList and replace the weight variable W attitude ; l) Repeat steps a) to k) N times and output the final overall sentiment index α out .

8. The AI method for analyzing speech recognition emotion states according to claim 7, characterized in that: In step k), the value of N is 1000.

Images