An AI interactive scene method with voice recognition function

Abstract

The present invention provides an AI interactive scene method with a speech recognition function, the system comprising: S1, obtaining a speech input signal, converting the speech input signal into a digital signal through an audio acquisition device and performing denoising and framing processing to obtain a pre-processed speech frame; S2, extracting a tone fluctuation parameter and a volume intensity parameter containing a time series from the pre-processed speech frame, and inputting the above parameters into an emotion-acoustic feature association model to generate an acoustic emotion label category, and calculating an acoustic emotion confidence value P audio By utilizing the emotion-acoustic feature association model and the emotion-text feature association model for emotion recognition, and by comparing the correlation between the two, we can ensure the accuracy of emotion recognition. By dynamically adjusting the interaction strategy, we can achieve emotionally intelligent human-computer interaction and improve the user experience.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and in particular to an AI interactive scene method with a speech recognition function. Background Art

[0002] In the field of artificial intelligence and human-computer interaction, emotional response technology based on speech recognition is becoming a key research direction. This field is important because it attempts to capture emotional signals in human speech, enhancing the intelligence and humanization of interactive systems, thereby meeting users' emotional needs in various situations. With the widespread use of voice interaction devices, how to enable AI to more naturally understand and adapt to changes in human emotions is not only crucial for optimizing user experience but also directly impacts the technology's ubiquity and acceptance.

[0003] However, existing methods still have significant limitations when handling emotional voice interactions. Many systems rely on analyzing single speech features, such as pitch or speech rate, and lack comprehensive assessment of multi-dimensional emotional cues, resulting in inaccurate emotion recognition. Furthermore, most solutions are overly mechanical in adjusting AI responses, making it difficult to dynamically match the user's emotional state. The interaction process often appears stilted and fails to truly meet the user's psychological needs. Summary of the Invention

[0004] The present invention provides an AI interactive scene method with a speech recognition function, the system comprising:

[0005] S1. Acquire a voice input signal, convert the voice input signal into a digital signal through an audio acquisition device, and perform denoising and framing processing to obtain a preprocessed voice frame;

[0006] S2, extracting the intonation fluctuation parameters and volume intensity parameters containing the time series from the pre-processed speech frame, and inputting the above parameters into the emotion-acoustic feature association model to generate the acoustic emotion label category and calculate the acoustic emotion confidence value P audio ;

[0007] S3, convert the pre-processed speech frame into text, input the text into the emotion-text feature association model, thereby generating the text emotion label category and calculating the text emotion confidence value P text ;

[0008] S4. According to the acoustic emotion category label and acoustic emotion confidence value P audio , text emotion category label, text emotion confidence value P text Perform interactive responses, where the types of interactive responses include:

[0009] When the acoustic emotion label category is exactly the same as the text emotion label category, and the acoustic emotion confidence value Paudio and text sentiment confidence value P text If both are greater than the set threshold, the AI ​​interaction scenario adopts a high-confidence emotional response;

[0010] When the acoustic emotion label category and the text emotion label category belong to the same category, and the acoustic emotion confidence value P audio and text sentiment confidence value P text If both are less than the set threshold, the AI ​​interaction scenario adopts a universal neutral response;

[0011] When the acoustic emotion label category is negative emotion and the acoustic emotion confidence value P audio Greater than the text sentiment confidence value P text , and the difference is greater than the set threshold, the AI ​​interactive scene uses the high-score label response;

[0012] When the acoustic emotion label category and the text emotion label category are mutually exclusive, and the acoustic emotion confidence value P audio and text sentiment confidence value P text If both are less than the set threshold, the AI ​​interaction scenario adopts detection response.

[0013] Furthermore, in step S4, the AI ​​interactive scenario adopts a high-confidence emotional response, specifically including: dynamically adjusting the tone and speed of the interaction based on a JSON template library of predefined different emotions to enhance emotional expression.

[0014] Furthermore, in step S4, the AI ​​interaction scenario adopts a universal neutral response, specifically including: maintaining a stable fundamental frequency, a moderate speaking speed, and avoiding emotional fluctuations.

[0015] Furthermore, in step S4, the AI ​​interactive scenario uses high-scoring label responses, specifically including: shortening speech synthesis delay, prioritizing the output of soothing content, and using emergency response templates.

[0016] Furthermore, in step S4, the detection response adopted in the AI ​​interactive scenario specifically includes: adopting a clarification mechanism to explicitly require the user to confirm the emotional tendency.

[0017] Furthermore, the text emotion label categories and acoustic emotion label categories include “happy label”, “sad label”, “angry label”, “fear label”, “surprise label”, “disgust label”, and “calm label”.

[0018] Furthermore, the “happy label” and “surprise label” are classified into the positive emotion category; the “angry label”, “fear label” and “disgust label” are classified into the negative emotion category; and the “calm label” is classified into the neutral emotion category.

[0019] Furthermore, in step S2, the emotion-acoustic feature association model is constructed by the following steps, specifically including:

[0020] S21. Collect audio samples in multiple languages ​​and scenarios;

[0021] S22. Label each audio sample with the corresponding emotion label:

[0022] S23. Extract acoustic features from audio samples and input them into a deep learning model to generate a mapping relationship between acoustic features and emotions, thereby defining a threshold range of feature parameters for each emotion and generating emotional baselines for various emotions, thereby forming an emotion-acoustic feature association model.

[0023] Furthermore, in step S3, the pre-processed speech frame is converted into text, and the text is input into the emotion-text feature association model to generate text emotion label categories, which specifically includes:

[0024] S31, performing text conversion on the pre-processed speech frame;

[0025] S32. Use a word frequency statistics algorithm to analyze the converted text, generate an emotional vocabulary feature vector based on the emotional dictionary, map the words in the text and generate text emotional label categories.

[0026] The beneficial effects of the present invention are: by utilizing the emotion-acoustic feature association model and the emotion-text feature association model to perform emotion recognition, and through the correlation between the two, the accuracy of emotion recognition is ensured, and by dynamically adjusting the interaction strategy, emotionally intelligent human-computer interaction is achieved, thereby improving user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a flow chart of an AI interactive scene method with speech recognition function of the present invention. DETAILED DESCRIPTION

[0028] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. It should be noted that the description of these embodiments is intended to help understand the present invention, but does not constitute a limitation of the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0029] The present invention provides an AI interactive scene method with a speech recognition function, the system comprising:

[0030] S1. Acquire a voice input signal, convert the voice input signal into a digital signal through an audio acquisition device, and perform denoising and framing processing to obtain a preprocessed voice frame;

[0031] S2, extracting the intonation fluctuation parameters and volume intensity parameters containing the time series from the pre-processed speech frame, and inputting the above parameters into the emotion-acoustic feature association model to generate the acoustic emotion label category and calculate the acoustic emotion confidence value P audio ;

[0032] S3, convert the pre-processed speech frame into text, input the text into the emotion-text feature association model, thereby generating the text emotion label category and calculating the text emotion confidence value P text ;

[0033] S4. According to the acoustic emotion category label and acoustic emotion confidence value P audio , text emotion category label, text emotion confidence value P text Perform interactive responses, where the interactive response types include: high-confidence emotional responses, general neutral responses, high-score label responses, and detection responses.

[0034] Specifically, when the acoustic emotion label category is exactly the same as the text emotion label category, and the acoustic emotion confidence value P audio and text sentiment confidence value P text are greater than the set threshold, for example, P audio >0.8, P text If the value is greater than 0.8, the AI ​​interaction scenario uses a high-confidence emotional response. High-confidence emotional responses are suitable for users who clearly express strong emotions, such as laughter or sad crying.

[0035] AI interactive scenarios use high-confidence emotional responses, specifically including: dynamically adjusting the tone and speed of interaction based on a JSON template library of predefined emotions to enhance emotional expression.

[0036] Specifically, when the acoustic emotion tag category matches the text emotion tag category and is completely consistent, the system responds to the user in a highly emotional and targeted interactive manner, adjusting the tone, speed, and emphasis to match the intensity of the emotion.

[0037] For example, when speaking in a "happy tag," the voice pitch is raised and the speaking speed is increased, for example, by 15% and 20%, with a slight smile added to the end. When speaking in a "sad tag," the voice pitch is lowered and the pauses are extended. When speaking in an "excited tag," the speech synthesizer adds a slight vibrato and a higher fundamental frequency.

[0038] When responding to high-confidence emotions, AI interactive scenarios can provide emotional vocabulary and targeted suggestions. Example (happy): "I've detected that you're in a very happy mood! I recommend today's popular activities: concerts, outdoor hiking, would you like to know more?"

[0039] Furthermore, visual feedback can be triggered, such as when the avatar in the AI ​​interaction scene performs corresponding actions. For example, when the avatar in the AI ​​interaction scene is labeled "happy", the triggered visual feedback includes the avatar laughing, active body movements, and the interface color is adjusted to bright tones such as yellow or orange.

[0040] When the acoustic emotion label category and the text emotion label category belong to the same category, and the acoustic emotion confidence value P audio and text sentiment confidence value P text If both are below the set threshold, the AI ​​interaction scenario adopts a universal neutral response. Universal neutral response is suitable for users who have a tendency to express emotions but are not clear about it.

[0041] Among them, the AI ​​interactive scenario adopts a universal neutral response to dynamically adjust the tone and speed of the interaction, specifically including: maintaining a stable fundamental frequency, a moderate speaking speed, and avoiding emotional fluctuations.

[0042] For example, the output voice for AI interactive scenarios is a news broadcast-style tone, without emotional fluctuations. The avatar for AI interactive scenarios is a standing avatar. The speech template for AI interactive scenarios uses open-ended questions or risk-free responses (such as "You seem to have a need. How can I help you?").

[0043] Specifically, the text emotion label categories and acoustic emotion label categories include "happy label," "sad label," "angry label," "fear label," "surprise label," "disgust label," and "calm label." Furthermore, the "happy label" and "surprise label" are classified as positive emotions; the "angry label," "fear label," and "disgust label" are classified as negative emotions; and the "calm label" is classified as neutral emotions.

[0044] The acoustic emotion label category and the text emotion label category belong to the same category, that is, the acoustic emotion label is "Excited" and the text emotion label is "Happy". They belong to the same category, but they are not completely consistent, and the acoustic emotion confidence value P audio and text sentiment confidence value P text are all less than the set threshold, for example, P audio <0.6, P text <0.6, the AI ​​interaction scenario adopts a universal neutral response.

[0045] When the acoustic emotion label category is negative emotion and the acoustic emotion confidence value P audio Greater than the text sentiment confidence value P text , and the difference is greater than the set threshold, the AI ​​interactive scene uses the high-score label response.

[0046] High-scoring label responses are appropriate for users experiencing emotional states such as anger or disgust. For example, the user gasps for air and says, "Your system crashed again?!" (with a sudden increase in volume and rapid fluctuations in intonation).

[0047] AI interactive scenarios use high-scoring label responses to dynamically adjust the tone and speed of interaction, specifically by shortening speech synthesis delays, prioritizing the output of soothing content, and using emergency response templates.

[0048] For example, lower your voice by 10%, add a short pause, and emphasize: "Sorry-" (imitate a soothing tone). Conversation content: "We've detected that you may be dissatisfied with our service! We've initiated an emergency repair process and expect it to be restored within 5 minutes."

[0049] In addition, AI interaction scenarios can also be implemented through the following operations, such as interface warnings: displaying a red warning icon and an emergency contact entry; resource allocation: triggering a priority increase in backend services (such as customer service system queue jumping), etc.

[0050] When the acoustic emotion label category and the text emotion label category are mutually exclusive, and the acoustic emotion confidence value P audio and text sentiment confidence value P text If both are below the set threshold, the AI ​​interaction scenario uses a detection response. Detection response is suitable for situations where the user's emotion cannot be confirmed, such as when the user inputs a "sarcastic" voice.

[0051] Among them, the detection response used in AI interactive scenarios specifically includes: adopting a clarification mechanism to explicitly require users to confirm their emotional tendencies.

[0052] For example, the user sneered and said "very good job" (the acoustic emotion label is "sad label", P audio =0.55<0.6; text label "happy label", P text =0.58<0.6), in this case, the AI ​​interactive scenario can output the content: "You emphasized 'really good'. Are you satisfied or do you have other suggestions?", thereby explicitly asking the user to confirm their emotional tendency.

[0053] Furthermore, in step S2, the emotion-acoustic feature association model is constructed by the following steps, specifically including:

[0054] S21. Collect audio samples in multiple languages ​​and scenarios;

[0055] S22. Label each audio sample with the corresponding emotion label:

[0056] S23. Extract acoustic features from audio samples and input them into a deep learning model to generate a mapping relationship between acoustic features and emotions, thereby defining a threshold range of feature parameters for each emotion and generating emotional baselines for various emotions, thereby forming an emotion-acoustic feature association model.

[0057] Mel-frequency cepstral coefficient algorithm and short-time Fourier transform algorithm are used to extract acoustic features from audio. The acoustic features include: volume feature vector (short-time energy mean, dynamic range (dB), energy entropy, zero-crossing rate), intonation feature vector (fundamental frequency (F0) mean, standard deviation, range, first-order difference of fundamental frequency change rate).

[0058] The acoustic features above are used as training data. Deep learning models such as random forests or neural networks are then used to learn the mapping between features and emotions. This allows for defining threshold ranges for feature parameters for each emotion and generating emotional baselines for each emotion. These emotional baselines include the threshold ranges for each emotion (e.g., "anger" requires μF0 > 180Hz, σF0 > 40Hz, and DRE > 25dB).

[0059] The Mel frequency cepstral coefficient algorithm is used to extract features using the following formula:

[0060]

[0061] In the above formula, Indicates the The first acoustic sample MFCC coefficients, Represents an audio sample.

[0062] Represents the audio sample The vector composed of the acoustic features extracted from Calculated by the following formula:

[0063]

[0064] In the above formula, They are MFCC coefficient, fundamental frequency, energy, spectrum centroid and other features respectively.

[0065] Then output the probability formula through LSTM:

[0066]

[0067] In the above formula, A vector representing the composition of acoustic features; Indicates the Emotion labels (such as "happy label", "angry label", "sad label", etc.); LSTM The output value of hidden units; Represents the weight parameter, which is used to adjust the contribution of different hidden units to emotion classification.

[0068] Generate the emotional baseline through the emotional baseline formula:

[0069]

[0070] In the above formula, Indicates the The MFCC coefficient features are The mean value under the class sentiment, Indicates the The first acoustic sample MFCC coefficients, Example of numbering emotion categories: =1 means "happy label", = means "angry label" etc.; Indicates that it belongs to The total number of samples of the emotion class.

[0071] Finally, the acoustic emotion confidence value P audio Perform the calculation:

[0072]

[0073] In the above formula, represents the feature matching function, Indicates the The first acoustic sample MFCC coefficients; Indicates the Emotional The baseline mean of the feature.

[0074] Furthermore, in step S3, the pre-processed speech frame is converted into text, and the text is input into the emotion-text feature association model to generate text emotion label categories, which specifically includes:

[0075] S31, performing text conversion on the pre-processed speech frame;

[0076] S32. Use a word frequency statistics algorithm to analyze the converted text, generate an emotional vocabulary feature vector based on the emotional dictionary, map the words in the text and generate text emotional label categories.

[0077] In step S32, the steps of analyzing the converted text using a word frequency statistics algorithm, generating an emotion vocabulary feature vector based on an emotion dictionary, and mapping the words in the text to generate text emotion label categories specifically include:

[0078] Define the input text as T={w1,w2,…,w n}, where w i Represents the i-th word in the text, n is the total number of words in the text, and the word frequency statistical vector is defined as f∈R m , where m is the size of the vocabulary, then Represents the frequency of the jth word in the vocabulary in the text T:

[0079]

[0080] In the above formula, is the first vocabulary, It is an indicator function that takes the value 1 when the condition is met and 0 otherwise.

[0081] In order to eliminate the influence of text length, the word frequency can be normalized:

[0082] .

[0083] The sentiment dictionary is a mapping table from vocabulary to sentiment categories. Assume that the sentiment dictionary contains There are emotion categories (such as happy, sad, angry, etc.), each emotion category corresponds to a weight vector w k ∈R m , where w k [j] represents the weight of the j-th word in the vocabulary for sentiment category k.

[0084] The sentiment lexicon can be represented as: .

[0085] The normalized word frequency vector f norm Combined with the sentiment dictionary W, the sentiment vocabulary feature vector is generated as follows:

[0086]

[0087] In the above formula, represents the sentiment score of text T for sentiment category k.

[0088] Based on sentiment score , select the emotion category with the highest score as the final emotion label:

[0089] .

[0090] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like to indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0091] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, features specified as "first" or "second" may explicitly or implicitly include one or more of the features.

[0092] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. An AI interactive scene method with speech recognition function, characterized in that: The AI ​​interaction method includes: S1. Acquire a voice input signal, convert the voice input signal into a digital signal through an audio acquisition device, and perform denoising and framing processing on the signal to obtain a preprocessed voice frame; S2, extracting the intonation fluctuation parameters and volume intensity parameters containing the time series from the pre-processed speech frame, and inputting the above parameters into the emotion-acoustic feature association model to generate the acoustic emotion label category and calculate the acoustic emotion confidence value P audio ; S3, convert the pre-processed speech frame into text, input the text into the emotion-text feature association model, thereby generating the text emotion label category and calculating the text emotion confidence value P text ; S4. According to the acoustic emotion category label and acoustic emotion confidence value P audio , text emotion category label, text emotion confidence value P text Perform interactive responses, where the types of interactive responses include: When the acoustic emotion label category is exactly the same as the text emotion label category, and the acoustic emotion confidence value P audio and text sentiment confidence value P text If both are greater than the set threshold, the AI ​​interaction scenario adopts a high-confidence emotional response; When the acoustic emotion label category and the text emotion label category belong to the same category, and the acoustic emotion confidence value P audio and text sentiment confidence value P text If both are less than the set threshold, the AI ​​interaction scenario adopts a universal neutral response; When the acoustic emotion label category is negative emotion and the acoustic emotion confidence value P audio Greater than the text sentiment confidence value P text , and the difference is greater than the set threshold, the AI ​​interactive scene uses the high-score label response; When the acoustic emotion label category and the text emotion label category are mutually exclusive, and the acoustic emotion confidence value P audio and text sentiment confidence value P text If both are less than the set threshold, the AI ​​interaction scenario adopts detection response.

2. The AI ​​interactive scene method with speech recognition function according to claim 1, characterized in that: In step S4, the AI ​​interactive scenario adopts high-confidence emotional response, specifically including: dynamically adjusting the tone and speed of interaction based on the JSON template library of predefined different emotions to enhance emotional expression.

3. The AI ​​interactive scene method with speech recognition function according to claim 1, characterized in that: In step S4, the AI ​​interaction scenario adopts a universal neutral response, specifically including: maintaining a stable fundamental frequency, a moderate speaking speed, and avoiding emotional fluctuations.

4. The AI ​​interactive scene method with speech recognition function according to claim 1, characterized in that: In step S4, the AI ​​interactive scenario uses high-scoring label responses, specifically including: shortening speech synthesis delay, prioritizing the output of soothing content, and using emergency response templates.

5. The AI ​​interactive scene method with speech recognition function according to claim 1, characterized in that: In step S4, the detection response adopted in the AI ​​interaction scenario specifically includes: adopting a clarification mechanism to explicitly require the user to confirm the emotional tendency.

6. The AI ​​interactive scene method with speech recognition function according to claim 1, characterized in that: The text emotion label categories and acoustic emotion label categories include "happy label", "sad label", "angry label", "fear label", "surprise label", "disgust label", and "calm label".

7. The AI ​​interactive scene method with speech recognition function according to claim 6, characterized in that: The "happy label" and "surprise label" are classified as positive emotions; the "anger label", "fear label" and "disgust label" are classified as negative emotions; and the "calm label" is classified as neutral emotions.

8. The AI ​​interactive scene method with speech recognition function according to claim 1, characterized in that: In step S2, the emotion-acoustic feature association model is constructed by the following steps, specifically including: S21. Collect audio samples in multiple languages ​​and scenarios; S22. Label each audio sample with the corresponding emotion label: S23. Extract acoustic features from audio samples and input them into a deep learning model to generate a mapping relationship between acoustic features and emotions, thereby defining a threshold range of feature parameters for each emotion, generating emotional baselines for various emotions, and thus forming an emotion-acoustic feature association model.

9. The AI ​​interactive scene method with speech recognition function according to claim 1, characterized in that: In step S3, the pre-processed speech frame is converted into text, and the text is input into the emotion-text feature association model to generate text emotion label categories, which specifically includes: S31, performing text conversion on the pre-processed speech frame; S32. Use a word frequency statistics algorithm to analyze the converted text, generate an emotional vocabulary feature vector based on the emotional dictionary, map the words in the text and generate text emotional label categories.

Images