Vehicle intelligent interaction method and device based on emotion perception and context understanding, electronic equipment and storage medium

Abstract

The invention provides a vehicle intelligent interaction method and device based on emotion perception and context understanding, electronic equipment and a storage medium, and the method comprises the steps: obtaining audio data, and generating a voice text based on the audio data, the audio data is generated based on user voice; acquiring context information based on the voice text, and identifying the emotion of the user based on the context information; and determining a response result for the user voice based on the user emotion. The emotion and context of the user can be recognized based on the voice of the user, so that the interaction request of the user is responded.

Description

Technical Field

[0001] The present application relates to the field of vehicle-computer interaction, and more specifically, to a vehicle intelligent interaction method, device, electronic device and storage medium based on emotion perception and context understanding. Background Art

[0002] Among the existing solutions, CN201810805123.0 proposes a method to judge the driver's emotions through facial data. This solution requires real-time image acquisition, which has high requirements on the computing power of the integrated module. In addition, the training process of emotion types requires processing a large number of images, which has high labor costs. Other similar solutions have a single method. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a vehicle intelligent interaction method, device, electronic device and storage medium based on emotion perception and context understanding, so as to recognize the user's emotions and context based on the user's voice, thereby responding to the user's interaction request.

[0004] In a first aspect, the present invention provides a vehicle intelligent interaction method based on emotion perception and context understanding, the method comprising:

[0005] Acquire audio data, and generate voice text based on the audio data, wherein the audio data is generated based on user voice;

[0006] Acquiring context information based on the voice text, and identifying the user emotion based on the context information;

[0007] A response result to the user speech is determined based on the user emotion.

[0008] In a first aspect, the present application acquires audio data and generates voice text based on the audio data, thereby acquiring context information based on the voice text, and further identifying the user's emotion based on the context information, thereby determining a response result to the user's voice based on the user's emotion. In this way, when responding to the user's voice, the user's emotion can be identified based on the context information, thereby improving the accuracy of the response result based on the user's emotion. Compared with the prior art, identifying the user's emotion based on the context information can more accurately acquire the user's emotion based on the context information.

[0009] In an optional embodiment, the user emotions include negative emotions, positive emotions and neutral emotions, the negative emotions include sensitivity, complaint, irritability, insult, complaining, impatience, anger, help-seeking and danger, and the positive emotions include comfort and pleasure.

[0010] This optional implementation can classify user emotions into negative emotions, positive emotions, and neutral emotions, so as to make corresponding response results for negative emotions, positive emotions, and neutral emotions, and further improve the response results. On the other hand, by further classifying negative emotions into sensitivity, complaint, irritability, abuse, complaint, impatience, anger, help-seeking, and danger, and classifying the positive emotions into comfort and pleasure, the matching accuracy of the response results and user emotions can be improved based on finer granularity emotional classification.

[0011] In an optional implementation, the identifying the user emotion based on the context information includes:

[0012] The context information is input into an emotion recognition model so that the emotion recognition model outputs an emotion label for characterizing the user's emotion, wherein the emotion label includes an emotion classification label, an emotion sub-classification label and a global emotion classification label, the emotion classification label is determined based on one of the negative emotion, the positive emotion and the neutral emotion, and the emotion sub-classification label is determined based on the sub-emotion type of the negative emotion, the sub-emotion type of the positive emotion and the sub-emotion type of the neutral emotion.

[0013] This optional implementation may input the context information into an emotion recognition model, thereby enabling the emotion recognition model to output an emotion label for characterizing the user's emotion, thereby determining a response result based on the emotion label.

[0014] In an optional embodiment, the method includes: the emotion recognition model is trained based on a first data set, a second data set and a third data set, wherein the first data set is samples carrying the emotion classification label, the second data set is samples carrying the emotion sub-classification label, and the third data set is samples carrying the global emotion classification label, and the training results of the emotion recognition model are based on training constraint penalty constraints.

[0015] This optional implementation manner may train an emotion recognition model based on the first data set, the second data set, and the third data set. In the process of training the emotion recognition model, by training the penalty constraint, it is possible to ensure that the parent-child relationship remains corresponding and that the parent-child recognition effect is more accurate. In this way, the emotional state of the driver and passengers can be identified more accurately and finely in the model, and the response and interaction mode of the vehicle can be adjusted accordingly.

[0016] In an optional embodiment, the method further comprises:

[0017] Generate sentiment analysis results based on statistical analysis and word vector embedding;

[0018] The output result of the emotion recognition model is optimized based on the sentiment analysis result.

[0019] This optional implementation can generate sentiment analysis results based on statistical analysis and word vector embedding, and can further optimize the output results of the emotion recognition model based on the sentiment analysis results, thereby improving the accuracy of the output results of the emotion recognition model.

[0020] In an optional implementation, the acquiring context information based on the speech text includes:

[0021] Determine semantic scope based on preset thresholds;

[0022] The context information is extracted from the speech text based on the semantic scope.

[0023] This optional implementation can determine the semantic scope based on a preset threshold, and can further flexibly extract the context information from the speech text based on the semantic scope.

[0024] In an optional implementation, the response result includes adjusting volume, pitch and speech speed.

[0025] In a second aspect, the present invention provides a vehicle intelligent interaction device based on emotion perception and context understanding, the device comprising:

[0026] A first acquisition module, used to acquire audio data and generate voice text based on the audio data, wherein the audio data is generated based on user voice;

[0027] A second acquisition module, configured to acquire context information based on the speech text, and identify the user emotion based on the context information;

[0028] A determination module is used to determine a response result to the user's voice based on the user's emotion.

[0029] The second aspect of the present application acquires audio data and generates voice text based on the audio data, thereby acquiring context information based on the voice text, and further identifying the user's emotions based on the context information, thereby determining a response result to the user's voice based on the user's emotions. In this way, when responding to the user's voice, the user's emotions can be identified based on the context information, thereby improving the accuracy of the response result based on the user's emotions. Compared with the prior art, identifying the user's emotions based on the context information can more accurately acquire the user's emotions based on the context information.

[0030] In a third aspect, the present invention provides an electronic device, comprising:

[0031] Processor; and

[0032] A memory configured to store machine-readable instructions, which, when executed by the processor, execute the vehicle intelligent interaction method based on emotion perception and context understanding as described in any of the aforementioned embodiments.

[0033] The electronic device of the third aspect of the present application can obtain audio data and generate voice text based on the audio data by executing a vehicle intelligent interaction method based on emotion perception and context understanding, and can then obtain context information based on the voice text, and can then identify the user's emotions based on the context information, and can then determine the response result to the user's voice based on the user's emotions. In this way, when responding to the user's voice, the user's emotions can be identified based on the context information, thereby improving the accuracy of the response result based on the user's emotions, wherein, compared with the prior art, identifying the user's emotions based on the context information can more accurately obtain the user's emotions based on the context information.

[0034] In a fourth aspect, the present invention provides a storage medium storing a computer program, wherein the computer program is executed by a processor as described in any one of the aforementioned embodiments for the vehicle intelligent interaction method based on emotion perception and context understanding.

[0035] The storage medium of the fourth aspect of the present application executes a vehicle intelligent interaction method based on emotion perception and context understanding, thereby being able to acquire audio data and generate voice text based on the audio data, thereby being able to acquire context information based on the voice text, thereby being able to identify the user's emotions based on the context information, thereby being able to determine a response result to the user's voice based on the user's emotions. In this way, when responding to the user's voice, the user's emotions can be identified based on the context information, thereby improving the accuracy of the response result based on the user's emotions. Compared with the prior art, identifying the user's emotions based on the context information can more accurately acquire the user's emotions based on the context information. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 It is a flowchart of a vehicle intelligent interaction method based on emotion perception and context understanding disclosed in an embodiment of the present application;

[0038] Figure 2 It is a flowchart of another vehicle intelligent interaction method based on emotion perception and context understanding disclosed in an embodiment of the present application;

[0039] Figure 3 is a structural schematic diagram of another vehicle intelligent interaction device based on emotion perception and context understanding disclosed in an embodiment of the present application;

[0040] Figure 4 It is a structural schematic diagram of an electronic device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0041] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0042] Embodiment 1

[0043] See also Figure 1 , Figure 1 is a flow chart of a vehicle intelligent interaction method based on emotion perception and context understanding disclosed in an embodiment of the present application. Figure 1 As shown, the method of the embodiment of the present application includes the following steps:

[0044] 101. Acquire audio data, and generate voice text based on the audio data, wherein the audio data is generated based on user voice;

[0045] 102. Obtaining context information based on the speech text, and identifying user emotions based on the context information;

[0046] 103. Determine a response result to the user's voice based on the user's emotion.

[0047] The embodiments of the present application acquire audio data and generate voice text based on the audio data, thereby acquiring context information based on the voice text, and further identifying the user's emotions based on the context information, thereby determining a response result to the user's voice based on the user's emotions. In this way, when responding to the user's voice, the user's emotions can be identified based on the context information, thereby improving the accuracy of the response result based on the user's emotions. Compared with the prior art, identifying the user's emotions based on the context information can more accurately acquire the user's emotions based on the context information.

[0048] In an embodiment of the present application, for example, in an interactive scenario, after the vehicle computer obtains the voice command input by the user, it recognizes the user's emotion based on the voice command, and then determines the response result to the voice command based on the user's emotion and the context information of each sentence in the voice command. However, the prior art determines the user's emotion based on a certain sentence itself. In comparison, the present application determines the user's emotion based on the context information of the sentence. For example, the user inputs the voice "My car is running out of power, please help me find nearby fast charging stations." The prior art only determines that the user's emotion is neutral based on "Please help me find nearby fast charging stations", and thus randomly selects a fast charging station from fast charging station A and fast charging station B for the user. In comparison, In other words, the present application can split the voice command ""My car is running out of battery, please help me find nearby fast charging stations"" into two sub-sentences ""My car is running out of battery, please help me find nearby fast charging stations", and thus use "My car is running out of battery" as the context of "Please help me find nearby fast charging stations", and thus recognize that the user's current emotion is negative based on "My car is running out of battery", that is, recognize that the user issued the voice command "Please help me find nearby fast charging stations" under a negative emotion, and then in response to the voice command "Please help me find nearby fast charging stations", it selects the nearest fast charging station from fast charging station A and fast charging station B and recommends it to the user, instead of randomly selecting a fast charging station for the user.

[0049] Based on this, the embodiments of the present application can adjust the navigation strategy of the vehicle system according to the emotional state of the driver to provide more suitable route and destination suggestions. For example, if the driver is angry or anxious, the vehicle can give priority to flat and unobstructed roads according to the destination in the script to avoid congestion and tense driving environment. In addition, according to the emotional state of the driver, the vehicle system can automatically adjust the driving mode and auxiliary function settings to provide a better driving experience and safety. For example, if the driver is in a low mood and tends to ask for help, the vehicle system can increase the sensitivity of the driving assistance function, provide more active warnings and interventions, and enhance the driver's sense of security.

[0050] It should be noted that a microphone array or other sound collection equipment can be installed in the car to capture the driver's voice input in real time, or the sound can be obtained by sharing existing equipment. After preprocessing and feature extraction of the driver's voice input data, real-time inference is performed. Based on the output results, the vehicle system can make corresponding responses, such as adjusting the navigation strategy, to provide a personalized driving experience.

[0051] In the embodiment of the present application, for step 101, the audio data can be obtained by audio acquisition, wherein the audio acquisition includes recording and cutting the user's voice and performing analog representation. The audio acquisition uses a microphone or other audio acquisition device to obtain a sound signal. To improve the recognition effect and processing speed, optionally, a 16k sampling rate can be selected for recording.

[0052] In the embodiment of the present application, for step 101, please refer to Figure 2 , Figure 2 FIG. 1 is a flow chart of another vehicle intelligent interaction method based on emotion perception and context understanding disclosed in an embodiment of the present application. Figure 2 As shown, after collecting the user voice, the user voice can also be preprocessed and analog represented, wherein the preprocessing includes: first, using a silence detection algorithm to detect silence segments and non-silence segments in the sound signal. According to the silence detection result, determine the starting point and the ending point of the voice signal, that is, the location of the sound cutting. According to the result of the endpoint detection, the sound signal is cut into independent voice segments. On the other hand, the analog representation includes: the analog representation of the sound signal of the voice segment: x(t), where t represents time. Analog to digital conversion (ADC): x[n] = ADC(x(t)), where x[n] represents a discrete digital signal and n represents the index of the sampling point.

[0053] In the embodiment of the present application, for step 101, after the user voice is pre-processed and simulated, the user voice may be further processed by voice signal enhancement, wherein the voice signal enhancement processing includes: changing the volume of the audio by adjusting the amplitude of the audio, thereby increasing the dynamic range of the data. The formula corresponding to changing the volume of the audio by adjusting the amplitude of the audio is:

[0054] Gain audio (t) = original audio (t) * gain factor.

[0055] Where t represents time, and the gain factor is a value greater than 1 to increase the volume, and a value less than 1 to decrease the volume.

[0056] In the embodiment of the present application, for the step of "generating speech text based on audio data", illustratively, assume that we have an audio input containing multiple audio frames, denoted as X = {x1, x2, ..., xT}, where xt represents the audio features of the tth frame. The goal of ASR is to find the text output Y that is most likely to generate the audio input X. To achieve this goal, ASR uses a triple λ = (A, B, π), where:

[0057] A is the state transition probability matrix, which represents the probability of transitioning from one state to another.

[0058] B is the observation probability matrix, which represents the probability of each state generating the observed feature.

[0059] π is the initial state probability vector, which indicates the probability of being in each state at the beginning.

[0060] The calculation process of ASR can be simply described as follows:

[0061] Initialization: α1(i) = π(i) · B(i, x1), 1 ≤ i ≤ N

[0062] Recursion:

[0063] termination:

[0064] Backtracking: Find the most likely state sequence by comparing the values ​​of αt(i) at each time step

[0065] Q={q1,q2,…,qT}, where qt represents the state of the tth frame.

[0066] Decoding: Use the language model and dictionary to generate the final text output Y based on the state sequence Q.

[0067] In the above ASR process, N represents the number of states, and T represents the number of audio frames. A, B, and π are parameter representations of the ASR inference model obtained by parameter estimation using training data.

[0068] In the embodiment of the present application, optionally, the VAD algorithm can be used to treat the audio stream information of continuous inactive speech not exceeding 750ms and before as a continuous sentence. If the inactive speech of the clause exceeds 750ms, the sentence is segmented, and the VAD-inactive part at the end is removed (the pause part during the speaking process), and the content is regarded as an independent sentence. This step is to perform independent operations on the results of continuous speaking.

[0069] In the embodiment of the present application, with respect to “obtaining context information based on speech text”, illustratively, it is assumed that we have a series of recognized clauses, denoted as S={s1, s2,…, st}, where st represents the recognition result of the tth sentence.

[0070] Initialize context variables: context = [] empty list;

[0071] For each clause st:

[0072] Drop the clause into the database.

[0073] Concatenate the clause st with the context variable and add it to the end of the context variable.

[0074] If the number of sentences in the context variable exceeds the preset context length N, the earliest sentence is removed to keep the context length N.

[0075] For each clause st:

[0076] Concatenate the sentences in the context variable to get the context Ct.

[0077] Furthermore, the clause st can be further processed using context-aware methods (such as language models or other context processing models) to obtain more accurate and coherent results.

[0078] In the above algorithm expression, T represents the number of clauses, N represents the context length, and Ct represents the context of the tth sentence.

[0079] In an optional implementation, user emotions include negative emotions, positive emotions and neutral emotions, negative emotions include sensitivity, complaint, irritability, insults, complaints, impatience, anger, help-seeking and danger, and positive emotions include comfort and pleasure.

[0080] This optional implementation can classify user emotions into negative emotions, positive emotions, and neutral emotions, so as to make corresponding response results for negative emotions, positive emotions, and neutral emotions, and further improve the response results. On the other hand, by further classifying negative emotions into sensitivity, complaint, irritability, insult, complaint, impatience, anger, help, and danger, and classifying positive emotions into comfort and pleasure, the matching accuracy of the response results and user emotions can be improved based on finer granularity emotional classification.

[0081] For the above optional implementation, as an example, according to the common emotions of drivers during driving and the common emotion discrimination criteria, three types of emotions are established, namely positive (id: 1), neutral (id: 0), and negative (id: -1), which are further divided into subclasses such as pleasant, neutral, angry, complaining, and irritable, and the corresponding relationship between subclasses and parent classes is established (the mapping of subclass as key and parent as value). Positive contains two subclasses, comfortable (id: 101) and pleasant (id: 102), neutral contains the subclass neutral (id: 501), and negative contains 9 subclasses, sensitive (id: -101), complaint (id: -102), irritable (id: -103), insult (id: -104), complain (id: -105), impatient (id: -106), angry (id: -107), asking for help (id: -108), and dangerous (id: -109). In this process, three types of emotion labels are established, namely emotion categories, emotion subcategories, and parent-child corresponding categories.

[0082] Furthermore, we complete the multi-category emotion data annotation with sentences as units, and use the above three emotion labels to obtain three data sets, including data set A with three parent classes, data set B with 12 child classes, and data set C with parent-child class corresponding class annotations. That is, there are three types of emotion classification for each sentence (emotion classification (Label: 'positive'}, emotion subclass {Label: 'comfortable'}, global emotion classification {Label: 'positive-thank you'}), and the ABC data sets are used together as further data for the method.

[0083] Furthermore, a network based on the BiLSTM+CRF model is designed for parent-child emotion recognition. First, the call text content Input (call content sentence by sentence) is constructed using lightweight sub-vectors to generate word vectors Vector, and the text is represented in vector form to facilitate the model's data processing. In this step, the input call text sentence (InputSequence) content is generated into a Vector.

[0084] Sample input:

[0085] Call text Input sequence = {"Hello Ford, I am on the highway now and my vehicle is running out of battery"};

[0086] Emotion type Label: Help;

[0087] emotion subcategory id Label_id = -108;

[0088] vector=word2vec(Input sequence)

[0089] Next, the model passes through a bidirectional long short-term memory network, BiLSTM, which can retain semantic information over longer distances in the conversation text.

[0090] Furthermore, the input vector sequence is X=[x1,x2,…,xn], where xi represents the i-th vector.

[0091] Further, BiLSTM layer calculation: The BiLSTM model captures the contextual information of the input sequence through forward and backward LSTM networks. The calculation process is as follows: Forward LSTM: hi+=LSTM(hi-1,xi); Backward LSTM: hi-=LSTM(hi+1,xi) Where hi+ represents the forward hidden state at the i-th moment, and hi- represents the backward hidden state at the i-th moment. Concatenate hidden states: Concatenate the forward and backward hidden states to obtain the hidden state representation of the entire sequence: hi=[hi+;hi-];

[0092] Furthermore, the output sequence data (classification result annotation) is constrained by the conditional random field of the conditional probability model to ensure the legitimacy of the predicted results. It is the Markov random field of the random variable Y under the condition of the given random variable X (here hi). The probability model CRF has a good effect on annotation and classification problems.

[0093] Furthermore, the interactive text emotion recognition model training: through the constructed BiLSTM-CRF model, the constructed data sets A, B, and C are used for training respectively. The specific training method and steps are as follows: use data sets A and B for training so that the model can learn the relationship between categories and texts at different levels, and then aggregate the prediction results of different levels (combine the results) to obtain the final prediction results. It can be understood as consisting of multiple hierarchical classification modules. The recognition of major emotion categories and sub-emotion categories can be understood as each having a local classifier. The classification method based on each level can utilize more fine-grained hierarchical information (major emotion categories, sub-emotion categories). During the training process using data sets A and B, the loss function loss1 of the major emotion category and the loss function loss2 of the sub-emotion category are obtained during training, and the two are gradually reduced through continuous iteration (the identification model prediction is closer to the real result). Then, based on the global layer modeling method, the global modeling is composed of only one classification module, and the established multi-level hierarchical structure information (major emotion category-sub-emotion category) is directly used for modeling. The loss function loss0 of the global sub-emotion category is obtained through training. Each local output layer corresponds to the loss function {loss i}, (i≠0) can force the gradient of the corresponding supervision information containing the corresponding granularity to be passed to the corresponding layer, and the loss function of the global output layer is loss0. The global model parameters can be updated according to the hierarchical structure.

[0094] Furthermore, the joint training constraint penalty is constructed by minimizing the loss function corresponding to the local output layer {loss i}, (i≠0) and the loss function loss0 of the global output layer. The consistency of hierarchical classification cannot be guaranteed. Therefore, the method solves the hierarchical emotion recognition problem by combining the local learning and global learning methods. This step combines multiple local output layers (corresponding to each layer of the hierarchy) and a global output layer. The penalty term pu based on the hierarchy is introduced to constrain the model prediction, ensuring that the parent recognition result corresponds to the child recognition result, reducing the error propagation of the hierarchical recognition result, where pu = λmax{0,loss i -loss0} 2 ,(i≠0), where {loss i}, (i≠0), loss0 represents the loss function of the child node and the parent node respectively, λ is an adjustable parameter used to control the influence of the penalty term pu. Through continuous iteration, the penalty term pu is minimized to achieve the combination of local step-by-step recognition and global recognition capabilities.

[0095] Furthermore, a real-time prediction model is constructed based on the constructed model and the weight information obtained by training the specific task data set. The interactive content during vehicle driving is transcribed through ASR, contextual meaning splicing, and emotion recognition to obtain the intention and emotion of the interacting person. For example, the call text Input sequence = {"Hello Ford, my car is running out of power, please help me find a nearby fast charging station"}, and the emotion result is {-1 negative, -108 help}.

[0096] In an optional implementation, identifying user emotions based on context information includes the following sub-steps:

[0097] The context information is input into the emotion recognition model so that the emotion recognition model outputs an emotion label for characterizing the user's emotion, wherein the emotion label includes an emotion classification label, an emotion sub-classification label and a global emotion classification label, the emotion classification label is determined based on one of negative emotions, positive emotions and neutral emotions, and the emotion sub-classification label is determined based on the sub-emotion type of negative emotions, the sub-emotion type of positive emotions and the sub-emotion type of neutral emotions.

[0098] This optional implementation can input context information into the emotion recognition model, which can then enable the emotion recognition model to output an emotion label for characterizing the user's emotion, thereby determining a response result based on the emotion label.

[0099] In an optional embodiment, the method includes: an emotion recognition model is trained based on a first data set, a second data set, and a third data set, wherein the first data set is samples carrying emotion classification labels, the second data set is samples carrying emotion sub-classification labels, and the third data set is samples carrying global emotion classification labels, and the training results of the emotion recognition model are based on training constraint penalty constraints.

[0100] This optional implementation manner may train an emotion recognition model based on the first data set, the second data set, and the third data set. In the process of training the emotion recognition model, by training the penalty constraint, it is possible to ensure that the parent-child relationship remains corresponding and that the parent-child recognition effect is more accurate. In this way, the emotional state of the driver and passengers can be identified more accurately and finely in the model, and the response and interaction mode of the vehicle can be adjusted accordingly.

[0101] In the embodiment of the present application, as an optional implementation, the method of the embodiment of the present application further includes the following steps:

[0102] Generate sentiment analysis results based on statistical analysis and word vector embedding;

[0103] Optimize the output of the emotion recognition model based on the sentiment analysis results.

[0104] This optional implementation can generate sentiment analysis results based on statistical analysis and word vector embedding, and can further optimize the output results of the emotion recognition model based on the sentiment analysis results, thereby improving the accuracy of the output results of the emotion recognition model.

[0105] In the embodiment of the present application, as an optional implementation, step: obtaining context information based on speech text includes the following steps:

[0106] Determine semantic scope based on preset thresholds;

[0107] Extracting contextual information from spoken text based on semantic scope.

[0108] This optional implementation can determine the semantic scope based on a preset threshold, and can further flexibly extract context information from the speech text based on the semantic scope.

[0109] In an optional embodiment, the response result includes adjusting the volume, pitch and speaking speed.

[0110] Embodiment 2

[0111] See also Figure 3 , Figure 3 is a schematic diagram of a vehicle intelligent interaction device based on emotion perception and context understanding disclosed in an embodiment of the present application. Figure 3 As shown, the device of the embodiment of the present application includes the following functional modules:

[0112] A first acquisition module 201 is used to acquire audio data and generate speech text based on the audio data, wherein the audio data is generated based on user speech;

[0113] A second acquisition module 202, configured to acquire context information based on the speech text, and recognize user emotions based on the context information;

[0114] The determination module 203 is used to determine a response result to the user's voice based on the user's emotion.

[0115] The embodiments of the present application acquire audio data and generate voice text based on the audio data, thereby acquiring context information based on the voice text, and further identifying the user's emotions based on the context information, thereby determining a response result to the user's voice based on the user's emotions. In this way, when responding to the user's voice, the user's emotions can be identified based on the context information, thereby improving the accuracy of the response result based on the user's emotions. Compared with the prior art, identifying the user's emotions based on the context information can more accurately acquire the user's emotions based on the context information.

[0116] Embodiment 3

[0117] See also Figure 4 , Figure 4 Schematic diagram of the structure of an electronic device disclosed in the embodiment of the present application. Figure 4 As shown, the electronic device of the embodiment of the present application includes:

[0118] Processor 301; and

[0119] The memory 302 is configured to store machine-readable instructions, which, when executed by the processor 301, executes a vehicle intelligent interaction method based on emotion perception and context understanding as described in any of the aforementioned embodiments.

[0120] The electronic device of the embodiment of the present application executes a vehicle intelligent interaction method based on emotion perception and context understanding, thereby being able to obtain audio data and generate voice text based on the audio data, thereby being able to obtain context information based on the voice text, thereby being able to identify the user's emotions based on the context information, thereby being able to determine the response result to the user's voice based on the user's emotions. In this way, when responding to the user's voice, the user's emotions can be identified based on the context information, thereby improving the accuracy of the response result based on the user's emotions. Compared with the prior art, identifying the user's emotions based on the context information can more accurately obtain the user's emotions based on the context information.

[0121] Embodiment 4

[0122] An embodiment of the present application provides a storage medium storing a computer program, and the computer program is executed by a processor to implement a vehicle intelligent interaction method based on emotion perception and context understanding as described in any of the aforementioned implementations.

[0123] The storage medium of the embodiment of the present application executes a vehicle intelligent interaction method based on emotion perception and context understanding, thereby being able to acquire audio data and generate voice text based on the audio data, thereby being able to acquire context information based on the voice text, thereby being able to recognize user emotions based on the context information, thereby being able to determine a response result to the user's voice based on the user's emotions. In this way, when responding to the user's voice, the user's emotions can be recognized based on the context information, thereby improving the accuracy of the response result based on the user's emotions. Compared with the prior art, recognizing the user's emotions based on the context information can more accurately acquire the user's emotions based on the context information.

[0124] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0125] In addition, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0126] Furthermore, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0127] It should be noted that if the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can essentially be embodied in the form of a software product, or the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM) random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0128] In this document, relational terms such as first and second, etc. are used merely to distinguish one entity or operation from another entity or operation, but do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0129] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A vehicle intelligent interaction method based on emotion perception and context understanding, characterized in that: The method comprises: Acquire audio data, and generate voice text based on the audio data, wherein the audio data is generated based on user voice; Acquiring context information based on the voice text, and identifying the user emotion based on the context information; A response result to the user speech is determined based on the user emotion.

2. The method according to claim 1, characterized in that The user emotions include negative emotions, positive emotions and neutral emotions. The negative emotions include sensitivity, complaint, irritability, insult, complaint, impatience, anger, help and danger. The positive emotions include comfort and pleasure.

3. The method according to claim 2, characterized in that The identifying the user emotion based on the context information includes: The context information is input into an emotion recognition model so that the emotion recognition model outputs an emotion label for characterizing the user's emotion, wherein the emotion label includes an emotion classification label, an emotion sub-classification label and a global emotion classification label, the emotion classification label is determined based on one of the negative emotion, the positive emotion and the neutral emotion, and the emotion sub-classification label is determined based on the sub-emotion type of the negative emotion, the sub-emotion type of the positive emotion and the sub-emotion type of the neutral emotion.

4. The method according to claim 3, characterized in that The method includes: the emotion recognition model is trained based on a first data set, a second data set and a third data set, wherein the first data set is samples carrying the emotion classification label, the second data set is samples carrying the emotion sub-classification label, and the third data set is samples carrying the global emotion classification label, and the training result of the emotion recognition model is based on a training constraint penalty constraint.

5. The method according to claim 3, characterized in that The method further comprises: Generate sentiment analysis results based on statistical analysis and word vector embedding; The output result of the emotion recognition model is optimized based on the sentiment analysis result.

6. The method according to claim 1, characterized in that The acquiring context information based on the speech text includes: Determine semantic scope based on preset thresholds; The context information is extracted from the speech text based on the semantic scope.

7. The method according to claim 1, characterized in that The response results include adjusting the volume, pitch and speaking speed.

8. A vehicle intelligent interaction device based on emotion perception and context understanding, characterized in that: The device comprises: A first acquisition module, used to acquire audio data and generate voice text based on the audio data, wherein the audio data is generated based on user voice; A second acquisition module, configured to acquire context information based on the speech text, and identify the user emotion based on the context information; A determination module is used to determine a response result to the user's voice based on the user's emotion.

9. An electronic device, characterized in that: include: processor; as well as A memory configured to store machine-readable instructions, which, when executed by the processor, execute the vehicle intelligent interaction method based on emotion perception and context understanding as described in any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium stores a computer program, and the computer program is executed by a processor as the vehicle intelligent interaction method based on emotion perception and context understanding as described in any one of claims 1 to 7.

Images