AI interaction method and system based on emotion recognition

Abstract

The invention belongs to the technical field of AI interaction, and discloses an AI interaction method and system based on emotion recognition. The method comprises the following steps that in a cloud data center, an emotion relieving database is built, and an emotion AI interaction engine is built; according to the real-time multi-mode monitoring data, using an emotion AI interaction engine to generate a real-time emotion recognition result and real-time AI interaction data; according to the real-time emotion recognition result, a real-time AI response strategy is generated, and real-time emotion relieving data is extracted; generating real-time AI response data according to the real-time AI response strategy, the real-time AI interaction data and the real-time emotion relieving data; and optimizing the emotion AI interaction engine according to the real-time feedback data to obtain an optimized emotion AI interaction engine. According to the method, the problems of low emotion recognition accuracy, low AI intelligence degree and lack of an evolution function in the prior art are solved.

Description

Technical Field

[0001] The present invention belongs to the field of AI interaction technology, and specifically relates to an AI interaction method and system based on emotion recognition. Background Art

[0002] With the acceleration of social pace and the increase of life pressure, more and more people are facing emotional problems. In recent years, the development of artificial intelligence technology has provided new solutions for emotional comfort, and emotional comfort based on artificial intelligence (AI) has been developed. AI-based emotional comfort refers to the use of artificial intelligence technology, especially natural language processing and machine learning, to understand and respond to human emotional needs. This technology can be used in a variety of scenarios, such as mental health support, customer service, educational counseling, etc. The AI ​​system judges the user's emotional state by analyzing the user's language, tone, expression, and body language, and provides corresponding comfort and suggestions.

[0003] Although existing technologies have made some progress in emotion recognition and intelligent emotional soothing, they still have some obvious defects and shortcomings: 1) Low accuracy of emotion recognition: In existing technologies, emotion recognition is often based on single-modal data, including single expressions, voice and text data. However, this simple method cannot represent the user's true emotional characteristics, and the accuracy of recognition is still challenged, especially in complex and changing environments. For example, the user's expression, voice and text data may be affected by factors such as light, noise, and speaking speed, resulting in deviations in the recognition results.

[0004] 2) Low intelligence level of AI: In existing technologies, simple large language models are often used for programmed question-and-answer dialogues, which have a low level of intelligence and are not suitable for multi-mode and multi-modal AI interactions. They also lack customized dialogue functions and have low practicality.

[0005] 3) Lack of evolutionary function: In the existing technology, the functional unit of AI interaction cannot automatically optimize the model based on customer feedback, resulting in the lack of evolutionary function of the model version, which cannot be iteratively updated according to usage, and poor user experience. Summary of the invention

[0006] In order to solve the problems of low accuracy of emotion recognition, low intelligence level of AI and lack of evolutionary function in the prior art, the purpose of the present invention is to provide an AI interaction method and system based on emotion recognition.

[0007] The technical solution adopted by the present invention is: An AI interaction method based on emotion recognition includes the following steps: In the cloud data center, we built an emotion relief database and used multimodal fusion artificial intelligence algorithms to build an emotional AI interaction engine. Collect users' real-time multimodal monitoring data, and use the emotion AI interaction engine to generate real-time emotion recognition results and real-time AI interaction data based on the real-time multimodal monitoring data; Based on the real-time emotion recognition results, the emotion AI interaction engine is used to generate a real-time AI response strategy, and based on the real-time AI response strategy, real-time emotion relief data is extracted from the emotion relief database; Generate real-time AI response data using the emotional AI interaction engine based on real-time AI response strategies, real-time AI interaction data, and real-time emotion relief data; Collect real-time feedback data from users, and optimize the emotion AI interaction engine based on the real-time feedback data to obtain an optimized emotion AI interaction engine.

[0008] Furthermore, the emotional AI interaction engine includes a multimodal feature fusion model, an emotion recognition model, a speech recognition model, an AI response strategy generation model, an AI response model, and a speech reconstruction model; The emotion relief database is provided with a plurality of preset emotion relief data of different emotion relief types, the preset emotion relief data including preset emotion relief picture data, preset emotion relief audio data, preset emotion relief text data and preset emotion relief video data of different emotion relief types, and each preset emotion relief data is provided with corresponding preset data attribute information; Real-time multimodal monitoring data includes real-time expression image monitoring data, real-time interactive voice data, and real-time interactive text data of users; Real-time AI response data includes real-time AI response text data, real-time AI response voice data, and real-time emotion relief data.

[0009] Furthermore, in the cloud data center, an emotion relief database is built, and an emotional AI interaction engine is constructed using a multimodal fusion artificial intelligence algorithm, including the following steps: In the cloud data center, a plurality of preset emotion relief data of different emotion relief types on the Internet are collected, and corresponding preset data attribute information is set to obtain an emotion relief database; Based on several historical multimodal monitoring data, a multimodal feature fusion model is constructed using a multimodal fusion deep learning algorithm, and several historical multimodal fusion features are generated; Based on several historical multimodal fusion features, a deep learning algorithm is used to build an emotion recognition model and generate several historical emotion recognition results; Based on some historical multimodal monitoring data, a speech recognition model is built using a deep learning and natural language processing fusion algorithm, and some historical AI interaction data is generated; Based on several historical emotion recognition results, a reinforcement learning algorithm is used to build an AI response strategy generation model, and several historical AI response strategies and corresponding historical AI response strategy generation experiences are generated; Based on several historical AI response strategies, several historical AI interaction data, and several preset emotion relief data, a large language model algorithm is used to build an AI response model and generate several historical AI response text data; Based on some historical AI response text data, a speech reconstruction model is constructed using audio signal processing algorithms.

[0010] The multimodal feature fusion model, emotion recognition model, speech recognition model, AI response strategy generation model, AI response model and speech reconstruction model are integrated to obtain the emotion AI interaction engine.

[0011] Furthermore, the multimodal feature fusion model is constructed based on the CNN-Transformer-logfBank-Attention-MLP algorithm; The emotion recognition model is built based on the BiLSTM algorithm; The speech recognition model is built based on the LSTM-CRF algorithm; The AI ​​response strategy generation model is built based on the DQN algorithm; The AI ​​response model is built on the BERT algorithm; The speech reconstruction model is built based on the GAN-Tacotron2 algorithm.

[0012] Furthermore, real-time multimodal monitoring data of users is collected, and based on the real-time multimodal monitoring data, an emotion AI interaction engine is used to generate real-time emotion recognition results and real-time AI interaction data, including the following steps: Collecting real-time multimodal monitoring data of users, and preprocessing the real-time multimodal monitoring data to obtain preprocessed real-time multimodal monitoring data; The pre-processed real-time multimodal monitoring data is input into the emotional AI interaction engine, and the corresponding real-time multimodal fusion features are generated using the multimodal feature fusion model; After preprocessing, the real-time interactive speech data in the real-time multimodal monitoring data is subjected to emotion recognition using an emotion recognition model to obtain corresponding real-time emotion recognition results; Based on the pre-processed real-time multimodal monitoring data, the speech recognition model is used to perform speech recognition to obtain the corresponding real-time AI interaction data.

[0013] Further, according to the real-time emotion recognition result, the emotion AI interaction engine is used to generate a real-time AI response strategy, and according to the real-time AI response strategy, the real-time emotion relief data is extracted from the emotion relief database, including the following steps: According to the real-time emotion recognition results, the AI ​​response strategy generation model of the emotion AI interaction engine is used to generate the AI ​​response strategy and obtain the real-time AI response strategy; According to the real-time AI response strategy, a search and match is performed in a number of preset data attribute information in the emotion relief database to obtain a number of successfully matched real-time data attribute information; A plurality of real-time data attribute information and corresponding preset emotion relief data are output to obtain a plurality of real-time emotion relief data.

[0014] Further, according to the real-time AI response strategy, the real-time AI interaction data and the real-time emotion relief data, the emotional AI interaction engine is used to generate real-time AI response data, including the following steps: Generate real-time AI response text data using the AI ​​response model of the emotional AI interaction engine based on the real-time AI response strategy, real-time AI interaction data, and real-time data attribute information; According to the real-time AI response text data, the speech reconstruction model is used to reconstruct the speech to obtain the real-time AI response speech data; Integrate real-time AI response text data, real-time AI response voice data, and real-time emotion relief data to obtain real-time AI response data.

[0015] Furthermore, real-time feedback data from users is collected, and the emotion AI interaction engine is optimized according to the real-time feedback data to obtain an optimized emotion AI interaction engine, including the following steps: Collect and analyze real-time feedback data from users to obtain performance parameters of the emotional AI interaction engine; According to the performance parameters, set the optimization objective function of the emotional AI interaction engine and set the solution vector encoding of the swarm intelligence optimization algorithm; According to the optimization objective function and solution vector encoding, the swarm intelligence optimization algorithm is used to iteratively search for the optimal solution to obtain the optimized model parameters corresponding to the optimal solution; According to the optimized model parameters, the emotion AI interaction engine is optimized to obtain the optimized emotion AI interaction engine.

[0016] Furthermore, the swarm intelligence optimization algorithm is an ICPO optimization algorithm. An AI interaction system based on emotion recognition is used to implement an AI interaction method. The system includes a cloud data center and several user terminals. The several user terminals are communicatively connected to the cloud data center. The cloud data center is provided with an emotion AI interaction engine and an emotion relief database.

[0017] The beneficial effects of the present invention are: The present invention discloses an AI interaction method and system based on emotion recognition, which performs emotion recognition according to multimodal fusion features, avoids recognition bias caused by interference information of single modal features in complex and changeable environments, and improves the accuracy of emotion recognition; an emotion AI interaction engine constructed based on an artificial intelligence algorithm realizes emotion recognition, speech recognition, AI response strategy generation and AI response generation, improves the degree of intelligence, can perform multi-mode and multi-modal customized AI interaction and dialogue according to user emotions, improves practicality, and outputs emotion relief data according to user emotions, realizing an accurate emotion soothing function; according to feedback data, the emotion AI interaction engine is optimized, the model automatic evolution function is realized, and it can be iteratively updated according to usage conditions to meet the needs of different users and improve user experience.

[0018] Other beneficial effects of the present invention will be further described in the specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flowchart of the AI ​​interaction method based on emotion recognition in the present invention.

[0020] Figure 2 It is a structural block diagram of the AI ​​interactive system based on emotion recognition in the present invention. DETAILED DESCRIPTION

[0021] The present invention will be further explained below in conjunction with the accompanying drawings and specific embodiments.

[0022] Embodiment 1: like Figure 1 As shown, this embodiment provides an AI interaction method based on emotion recognition, comprising the following steps: S1: In the cloud data center, build an emotion relief database and use multimodal fusion artificial intelligence algorithms to build an emotional AI interaction engine; The emotional AI interaction engine includes a multimodal feature fusion model, an emotion recognition model, a speech recognition model, an AI response strategy generation model, an AI response model, and a speech reconstruction model; In the cloud data center, we build an emotion relief database and use a multimodal fusion artificial intelligence algorithm to build an emotional AI interaction engine, including the following steps: S1-1: In a cloud data center, a plurality of preset emotion relief data of different emotion relief types on the Internet are collected, and corresponding preset data attribute information is set to obtain an emotion relief database; The emotion relief database is provided with a plurality of preset emotion relief data of different emotion relief types, the preset emotion relief data including preset emotion relief picture data, preset emotion relief audio data, preset emotion relief text data and preset emotion relief video data of different emotion relief types, and each preset emotion relief data is provided with corresponding preset data attribute information; S1-2: Based on several historical multimodal monitoring data, a multimodal feature fusion model is constructed using a multimodal fusion deep learning algorithm, and several historical multimodal fusion features are generated; The multimodal feature fusion model is built based on the Convolutional Neural Networks (CNN)-Transformer-logfBank-Attention-Multilayer Perceptron (MLP) algorithm; The CNN network is used to extract image data features from multimodal monitoring data, the Transformer module is used to extract text data features from multimodal monitoring data, and the logfBank module is used to extract voice data features from multimodal monitoring data, which are generally spectrograms of voice data after short-time Fourier transform (STFT). The extracted spectral features use the Attention mechanism to set corresponding attention weights, enhance the image data features, text data features, and voice data features, and use the MLP network to fuse the enhanced image data features, text data features, and voice data features, and output multimodal fusion features. S1-3: Based on several historical multimodal fusion features, a deep learning algorithm is used to build an emotion recognition model and generate several historical emotion recognition results; The emotion recognition model is built based on the Bi-directional Long Short-Term Memory (BiLSTM) algorithm. The BiLSTM network can effectively capture the emotional information in different modal data and improve the accuracy of emotion recognition by learning the interaction between different modalities. S1-4: Based on the historical interactive voice data in several historical multimodal monitoring data, a deep learning and natural language processing fusion algorithm is used to build a voice recognition model and generate several historical AI interactive data; The speech recognition model is built based on the Long Short-Term Memory (LSTM)-Linear Chain Conditional Random Field (CRF) algorithm. The LSTM network can capture long-term dependencies and is particularly suitable for tasks that require contextual information, such as speech recognition. It extracts semantic features from the spectral features output by the logfBank module and performs post-processing based on the semantic features to optimize the prediction of sequence labels. CRF can capture the dependencies between labels, improve recognition accuracy, and realize recognition of text sequence annotation. S1-5: Based on several historical emotion recognition results, use the reinforcement learning algorithm to build an AI response strategy generation model, and generate several historical AI response strategies and corresponding historical AI response strategy generation experiences; The AI ​​response strategy generation model is built based on the Deep Q Network (DQN) algorithm, and the AI ​​response strategy generation model is equipped with an intelligent agent, an experience replay pool, and a deep Q network; Based on several historical emotion recognition results, a reinforcement learning algorithm is used to build an AI response strategy generation model, and several historical AI response strategies and corresponding historical AI response strategy generation experiences are generated, including the following steps: S1-5-1: Generate questions based on the AI ​​response strategy, set up the simulation environment for the DQN algorithm, and build the agent and experience replay pool; S1-5-2: Define the state space of the DQN algorithm based on the states involved in the historical emotion recognition results, and construct the input layer of the deep Q network based on the data structure of the historical emotion recognition results; S1-5-3: Define the action space of the DQN algorithm based on the actions involved in the AI ​​response strategy, and construct the output layer of the deep Q network based on the data structure of the AI ​​response strategy; S1-5-4: According to the input layer and the output layer, set several hidden layers, build the corresponding deep Q network, connect the input layer to the state space, and connect the output layer to the action space; S1-5-5: Define the reward function of the DQN algorithm based on the impact of each action in the action space; S1-5-6: Based on the state space, action space and reward function, use several historical emotion recognition results to optimize the training of the deep Q network and the agent, build an AI response strategy generation model, and generate several historical AI response strategies and corresponding historical AI response strategy generation experiences; AI response strategies include emotional resonance strategies (including empathy expression and emotional confirmation), personalized interaction strategies (including user preference considerations and personalized responses), crisis intervention strategies (including emergency responses and professional advice), emotional relief strategies (including entertainment relief, emotional regulation, and supportive responses), etc. Emotional resonance strategies, personalized interaction strategies, and crisis intervention strategies affect the AI ​​response text content generated by the AI ​​response model. Emotion relief strategies affect what type of emotion relief data is extracted from the emotion relief database. S1-5-7: Store some historical AI response strategy generation experiences into the experience replay pool of the AI ​​response strategy generation model; S1-6: Based on several historical AI response strategies, several historical AI interaction data, and several preset emotion relief data, use the large language model algorithm to build an AI response model and generate several historical AI response text data; The AI ​​response model is built on the Bidirectional Encoder Representations from Transformers (BERT) algorithm. It uses the bidirectional Transformer structure to pre-train the language model and has achieved significant performance improvements in various natural language processing tasks. In multi-round conversations, BERT can help the Q&A robot understand the context of the conversation and provide more coherent answers. Several historical AI response strategies, several historical AI interaction data, and several preset emotion relief data can be used to fine-tune the existing BERT model and generate a customized AI response model to make the model better suited to specific AI response text generation tasks. S1-7: Based on some historical AI response text data, use audio signal processing algorithms to build a speech reconstruction model; The speech reconstruction model is built based on the Generative Adversarial Network (GAN)-Tacotron2 algorithm; the GAN network can generate corresponding prosodic features according to the input AI response strategy, including pitch, stress, speaking speed and rhythm, etc. The Tacotron2 module converts the AI ​​response text data into corresponding speech data, and then adjusts the speech data according to the prosodic features to obtain the final AI response speech data; S2: Collecting real-time multimodal monitoring data of users, and using the emotion AI interaction engine to generate real-time emotion recognition results and real-time AI interaction data based on the real-time multimodal monitoring data, including the following steps: S2-1: collecting real-time multimodal monitoring data of the user, and preprocessing the real-time multimodal monitoring data to obtain preprocessed real-time multimodal monitoring data; Real-time multimodal monitoring data includes real-time expression image monitoring data, real-time interactive voice data, and real-time interactive text data of users; S2-2: Input the pre-processed real-time multimodal monitoring data into the emotion AI interaction engine, and use the multimodal feature fusion model to generate the corresponding real-time multimodal fusion features; S2-3: After preprocessing, the real-time interactive speech data in the real-time multimodal monitoring data is subjected to emotion recognition using an emotion recognition model to obtain corresponding real-time emotion recognition results; S2-4: Based on the pre-processed real-time multimodal monitoring data, use the speech recognition model to perform speech recognition and obtain the corresponding real-time AI interaction data; If the real-time multimodal monitoring data after preprocessing does not contain real-time interactive voice data, the real-time interactive text data will be used as the final real-time AI interactive data; In this embodiment, the real-time emotion recognition result is sadness, and the recognized real-time AI interaction data is "I feel very sad"; S3: Generate a real-time AI response strategy based on the real-time emotion recognition result using the emotion AI interaction engine, and extract real-time emotion relief data from the emotion relief database based on the real-time AI response strategy, including the following steps: S3-1: Based on the real-time emotion recognition results, the AI ​​response strategy generation model of the emotion AI interaction engine is used to generate an AI response strategy to obtain a real-time AI response strategy, including the following steps: S3-1-1: Update the state space of the AI ​​response strategy generation model based on the real-time emotion recognition results to obtain an updated state space ,in, For the updated Status value, is the state indicator, is the total number of state space dimensions; S3-1-2: Extract some historical AI response strategy generation experiences from the experience replay pool, and update the action space of the AI ​​response strategy generation model based on some historical AI response strategy generation experiences to obtain an updated action space ,in, For the updated Action value, is the action indicator, is the total number of action space dimensions; S3-1-3: Input the updated state space into the input layer of the deep Q network of the AI ​​response strategy generation model, and connect the updated action space to the output layer of the deep Q network to obtain an updated deep Q network; S3-1-4: Based on the updated deep Q network, use the agent to control the updated deep Q network and output the Q values ​​of possible actions in the updated action space; S3-1-5: Iteratively update the Q value according to the preset reward function to obtain an updated Q value until the number of iterations reaches the threshold; The formula is:

[0023] In the formula, The updated status value and the updated action value The corresponding updated Q value; Status value and action value The corresponding predicted Q value; is the learning rate; is the highest predicted Q value; It is a comprehensive indicator; To update the parameters; S3-1-6: According to the greedy strategy, the possible action with the highest updated Q value in each iteration is used as the execution action of the corresponding state; S3-1-7: Integrate the execution actions corresponding to all states in the updated state space to obtain the real-time AI response strategy; In this embodiment, the real-time AI response strategy includes empathetic expression, supportive response, etc.; S3-2: According to the real-time AI response strategy, a search and match is performed in a number of preset data attribute information in the emotion relief database to obtain a number of successfully matched real-time data attribute information; The data attribute information includes basic information such as the emotion type, data format, and data keywords of the emotion relief data; S3-3: Outputting a number of real-time data attribute information and corresponding preset emotion relief data to obtain a number of real-time emotion relief data; S4: Generate real-time AI response data using the emotional AI interaction engine based on the real-time AI response strategy, real-time AI interaction data, and real-time emotion relief data; Real-time AI response data includes real-time AI response text data, real-time AI response voice data, and real-time emotion relief data; According to the real-time AI response strategy, real-time AI interaction data and real-time emotion relief data, the emotional AI interaction engine is used to generate real-time AI response data, including the following steps: S4-1: Generate real-time AI response text data using the AI ​​response model of the emotional AI interaction engine based on the real-time AI response strategy, real-time AI interaction data, and real-time data attribute information; In this embodiment, the real-time AI response text data is "I understand how you feel now. Here is a soothing piece of music and a warm picture. I hope they can bring you comfort." S4-2: Based on the real-time AI response text data, use the speech reconstruction model to perform speech reconstruction to obtain real-time AI response speech data; S4-3: Integrate real-time AI response text data, real-time AI response voice data, and real-time emotion relief data to obtain real-time AI response data; S5: Collecting real-time feedback data from users, and optimizing the emotion AI interaction engine according to the real-time feedback data to obtain an optimized emotion AI interaction engine, including the following steps: S5-1: Collect and analyze real-time feedback data from users to obtain performance parameters of the emotional AI interaction engine, including the accuracy of the emotion recognition model, the response time of the AI ​​response strategy model, and the reconstruction naturalness of the speech reconstruction model; S5-2: According to the performance parameters, set the optimization objective function of the emotional AI interaction engine and set the solution vector encoding of the swarm intelligence optimization algorithm;

[0024] In the formula, To optimize the objective function; is the accuracy of the emotion recognition model; The response time of the AI ​​response strategy model; The reconstruction naturalness of the speech reconstruction model; is the weight factor adjusted according to the feedback data; Refers to the parameters for the solution; Refers to parameters for feedback data; is the weight value; S5-3: According to the optimization objective function and the solution vector encoding, the Improved Crested Porcupine Optimizer (ICPO) optimization algorithm is used to perform iterative optimization to obtain the optimized model parameters corresponding to the optimal solution, including the following steps: S5-3-1: According to the optimization objective function, set the fitness function of the ICPO optimization algorithm, and set the ICPO population parameters and the maximum number of iterations of the ICPO optimization algorithm; The formula is:

[0025] In the formula, is the fitness function; S5-3-2: Initialize the ICPO population using the Circle chaotic mapping sequence according to the solution vector code and ICPO population parameters to obtain an initial ICPO population containing a number of initial ICPO individuals; The formula is:

[0026] In the formula, is the initial ICPO individual of the Circle chaos map, corresponding to an initial solution; is the randomly generated initial ICPO individual; It is the ICPO individual indicator; S5-3-3: According to the fitness function, ICPO population parameters and the maximum number of iterations, the initial ICPO population is iteratively optimized to obtain the optimal ICPO individual, including the following steps: S5-3-3-1: Introduce a cyclic population reduction mechanism to limit the number of individuals in the ICPO population parameters and obtain updated ICPO population parameters for the next iteration; The formula is:

[0027] In the formula, For the The number of individuals in the ICPO population parameter of the iteration; For the The number of individuals in the ICPO population parameter of the iteration; is the minimum number of individuals in the ICPO population parameter; Evaluate arguments for functions; Evaluate loop parameters for a function; Evaluate loop parameters for the maximum function; t is the indicator of the number of iterations; S5-3-3-2: Calculate the initial fitness value of the initial ICPO individuals in the initial ICPO population according to the fitness function; S5-3-3-3: According to the initial fitness value and the updated ICPO population parameters, the first defense strategy, the second defense strategy, the third defense strategy and the fourth defense strategy are used to update the initial ICPO population to obtain an updated ICPO population; The formula for the first defense strategy is:

[0028] In the formula, For the updated ICPO individuals within the first defense range; The initial ICPO individual within the first defense range; is a random number based on normal distribution; is a random value in the interval [0,1]; It is the optimal solution within the first defense range; is the vector generated between the true optimal solution within the first defense range and the optimal solution randomly selected from the ICPO population; It is the ICPO individual indicator; is the iteration indicator; The formula for the second defense strategy is:

[0029] In the formula, For the updated ICPO individuals within the second defense range; The initial ICPO individual within the second defense range; is the search upper limit vector of the second defense range; is a random value in the interval [0,1]; Respectively Initial ICPO individuals; All are [1, ]; is the vector generated between the true optimal solution within the second defense range and the optimal solution randomly selected from the ICPO population; The formula for the third defense strategy is:

[0030] In the formula, For the updated ICPO individuals within the third defense range; The initial ICPO individual within the third defense range; is the search upper limit vector of the third defense range; Respectively Initial ICPO individuals; is [1, ]; The odor diffusion factor defined for the fitness function; It is a defense factor; Control parameters for search direction; The formula for the fourth defense strategy is:

[0031] In the formula, For the updated ICPO individuals within the fourth defense range; The initial ICPO individual within the fourth defense range; It is the optimal solution within the fourth defense range; All are random values ​​in the interval [0,1]; It is a defense factor; Control parameters for search direction; is the average force affecting the search direction; is the convergence speed factor; S5-3-3-4: Use the dynamic reverse learning algorithm to perform dynamic reverse learning on the updated ICPO population to generate a dynamic reverse ICPO population; The formula is:

[0032] In the formula, It is a dynamically reversed ICPO individual; is the decreasing inertia coefficient; are the maximum and minimum values ​​of the vector space respectively; For the updated ICPO individual; S5-3-3-5: According to the fitness function, calculate the fitness values ​​of all ICPO individuals in the updated ICPO population and the dynamically reversed ICPO population, take the ICPO individual with the minimum fitness value as the optimal ICPO individual, and retain the optimal ICPO individual; S5-3-3-6: If the number of iterations of iterative optimization reaches the maximum number of iterations or the fitness value of the optimal ICPO individual meets the requirements, the optimal solution corresponding to the optimal ICPO individual is output; S1-1-5: Decode the solution vector encoding corresponding to the optimal solution to obtain the optimal optimized model parameters of the emotional AI interaction engine; The optimal optimized model parameters of the emotional AI interaction engine include the optimal network parameters of the emotion recognition model, the optimal learning network parameters of the AI ​​response strategy model, and the optimal network parameters of the speech reconstruction model; S5-4: Optimize the emotion AI interaction engine according to the optimized model parameters to obtain an optimized emotion AI interaction engine.

[0033] Embodiment 2: like Figure 2 As shown, this embodiment provides an AI interaction system based on emotion recognition, which is used to implement an AI interaction method. The system includes a cloud data center and a plurality of user terminals, the plurality of user terminals are communicatively connected with the cloud data center, and the cloud data center is provided with an emotion AI interaction engine and an emotion relief database; The user terminal is used to collect the user's real-time multimodal monitoring data and upload the real-time multimodal monitoring data to the cloud data center; The cloud data center includes an initialization unit, an emotion recognition unit, an AI response strategy generation unit, an AI response generation unit, and an engine optimization unit; Initialization unit, used to build an emotion relief database and use multimodal fusion artificial intelligence algorithms to build an emotional AI interaction engine; The emotion recognition unit is used to generate real-time emotion recognition results and real-time AI interaction data based on real-time multimodal monitoring data using the emotion AI interaction engine; An AI response strategy generation unit, used to generate a real-time AI response strategy based on the real-time emotion recognition result and using the emotion AI interaction engine, and to extract real-time emotion relief data from the emotion relief database based on the real-time AI response strategy; An AI response generation unit, for generating real-time AI response data using an emotional AI interaction engine according to the real-time AI response strategy, the real-time AI interaction data, and the real-time emotion relief data; The engine optimization unit is used to collect real-time feedback data from users and optimize the emotion AI interaction engine based on the real-time feedback data to obtain an optimized emotion AI interaction engine.

[0034] The present invention discloses an AI interaction method and system based on emotion recognition, which performs emotion recognition according to multimodal fusion features, avoids recognition bias caused by interference information of single modal features in complex and changeable environments, and improves the accuracy of emotion recognition; an emotion AI interaction engine constructed based on an artificial intelligence algorithm realizes emotion recognition, speech recognition, AI response strategy generation and AI response generation, improves the degree of intelligence, can perform multi-mode and multi-modal customized AI interaction and dialogue according to user emotions, improves practicality, and outputs emotion relief data according to user emotions, realizing an accurate emotion soothing function; according to feedback data, the emotion AI interaction engine is optimized, the model automatic evolution function is realized, and it can be iteratively updated according to usage conditions to meet the needs of different users and improve user experience.

[0035] The present invention is not limited to the above optional implementations, and anyone can derive other various forms of products under the enlightenment of the present invention. The above specific implementations should not be understood as limiting the scope of protection of the present invention. The scope of protection of the present invention should be based on the definition in the claims, and the description can be used to interpret the claims.

Claims

1. An AI interaction method based on emotion recognition, characterized in that: The steps include: In the cloud data center, we build an emotion relief database and use multimodal fusion artificial intelligence algorithms to build an emotional AI interaction engine; Collect users' real-time multimodal monitoring data, and use the emotion AI interaction engine to generate real-time emotion recognition results and real-time AI interaction data based on the real-time multimodal monitoring data; Based on the real-time emotion recognition results, the emotion AI interaction engine is used to generate a real-time AI response strategy, and based on the real-time AI response strategy, real-time emotion relief data is extracted from the emotion relief database; Generate real-time AI response data using the emotional AI interaction engine based on real-time AI response strategies, real-time AI interaction data, and real-time emotion relief data; Collect real-time feedback data from users, and optimize the emotion AI interaction engine based on the real-time feedback data to obtain an optimized emotion AI interaction engine.

2. The AI ​​interaction method based on emotion recognition according to claim 1, characterized in that: The emotional AI interaction engine includes a multimodal feature fusion model, an emotion recognition model, a speech recognition model, an AI response strategy generation model, an AI response model, and a speech reconstruction model; The emotion relief database is provided with a plurality of preset emotion relief data of different emotion relief types, wherein the preset emotion relief data include preset emotion relief picture data, preset emotion relief audio data, preset emotion relief text data and preset emotion relief video data of different emotion relief types, and each preset emotion relief data is provided with corresponding preset data attribute information; The real-time multimodal monitoring data includes the user's real-time expression image monitoring data, real-time interactive voice data and real-time interactive text data; The real-time AI response data includes real-time AI response text data, real-time AI response voice data and real-time emotion relief data.

3. The AI ​​interaction method based on emotion recognition according to claim 2, characterized in that: In the cloud data center, we build an emotion relief database and use a multimodal fusion artificial intelligence algorithm to build an emotional AI interaction engine, including the following steps: In the cloud data center, a plurality of preset emotion relief data of different emotion relief types on the Internet are collected, and corresponding preset data attribute information is set to obtain an emotion relief database; Based on several historical multimodal monitoring data, a multimodal feature fusion model is constructed using a multimodal fusion deep learning algorithm, and several historical multimodal fusion features are generated; Based on several historical multimodal fusion features, a deep learning algorithm is used to build an emotion recognition model and generate several historical emotion recognition results; Based on some historical multimodal monitoring data, a speech recognition model is built using a deep learning and natural language processing fusion algorithm, and some historical AI interaction data is generated; Based on several historical emotion recognition results, a reinforcement learning algorithm is used to build an AI response strategy generation model, and several historical AI response strategies and corresponding historical AI response strategy generation experiences are generated; Based on several historical AI response strategies, several historical AI interaction data, and several preset emotion relief data, a large language model algorithm is used to build an AI response model and generate several historical AI response text data; Based on some historical AI response text data, a speech reconstruction model is constructed using audio signal processing algorithms; The multimodal feature fusion model, emotion recognition model, speech recognition model, AI response strategy generation model, AI response model and speech reconstruction model are integrated to obtain the emotion AI interaction engine.

4. The AI ​​interaction method based on emotion recognition according to claim 3, characterized in that: The multimodal feature fusion model is constructed based on the CNN-Transformer-logfBank-Attention-MLP algorithm; The emotion recognition model is built based on the BiLSTM algorithm; The speech recognition model is constructed based on the LSTM-CRF algorithm; The AI ​​response strategy generation model is constructed based on the DQN algorithm; The AI ​​response model described is built based on the BERT algorithm; The speech reconstruction model is built based on the GAN-Tacotron2 algorithm.

5. The AI ​​interaction method based on emotion recognition according to claim 4, characterized in that: Collecting real-time multimodal monitoring data of users, and using the emotion AI interaction engine to generate real-time emotion recognition results and real-time AI interaction data based on the real-time multimodal monitoring data, including the following steps: Collecting real-time multimodal monitoring data of users, and preprocessing the real-time multimodal monitoring data to obtain preprocessed real-time multimodal monitoring data; The pre-processed real-time multimodal monitoring data is input into the emotional AI interaction engine, and the corresponding real-time multimodal fusion features are generated using the multimodal feature fusion model; After preprocessing, the real-time interactive speech data in the real-time multimodal monitoring data is subjected to emotion recognition using an emotion recognition model to obtain corresponding real-time emotion recognition results; Based on the pre-processed real-time multimodal monitoring data, the speech recognition model is used to perform speech recognition to obtain the corresponding real-time AI interaction data.

6. The AI ​​interaction method based on emotion recognition according to claim 5, characterized in that: According to the real-time emotion recognition result, the emotion AI interaction engine is used to generate a real-time AI response strategy, and according to the real-time AI response strategy, the real-time emotion relief data is extracted from the emotion relief database, including the following steps: According to the real-time emotion recognition results, the AI ​​response strategy generation model of the emotion AI interaction engine is used to generate the AI ​​response strategy and obtain the real-time AI response strategy; According to the real-time AI response strategy, a search and match is performed in a number of preset data attribute information in the emotion relief database to obtain a number of successfully matched real-time data attribute information; A plurality of real-time data attribute information and corresponding preset emotion relief data are output to obtain a plurality of real-time emotion relief data.

7. The AI ​​interaction method based on emotion recognition according to claim 6, characterized in that: According to the real-time AI response strategy, real-time AI interaction data and real-time emotion relief data, the emotional AI interaction engine is used to generate real-time AI response data, including the following steps: Generate real-time AI response text data using the AI ​​response model of the emotional AI interaction engine based on the real-time AI response strategy, real-time AI interaction data, and real-time data attribute information; According to the real-time AI response text data, the speech reconstruction model is used to reconstruct the speech to obtain the real-time AI response speech data; Integrate real-time AI response text data, real-time AI response voice data, and real-time emotion relief data to obtain real-time AI response data.

8. The AI ​​interaction method based on emotion recognition according to claim 4, characterized in that: Collecting real-time feedback data from users, and optimizing the emotion AI interaction engine according to the real-time feedback data to obtain an optimized emotion AI interaction engine, including the following steps: Collect and analyze real-time feedback data from users to obtain performance parameters of the emotional AI interaction engine; According to the performance parameters, set the optimization objective function of the emotional AI interaction engine and set the solution vector encoding of the swarm intelligence optimization algorithm; According to the optimization objective function and solution vector encoding, the swarm intelligence optimization algorithm is used to iteratively search for the optimal solution to obtain the optimized model parameters corresponding to the optimal solution; According to the optimized model parameters, the emotional AI interaction engine is optimized to obtain an optimized emotional AI interaction engine.

9. The AI ​​interaction method based on emotion recognition according to claim 8, characterized in that: The swarm intelligence optimization algorithm is the ICPO optimization algorithm.

10. An AI interaction system based on emotion recognition, used to implement the AI ​​interaction method according to any one of claims 1 to 9, characterized in that: The system includes a cloud data center and several user terminals. The several user terminals are communicatively connected to the cloud data center. The cloud data center is provided with an emotional AI interaction engine and an emotional relief database.

Images