A human-computer interaction method and system for cognitive impairment based on emotion analysis

Abstract

The present invention discloses a human-computer interaction method and system for cognitive impairment based on emotion analysis, which relates to the field of human-computer interaction for cognitive impairment, including: real-time collection of physiological information, language information, and expression information of users for feature fusion to form a time series data set, and emotion recognition and classification; based on the user emotion recognition and classification results, a generative adversarial network is established to generate human-computer interaction dialogue content matching the current context and emotion in real time; when it is detected that the user's emotion recognition result is a negative emotion, a cognitive impairment emotion soothing strategy is retrieved based on big data, the user is emotionally soothed and relevant data is recorded; by analyzing the user's negative emotion history related data, an emotion prediction model is constructed to identify possible emotion changes in advance and adjust the human-computer interaction soothing strategy.

Description

Technical Field

[0001] The present invention relates to the field of cognitive impairment human-computer interaction, and in particular to a cognitive impairment human-computer interaction method and system based on emotion analysis. Background Art

[0002] With the aging society, the number of patients with cognitive impairment (such as Alzheimer's disease) is gradually increasing. They have difficulty understanding and expressing emotions, which affects their quality of life and social interaction. Traditional human-computer interaction methods often rely on static language and behavior, and lack response to emotional states. When patients with cognitive impairment are in a positive emotional state, introducing scene-appropriate human-computer interaction dialogue training content based on the emotional state and cognitive ability of patients with cognitive impairment can help patients improve their language expression ability and restore normal social skills. In addition, traditional human-computer interaction systems often lack sensitivity to patients' negative emotional fluctuations and cannot respond to users' emotional changes in a timely manner to comfort them. In this case, it is easy for patients to cause negative emotions to intensify, causing mental health damage to patients.

[0003] How to propose a human-computer interaction method and system for cognitive impairment based on emotion analysis, which can quickly identify the patient's emotional state, enable the patient to have a better human-computer interaction dialogue experience, and at the same time predict the patient's negative emotions and soothe the emotion in advance to protect the patient's mental health, is an urgent problem that needs to be solved. Summary of the invention

[0004] In order to solve the above technical problems, a method and system for human-computer interaction for people with cognitive impairment based on emotion analysis are provided. This technical solution solves the problems raised in the above background technology.

[0005] In order to achieve the above purpose, the technical solution adopted by the present invention is:

[0006] A cognitive impairment human-computer interaction method based on emotion analysis, comprising:

[0007] Collect users' physiological information, language information, and facial expression information in real time for feature fusion to form a time series data set for emotion recognition and classification;

[0008] Based on the user emotion recognition and classification results, a generative adversarial network is established to generate human-computer interaction dialogue content that matches the current context and emotions in real time;

[0009] When the user's emotion recognition result is detected as negative emotion, the cognitive impairment emotion soothing strategy is retrieved based on big data, the user's emotion is soothed and relevant data is recorded;

[0010] By analyzing the user's negative emotion history data, an emotion prediction model is constructed to identify possible emotion changes in advance and adjust the human-computer interaction soothing strategy.

[0011] Preferably, the real-time collection of the user's physiological information, language information, and expression information for feature fusion to form a time series data set, and the emotion recognition and classification specifically include:

[0012] Collect the user's physiological data within a set period of time, including heart rate, skin electrical response and breathing rate, to form a time series physiological emotion feature data set;

[0013] Use natural language processing technology to analyze the user's voice, extract the intonation, speaking speed and pitch acoustic feature information, generate corresponding voice features, and construct a language emotion feature dataset;

[0014] Capture the user's facial expressions, use computer vision technology to detect facial feature points, extract the facial feature information of the eyes, mouth and eyebrows, generate corresponding facial features, and construct a facial emotion feature dataset;

[0015] The ensemble learning method is used to fuse the language emotion feature dataset and the facial emotion feature dataset based on the physiological emotion feature dataset. By building a regression model, the comprehensive emotion score is obtained by integrating physiological, speech and facial features, and the score result is marked as ;

[0016] The comprehensive sentiment score within at least one set period is calculated through a multi-class classification algorithm. Map to specific emotion label categories, use random forest classification model for training, and build emotion recognition classification model to identify the user's emotional state. The output includes emotion category label and its confidence.

[0017] Emotional labels are manually classified into major categories, and emotional category labels are divided into negative emotions and non-negative emotions.

[0018] Preferably, the step of establishing a generative adversarial network to generate human-computer interaction dialogue content matching the current context and emotion in real time based on the user emotion recognition and classification results specifically includes:

[0019] Monitor and collect the user's physiological emotional features, language emotional features, and facial emotional features, fuse the features and input them into the emotion recognition and classification model to obtain the emotion classification results;

[0020] When the recognition result of the emotion recognition classification model is attributed to non-negative emotions, the start and end time of the human-computer interaction dialogue and the dialogue density are retrieved, and the dynamic dialogue context extraction window is comprehensively set to extract the context text of the dialogue data;

[0021] Perform data preprocessing on the extracted context text and split the context text into words and phrases using word segmentation technology;

[0022] Apply natural language processing technology to perform semantic analysis on the preprocessed context text, extract key entities, topics, and intentions, and vectorize the extracted context information to form a context vector ;

[0023] Based on the real-time response requirements of dialogue generation and the remaining system resources, the trained dialogue generation model and dialogue emotion recognition model are selected, and the models are optimized and trained using historical dialogue datasets. The dialogue generation model and dialogue emotion recognition model are marked as generator G and discriminator D respectively.

[0024] The role of the generator is to generate candidate responses based on the collected context vector and the emotion label category recognition results, and the role of the discriminator is to judge the emotional matching degree of the response. The generation and discrimination process is expressed as: in, is the candidate response, Score the emotional match, Emotion category recognition result for the current user;

[0025] The iterative process of adversarial training is introduced to establish a generative adversarial network based on the generator and the discriminator, and the generator parameters are dynamically adjusted through the emotion matching score feedback;

[0026] Generate real-time scene dialogue content based on the context vectors and emotion category recognition results collected in real time.

[0027] Preferably, when the emotion recognition result of the user is detected to be a negative emotion, retrieving the cognitive impairment emotion soothing strategy based on big data, soothing the user's emotion and recording relevant data specifically includes:

[0028] Obtain all emotion tags, use each emotion tag as a keyword to retrieve at least one corresponding human-computer interaction soothing strategy through big data, and establish a corresponding association relationship between emotion tags and soothing strategy sets;

[0029] When the recognition result of the emotion recognition classification model is attributed to negative emotion, the corresponding soothing strategy set is obtained through the specific emotion label of the recognition result, and the same size of emotion impact monitoring window is set for each soothing strategy to soothe the user's emotions;

[0030] Monitor the change of the user's emotion score at a set frequency, mark the corresponding soothing strategy with the largest change in emotion score as the optimal soothing strategy for the emotion, and establish a corresponding association between emotion label and optimal soothing strategy;

[0031] Collect and record real-time relevant data around users when they have negative emotions, including emotion scores, environmental data, noise data, light data, and conversation context of human-computer interaction, to establish an emotion prediction training dataset.

[0032] Preferably, the analysis of the user's negative emotion history data to construct an emotion prediction model, identify possible emotion changes in advance and adjust the human-computer interaction soothing strategy specifically includes:

[0033] Obtain the user's emotion prediction training data set and extract dimensional features from the data. The human-computer interaction dialogue context in the historical data needs to use natural language processing technology to extract key entities and perform vectorization conversion processing. The remaining data needs to remove units and then perform vectorization conversion processing.

[0034] The extracted features are aggregated and standardized, specifically: In the formula, They are environment data, light data, noise data, the emotion score of the last non-negative emotion detected, and the feature vector of the human-computer interaction dialogue context. is the aggregated feature vector, is the normalized feature vector, is the feature mean, is the characteristic standard deviation;

[0035] Select convolutional neural network as the emotion prediction model and use the time-series feature vector As feature input, the current sentiment score is used as feature output to train the sentiment prediction model, and the accuracy of the model is evaluated using the cross-validation method until the recognition accuracy of the prediction model reaches the preset standard;

[0036] Monitor the user's emotion score, environmental data, noise data, light data, and human-computer interaction conversation context data in real time, and predict the user's emotion according to the emotion prediction model at a set frequency;

[0037] Get the emotion label corresponding to the emotion score prediction result, take the emotion label confidence exceeding the set confidence threshold as the predicted emotion combination, and select the optimal soothing strategy combination for human-computer interaction based on the predicted emotion combination according to the corresponding association relationship between emotion label and optimal soothing strategy, soothe the user in advance.

[0038] Furthermore, a cognitive impairment human-computer interaction method system based on emotion analysis is proposed, which is used to implement the cognitive impairment human-computer interaction method based on emotion analysis as described above, including:

[0039] A multi-dimensional emotion recognition module, which performs feature construction and time series fusion on the user's physiological information data, voice information data, and facial expression information data, respectively, constructs a regression model, integrates physiological, voice and facial features to perform comprehensive emotion scoring, and establishes an emotion recognition classification model based on emotion category mapping annotations to identify the user's emotional state;

[0040] A dynamic dialogue generation module, which extracts context text from human-computer interaction dialogue data, selects and optimizes a dialogue generation model and a dialogue emotion recognition model, introduces an iterative process of adversarial training to establish a generative adversarial network, and generates real-time dialogue content for a scene based on the context vectors and emotion category recognition results collected in real time;

[0041] A human-computer interaction soothing strategy retrieval module, which retrieves cognitive impairment emotion soothing strategies based on big data, establishes a corresponding association relationship between emotion labels and optimal soothing strategies according to changes in the emotion scores of monitored users, and establishes an emotion prediction training data set;

[0042] The emotion change prediction module selects a convolutional neural network as an emotion prediction model according to an emotion prediction training data set to predict the user's emotions, and then selects the optimal human-computer interaction soothing strategy to soothe the user's emotions in advance according to the corresponding association relationship between emotion label and optimal soothing strategy.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] The user's physiological information, language information, and expression information are collected in real time for feature fusion to form a time series data set, so that the system can fully understand the user's emotional state and improve the accuracy of emotion recognition. Based on the user's emotion recognition classification results, a generative adversarial network is established to generate human-computer interaction dialogue content that matches the current context and emotions in real time, ensuring the adaptability and flexibility of the interaction and enhancing the user experience. When the user's emotion recognition result is detected as a negative emotion, the cognitive disorder emotion soothing strategy is retrieved based on big data, so that the system can provide personalized emotional support for each user. By analyzing the user's negative emotion history related data and building an emotion prediction model, it is possible to identify potential emotional fluctuations in advance and adjust the human-computer interaction strategy in time, thereby reducing the occurrence of emotional crises.

[0045] It has achieved a method that can quickly identify the patient's emotional state, meet the needs of intelligent human-computer interactive dialogue training, predict negative emotions and timely comfort emotions, help improve patients' language expression ability, reduce the occurrence of emotional crises in patients, and protect patients' mental health. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A flow chart of a human-computer interaction method for cognitive impairment based on emotion analysis of the present invention;

[0047] Figure 2 The present invention collects the user's physiological information, language information, and expression information in real time for feature fusion to form a time series data set and perform emotion recognition and classification flow chart;

[0048] Figure 3 A flowchart of establishing a generative adversarial network to generate human-computer interaction dialogue content matching the current context and emotions in real time based on the user emotion recognition and classification results of the present invention;

[0049] Figure 4 When the emotion recognition result of the user is detected as negative emotion, the emotional soothing strategy for cognitive impairment is retrieved based on big data, the user is emotionally soothed and relevant data is recorded;

[0050] Figure 5 The present invention constructs an emotion prediction model by analyzing the user's negative emotion history related data, identifies possible emotion changes in advance and adjusts the human-computer interaction soothing strategy flow chart;

[0051] Figure 6 This is a schematic diagram of the structure of a human-computer interaction system for cognitive impairment based on emotion analysis of the present invention. DETAILED DESCRIPTION

[0052] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art may think of other obvious variations.

[0053] Reference Figure 1 As shown, a cognitive impairment human-computer interaction method based on emotion analysis includes:

[0054] Collect users' physiological information, language information, and facial expression information in real time for feature fusion to form a time series data set for emotion recognition and classification;

[0055] Based on the user emotion recognition and classification results, a generative adversarial network is established to generate human-computer interaction dialogue content that matches the current context and emotions in real time;

[0056] When the user's emotion recognition result is detected as negative emotion, the cognitive impairment emotion soothing strategy is retrieved based on big data, the user's emotion is soothed and relevant data is recorded;

[0057] By analyzing the user's negative emotion history data, an emotion prediction model is constructed to identify possible emotion changes in advance and adjust the human-computer interaction soothing strategy.

[0058] Reference Figure 2 As shown, the user's physiological information, language information, and facial expression information are collected in real time for feature fusion to form a time series data set for emotion recognition and classification. By integrating physiological, language, and facial expression information, the system can fully understand the user's emotional state and improve the accuracy of emotion recognition.

[0059] Collect the user's physiological data within a set period of time, including heart rate, skin electrical response and breathing rate, to form a time series physiological emotion feature data set;

[0060] Use natural language processing technology to analyze the user's voice, extract the intonation, speaking speed and pitch acoustic feature information, generate corresponding voice features, and construct a language emotion feature dataset;

[0061] Capture the user's facial expressions, use computer vision technology to detect facial feature points, extract the facial feature information of the eyes, mouth and eyebrows, generate corresponding facial features, and construct a facial emotion feature dataset;

[0062] The ensemble learning method is used to fuse the language emotion feature dataset and the facial emotion feature dataset based on the physiological emotion feature dataset. By building a regression model, the comprehensive emotion score is obtained by integrating physiological, speech and facial features, and the score result is marked as ; Use a multi-class classification algorithm to classify the comprehensive sentiment score within at least one set period Map to specific emotion label categories, use random forest classification model for training, and build emotion recognition classification model to identify the user's emotional state. The output includes emotion category label and its confidence.

[0063] Emotional labels are manually classified into major categories, and emotional category labels are divided into negative emotions and non-negative emotions. Negative emotions are divided into manually pre-labeled keywords, such as anger, sadness and other emotional states as negative emotions, and all emotions that are not negative emotions are non-negative emotions.

[0064] Reference Figure 3 As shown, based on the user emotion recognition and classification results, a generative adversarial network is established to generate human-computer interaction dialogue content that matches the current context and emotions in real time, ensuring the naturalness of the interaction and the relevance of the context, improving the user communication experience, and improving the user's communication level.

[0065] Monitor and collect the user's physiological emotional features, language emotional features, and facial emotional features, fuse the features and input them into the emotion recognition and classification model to obtain the emotion classification results;

[0066] When the recognition result of the emotion recognition classification model is attributed to non-negative emotions, the start and end time of the human-computer interaction dialogue and the dialogue density are retrieved, and a dynamic dialogue context extraction window is comprehensively set to extract context text from the dialogue data. The setting of the context extraction window is based on the size of the dialogue paragraph whose dialogue density exceeds the set density value from the start to the end of this dialogue, which is used to filter out the human-computer accidental interaction dialogue data;

[0067] Perform data preprocessing on the extracted context text and split the context text into words and phrases using word segmentation technology;

[0068] Apply natural language processing technology to perform semantic analysis on the preprocessed context text, extract key entities, topics, and intentions, and vectorize the extracted context information to form a context vector ;

[0069] Based on the real-time response requirements of dialogue generation and the remaining system resources, the trained dialogue generation model and dialogue emotion recognition model are selected, and the models are optimized and trained using historical dialogue datasets. The dialogue generation model and dialogue emotion recognition model are marked as generator G and discriminator D, respectively. When system resources are insufficient or a quick response is required, the dialogue generation model and dialogue emotion recognition model can select existing models for migration, and then optimize and train them using historical dialogue datasets to save system resources.

[0070] The role of the generator is to generate candidate responses based on the collected context vector and the emotion label category recognition results, and the role of the discriminator is to judge the emotional matching degree of the response. The generation and discrimination process is expressed as: in, is the candidate response, Score the emotional match, Emotion category recognition result for the current user;

[0071] The iterative process of adversarial training is introduced. A generative adversarial network is established based on the generator and the discriminator. The generator parameters are dynamically adjusted through the emotion matching score feedback. The objective function of adversarial training is: In the formula, y is the real dialogue response, z is the random noise, and E is the corresponding calculated expected value. During the training process, the parameters of G and D are continuously optimized until the Nash equilibrium is reached.

[0072] During the training process, the verification data is used regularly to evaluate the performance of the generator and discriminator, and the performance is evaluated through indicators such as accuracy, recall, and F1 score. The model parameters are adjusted according to the evaluation results, and the structure of the generator and discriminator is optimized. The strategy can also be adjusted or replaced according to the user's real-time emotion recognition results. I will not go into details here.

[0073] Generate real-time scene dialogue content based on the context vectors and emotion category recognition results collected in real time.

[0074] Reference Figure 4 As shown, when it is detected that the user's emotion recognition result is a negative emotion, the cognitive impairment emotion soothing strategy is retrieved based on big data, the user is emotionally soothed and relevant data is recorded.

[0075] Obtain all emotion tags, use each emotion tag as a keyword to retrieve at least one corresponding human-computer interaction soothing strategy through big data, and establish a corresponding association relationship between emotion tags and soothing strategy sets;

[0076] When the recognition result of the emotion recognition classification model is attributed to negative emotion, the corresponding soothing strategy set is obtained through the specific emotion label of the recognition result, and the same size of emotion impact monitoring window is set for each soothing strategy to soothe the user's emotions;

[0077] Monitor the change of the user's emotion score at a set frequency, mark the corresponding soothing strategy with the largest change in emotion score as the optimal soothing strategy for the emotion, and establish a corresponding association between emotion label and optimal soothing strategy;

[0078] Collect and record real-time relevant data around users when they have negative emotions, including emotion scores, environmental data, noise data, light data, and conversation context of human-computer interaction, and establish an emotion prediction training data set for training the user's emotion prediction model.

[0079] Reference Figure 5 As shown in the figure, by analyzing the user's negative emotion history data, an emotion prediction model is constructed to identify possible emotion changes in advance and adjust the human-computer interaction soothing strategy, including:

[0080] Obtain the user's emotion prediction training data set and extract dimensional features from the data. The human-computer interaction dialogue context in the historical data needs to use natural language processing technology to extract key entities and perform vectorization conversion processing. The remaining data needs to remove units and then perform vectorization conversion processing.

[0081] The extracted features are aggregated and standardized, specifically: In the formula, They are environment data, light data, noise data, the emotion score of the last non-negative emotion detected, and the feature vector of the human-computer interaction dialogue context. is the aggregated feature vector, is the normalized feature vector, is the feature mean, is the characteristic standard deviation;

[0082] Select convolutional neural network as the emotion prediction model and use the time-series feature vector As feature input, the current sentiment score is used as feature output to train the sentiment prediction model, and the accuracy of the model is evaluated using the cross-validation method until the recognition accuracy of the prediction model reaches the preset standard;

[0083] Monitor the user's emotion score, environmental data, noise data, light data, and human-computer interaction conversation context data in real time, and predict the user's emotion according to the emotion prediction model at a set frequency;

[0084] Get the emotion label corresponding to the emotion score prediction result, take the emotion label whose confidence exceeds the set confidence threshold as the predicted emotion combination, select the optimal soothing strategy combination for human-computer interaction based on the predicted emotion combination according to the corresponding association relationship between emotion label and optimal soothing strategy, and provide advance emotional soothing to the user. The confidence threshold is set based on a comprehensive evaluation of the intelligence level of the human-computer interaction device and the level of cognitive impairment of the user, which will not be repeated here.

[0085] For further information, see Figure 6 As shown, based on the same inventive concept as the above-mentioned cognitive impairment human-computer interaction method based on emotion analysis, this solution also proposes a cognitive impairment human-computer interaction system based on emotion analysis, including:

[0086] A multi-dimensional emotion recognition module, which performs feature construction and time series fusion on the user's physiological information data, voice information data, and facial expression information data, respectively, constructs a regression model, integrates physiological, voice and facial features to perform comprehensive emotion scoring, and establishes an emotion recognition classification model based on emotion category mapping annotations to identify the user's emotional state;

[0087] A dynamic dialogue generation module, which extracts context text from human-computer interaction dialogue data, selects and optimizes a dialogue generation model and a dialogue emotion recognition model, introduces an iterative process of adversarial training to establish a generative adversarial network, and generates real-time dialogue content for a scene based on the context vectors and emotion category recognition results collected in real time;

[0088] A human-computer interaction soothing strategy retrieval module, which retrieves cognitive impairment emotion soothing strategies based on big data, establishes a corresponding association relationship between emotion labels and optimal soothing strategies according to changes in the emotion scores of monitored users, and establishes an emotion prediction training data set;

[0089] The emotion change prediction module selects a convolutional neural network as an emotion prediction model according to an emotion prediction training data set to predict the user's emotions, and then selects the optimal human-computer interaction soothing strategy to soothe the user's emotions in advance according to the corresponding association relationship between emotion label and optimal soothing strategy.

[0090] Furthermore, the present solution also proposes a method and system for human-computer interaction for cognitive impairment based on emotion analysis, on which a computer-readable program is stored. When the computer-readable program is called, the method and system for human-computer interaction for cognitive impairment based on emotion analysis are executed.

[0091] It is understandable that the storage medium may be a magnetic medium, such as a floppy disk, a hard disk, or a magnetic tape; an optical medium, such as a DVD; or a semiconductor medium, such as a solid state drive (SSD).

[0092] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions only describe the principles of the present invention. The present invention may be subject to various changes and improvements without departing from the spirit and scope of the present invention. These changes and improvements fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the attached claims and their equivalents.

Claims

1. A cognitive impairment human-computer interaction method based on emotion analysis, characterized in that: include: Collect users' physiological information, language information, and facial expression information in real time for feature fusion to form a time series data set for emotion recognition and classification; Based on the user emotion recognition and classification results, a generative adversarial network is established to generate human-computer interaction dialogue content that matches the current context and emotions in real time; When the user's emotion recognition result is detected as negative emotion, the cognitive impairment emotion soothing strategy is retrieved based on big data, the user's emotion is soothed and relevant data is recorded; By analyzing the user's negative emotion history data, an emotion prediction model is constructed to identify possible emotion changes in advance and adjust the human-computer interaction soothing strategy; When the emotion recognition result of the user is detected as negative emotion, the cognitive impairment emotion soothing strategy is retrieved based on big data, the user is emotionally soothed and relevant data is recorded, specifically including: Obtain all emotion tags, use each emotion tag as a keyword to retrieve at least one corresponding human-computer interaction soothing strategy through big data, and establish a corresponding association relationship between emotion tags and soothing strategy sets; When the recognition result of the emotion recognition classification model is attributed to negative emotion, the corresponding soothing strategy set is obtained through the specific emotion label of the recognition result, and the same size of emotion impact monitoring window is set for each soothing strategy to soothe the user's emotions; Monitor the change of the user's emotion score at a set frequency, mark the corresponding soothing strategy with the largest change in emotion score as the optimal soothing strategy for the emotion, and establish a corresponding association between emotion label and optimal soothing strategy; Collect and record the real-time relevant data around the user when he / she has negative emotions, including emotion scores, environmental data, noise data, light data, and the conversation context of human-computer interaction, to establish an emotion prediction training data set; The analysis of the user's negative emotion history data to build an emotion prediction model, identify possible emotion changes in advance, and adjust the human-computer interaction soothing strategy specifically includes: Obtain the user's emotion prediction training data set and extract dimensional features from the data. The human-computer interaction dialogue context in the historical data needs to use natural language processing technology to extract key entities and perform vectorization conversion processing. The remaining data needs to remove units and then perform vectorization conversion processing. The extracted features are aggregated and standardized, specifically: In the formula, They are environment data, light data, noise data, the emotion score of the last non-negative emotion detected, and the feature vector of the human-computer interaction dialogue context. is the aggregated feature vector, is the normalized feature vector, is the feature mean, is the characteristic standard deviation; Select convolutional neural network as the emotion prediction model and use the time-series feature vector As feature input, the current sentiment score is used as feature output to train the sentiment prediction model, and the accuracy of the model is evaluated using the cross-validation method until the recognition accuracy of the prediction model reaches the preset standard; Monitor the user's emotion score, environmental data, noise data, light data, and human-computer interaction conversation context data in real time, and predict the user's emotion according to the emotion prediction model at a set frequency; Get the emotion label corresponding to the emotion score prediction result, take the emotion label confidence exceeding the set confidence threshold as the predicted emotion combination, and select the optimal soothing strategy combination for human-computer interaction based on the predicted emotion combination according to the corresponding association relationship between emotion label and optimal soothing strategy, soothe the user in advance.

2. According to claim 1, a cognitive impairment human-computer interaction method based on emotion analysis is characterized in that: The real-time collection of the user's physiological information, language information, and expression information for feature fusion to form a time series data set and the emotion recognition and classification specifically include: Collect the user's physiological data within a set period of time, including heart rate, skin electrical response and breathing rate, to form a time series physiological emotion feature data set; Use natural language processing technology to analyze the user's voice, extract the intonation, speaking speed and pitch acoustic feature information, generate corresponding voice features, and construct a language emotion feature dataset; Capture the user's facial expressions, use computer vision technology to detect facial feature points, extract the facial feature information of the eyes, mouth and eyebrows, generate corresponding facial features, and construct a facial emotion feature dataset; The ensemble learning method is used to fuse the language emotion feature dataset and the facial emotion feature dataset based on the physiological emotion feature dataset. By building a regression model, the comprehensive emotion score is obtained by integrating physiological, speech and facial features, and the score result is marked as ; The comprehensive sentiment score within at least one set period is calculated through a multi-class classification algorithm. Map to specific emotion label categories, use random forest classification model for training, and build emotion recognition classification model to identify the user's emotional state. The output includes emotion category label and its confidence. Emotional labels are manually classified into major categories, and emotional category labels are divided into negative emotions and non-negative emotions.

3. The method for human-computer interaction for cognitive impairment based on emotion analysis according to claim 2, characterized in that: The method of establishing a generative adversarial network to generate human-computer interaction dialogue content matching the current context and emotions in real time based on the user emotion recognition and classification results specifically includes: Monitor and collect the user's physiological emotional features, language emotional features, and facial emotional features, fuse the features and input them into the emotion recognition and classification model to obtain the emotion classification results; When the recognition result of the emotion recognition classification model is attributed to non-negative emotions, the start and end time of the human-computer interaction dialogue and the dialogue density are retrieved, and the dynamic dialogue context extraction window is comprehensively set to extract the context text of the dialogue data; Perform data preprocessing on the extracted context text and split the context text into words and phrases using word segmentation technology; Apply natural language processing technology to perform semantic analysis on the preprocessed context text, extract key entities, topics, and intentions, and vectorize the extracted context information to form a context vector ; Based on the real-time response requirements of dialogue generation and the remaining system resources, the trained dialogue generation model and dialogue emotion recognition model are selected, and the models are optimized and trained using historical dialogue datasets. The dialogue generation model and dialogue emotion recognition model are marked as generator G and discriminator D respectively. The role of the generator is to generate candidate responses based on the collected context vector and the emotion label category recognition results, and the role of the discriminator is to judge the emotional matching degree of the response. The generation and discrimination process is expressed as: in, is the candidate response, Score the emotional match, Emotion category recognition result for the current user; The iterative process of adversarial training is introduced to establish a generative adversarial network based on the generator and the discriminator, and the generator parameters are dynamically adjusted through the emotion matching score feedback; Generate real-time scene dialogue content based on the context vectors and emotion category recognition results collected in real time.

4. A cognitive impairment human-computer interaction system based on emotion analysis, used to implement a cognitive impairment human-computer interaction method based on emotion analysis as claimed in any one of claims 1 to 3, characterized in that: include: A multi-dimensional emotion recognition module, which performs feature construction and time series fusion on the user's physiological information data, voice information data, and facial expression information data, respectively, constructs a regression model, integrates physiological, voice and facial features to perform comprehensive emotion scoring, and establishes an emotion recognition classification model based on emotion category mapping annotations to identify the user's emotional state; A dynamic dialogue generation module, which extracts context text from human-computer interaction dialogue data, selects and optimizes a dialogue generation model and a dialogue emotion recognition model, introduces an iterative process of adversarial training to establish a generative adversarial network, and generates real-time dialogue content for a scene based on the context vectors and emotion category recognition results collected in real time; A human-computer interaction soothing strategy retrieval module, which retrieves cognitive impairment emotion soothing strategies based on big data, establishes a corresponding association relationship between emotion labels and optimal soothing strategies according to changes in the emotion scores of monitored users, and establishes an emotion prediction training data set; The emotion change prediction module selects a convolutional neural network as an emotion prediction model based on an emotion prediction training data set to predict the user's emotions, and then selects the optimal human-computer interaction soothing strategy to soothe the user's emotions in advance based on the corresponding association relationship between emotion labels and optimal soothing strategies.

Images