Recognition equipment and recognition method for emotional communication mode

Abstract

The invention aims to provide recognition equipment and recognition method for an emotional communication mode, and the equipment comprises a camera, a microphone, a feature extraction unit, a first emotion recognition unit, a second emotion recognition unit and an emotional communication mode recognition unit, and the camera collects images of family members and outputs video signals; the microphone collects the sound of the family members and outputs an audio signal; the feature extraction unit receives the video signal and the audio signal, and detects and identifies registered family members; the first emotion recognition unit recognizes a first emotion track of the first family member according to the first feature vector; a second emotion recognition unit recognizes a second emotion track of the second family member according to the second feature vector; the emotional communication mode recognition unit recognizes emotional communication modes of the two family members according to the first emotional track and the second emotional track. The method is applied to the technical field of man-machine interaction.

Description

Technical Field

[0001] The present invention is applied to the technical field of human-computer interaction, and in particular relates to an emotion communication mode recognition device and a recognition method. Background Art

[0002] Emotion recognition can be used to detect human emotions and intentions through computer analysis. It can be widely used in the fields of medicine, automobiles, etc. For example, it can be used to identify and detect emotional disorders in the medical field, and it can help understand the emotional state of drivers in the field of smart cars. Retired elderly people mainly spend their time at home, but many elderly couples face the problem of poor emotional communication. This dilemma may lead to many hidden dangers and consequences, adversely affecting the relationship between husband and wife, personal happiness and the entire family, and even triggering acute attacks of chronic diseases such as hypertension, heart disease, and stroke. Compared with the current individual emotion recognition, emotional communication pattern recognition can timely detect the emotional communication barriers of elderly couples by identifying the emotional patterns of two people, and provide appropriate help to help them get out of trouble in time and rebuild and maintain a harmonious relationship between husband and wife.

[0003] Emotion recognition refers to the extraction of features and classification of emotions using traditional methods or deep learning methods. Emotion recognition can be obtained from various modalities, such as facial expressions, audio, text, EEG signals, etc. Existing emotion recognition methods are mainly divided into traditional methods and deep learning methods. Traditional methods rely on the knowledge and experience of domain experts, manually extract facial features, and then use machine learning algorithms such as support vector machines or random forests to complete emotion classification. They have strong interpretability and may perform well even in small sample situations, but they cannot adaptively learn and adapt to new data, and cannot handle emotion classification tasks in complex scenarios.

[0004] With the development of deep learning, emotion recognition methods based on deep learning have made new progress. Yoon et al. proposed a convolutional neural network method based on convolutional layers and pooling layers, which can be used to extract emotion-related features from text and classify them. The post-cyclic convolutional neural network has also been applied to the emotion classification tasks of images and texts. As some models pre-trained in large-scale data sets are used in various research fields, such as VGGNet, ResNet, Bert, FaceNet, etc., they have also been applied to emotional computing and achieved good results. Ding et al. proposed an architecture that integrates deep facial recognition framework with emotion recognition. They extended FaceNet's FaceNet2ExpNet and were adopted by Google's artificial intelligence (AI) team. Li et al. improved the architecture through transfer learning. In addition, Yagyaraj et al. demonstrated an emotional computing system that uses music, video and facial expressions to recognize emotions. In order to reduce the computational cost of neural networks, they decomposed the standard 2D / 3D convolutional layers into independent channels and spatiotemporal interactions. Amir et al. proposed a learnable graph inception network that can simultaneously perceive emotions and recognize potential graph structures in dynamic data, using three modalities: facial expressions, sounds and body movements.

[0005] In addition, models based on the self-attention mechanism are also widely used in emotion recognition tasks. The self-attention mechanism-based model transfomer proposed by Vaswani et al. uses the self-attention mechanism to capture long-range dependencies in the input sequence, can process the input sequence in parallel, and is widely used in sentiment computing in text modality. In order to apply the transfomer to the image field, Alexey Dosovitskiy et al. proposed an image classification model (Vision Transformer) based on the self-attention mechanism, which was then applied to the facial feature modality, which can capture rich feature information from the context and provide a powerful framework for emotion classification.

[0006] In the prior art, there are the following three methods: 1. Single-modal emotion recognition Technical solutions for facial expression emotion recognition: FACS: A Facial Expression Recognition System Based on Facial Muscle Movements.

[0007] EmotionNet: A method that uses a convolutional neural network (CNN) architecture to predict different emotion categories by learning features from facial images.

[0008] Vit: A variant approach to applying the Transformer model to image classification tasks.

[0009] Technical solutions for audio emotion recognition: HMM: A classic probabilistic graphical model that can model audio signals as HMMs and perform emotion recognition by learning the transition probabilities between observation sequences and emotional states.

[0010] RBM: An unsupervised approach to audio feature learning for emotion recognition.

[0011] Technical solutions for text emotion recognition: EmoBert: A BERT-based approach to emotion recognition.

[0012] EmotionFlows: An emotion recognition method based on attention mechanism recurrent neural network architecture.

[0013] InstructERC: An approach for the conversational emotion recognition task.

[0014] SACL-LSTM: A method for emotion recognition combining the properties of self-attention mechanism and long short-term memory network.

[0015] 2. Multimodal emotion recognition MMGCN: A graph convolutional network approach for multimodal data processing.

[0016] ICON: A multimodal emotion recognition approach based on cyclic feature set fusion.

[0017] (III) A family atmosphere adjustment device and its working control method When it is determined to be the voice of a family member, the volume, speaking speed, and whether there is a swear word are used to determine whether it is a quarrel. When it is determined to be a quarrel, the microcomputer controller can call the quarrel prevention terminal to dissuade the quarrel.

[0018] Although the current emotion recognition methods based on deep learning have made some progress, there are still many problems. For example, the application scenarios are limited: it is difficult to identify individual emotions and classify emotional exchanges in multi-person scenarios. Most of the existing methods in the field of emotion recognition are aimed at individual emotion recognition, and do not consider the emotional state between people in scenarios with two or more people. Users hope to build a method that can recognize the emotional state between two people through deep learning methods and data processing, such as: arguing, cold war, bickering, quarreling, etc. For some groups, such as retired couples or children without adults around, when it is found that the emotions between the two have been in a low or unhealthy state, corresponding solutions can be given in a timely manner.

[0019] Therefore, it is necessary to provide an emotion communication pattern recognition device and recognition method, which has the advantages of higher accuracy in emotion recognition, stronger robustness in emotion trajectory, more stable prediction of emotion communication pattern, realization of emotion communication pattern recognition task, and ability to send out help information in time. Summary of the invention

[0020] The technical problem to be solved by the present invention is to overcome the shortcomings of the prior art and provide an emotional communication pattern recognition device and recognition method, which has the advantages of higher accuracy in emotion recognition, stronger robustness of emotional trajectories, more stable prediction of emotional communication patterns, realization of emotional communication pattern recognition tasks, and the ability to send out help information in a timely manner.

[0021] The technical solution adopted by the present invention is: the present invention includes a camera, a microphone, a feature extraction unit, a first emotion recognition unit, a second emotion recognition unit, and an emotion communication pattern recognition unit. The camera collects images of family members and outputs video signals; The microphone collects the voices of the family members and outputs an audio signal; The feature extraction unit receives the video signal and the audio signal, detects and identifies registered family members, extracts features of the family members, and outputs a first feature vector of a first family member and a second feature vector of a second family member; The first emotion recognition unit recognizes a first emotion trajectory of a first family member according to the first feature vector; The second emotion recognition unit recognizes a second emotion trajectory of a second family member according to the second feature vector; The emotion communication pattern recognition unit recognizes the emotion communication pattern of the two family members according to the first emotion trajectory and the second emotion trajectory.

[0022] It can be seen from the above scheme that compared with the prior art, the present application combines the emotion recognition classification results of the video modality and the audio modality, so that the accuracy of emotion recognition is higher; and the emotion trajectory is obtained through the concept of sliding window, so that the emotion trajectory has stronger robustness and the prediction of the final emotional communication pattern is more stable; finally, the recognition task of the emotional communication pattern realized can send out help information in time, which is conducive to rebuilding and maintaining harmonious relationships among family members, especially husband-wife relationships and elderly husband-wife relationships.

[0023] A preferred solution is that the emotion communication pattern recognition device further includes a help-seeking decision unit; the help-seeking decision unit tracks the emotion communication pattern of the two family members, determines whether help-seeking is needed and the method of help-seeking, and issues a help-seeking message.

[0024] A preferred embodiment is that the test method is as follows: Step S1, the camera collects images of family members and outputs video signals; the microphone collects voices of family members and outputs voice signals; Step S2, the feature extraction unit receives the video signal and the audio signal, detects and identifies the registered family members, extracts the video features and the audio features of the family members, outputs the first feature vector of the first family member and the second feature vector of the second family member, and repeats the above process every 5 seconds; Step S3, the first emotion recognition unit receives the first feature vector, performs emotion recognition through a video emotion classification network and a speech emotion classification network, and obtains a first emotion trajectory; Step S4, the second emotion recognition unit receives the second feature vector, performs emotion recognition through the video emotion classification network and the speech emotion classification network, and obtains a second emotion trajectory; Step S5: The emotion communication pattern recognition unit receives the first emotion trajectory and the second emotion trajectory and splices the two trajectories, performs emotion communication pattern recognition through an emotion recognition classification network, and obtains the emotion communication pattern of the two family members.

[0025] A preferred solution is that step S2 includes the following subdivision steps: Step S2-1, face detection and identification of registered family members, first use dlib to perform face detection, obtain the detection frame, and then use the existing face recognition network (FaceNet, DeepFace and ArcFace) to implement face recognition and complete the identification of registered family members for subsequent video signal and audio signal processing; Step S2-2, first process the video signal and the audio signal, take the current moment as the end point, take one minute of video and audio data forward, for a one-minute video, extract 32 frames at equal intervals and obtain a frame sequence, and decode the image corresponding to the frame sequence through the existing audio and video processing library av, and then use the image processor of the pre-trained model of vivit to process to obtain a feature vector V, at this time the size of the feature vector is (3, 224, 224); for a one-minute audio signal, the feature vector A can be obtained by preprocessing through the existing technology; Step S2-3: concatenate the feature vector A and the feature vector V as the first feature vector, and perform the above operation every 5 seconds.

[0026] A preferred solution is that step S3 includes the following specific contents: The emotion recognition unit classifies emotions into seven categories, namely: neutral, happy, sad, angry, afraid, terrified, and surprised; after obtaining the first feature vector, the first emotion recognition unit sends the feature vector V to the video emotion recognition model to obtain the emotion recognition result v, wherein the video emotion recognition model is a fine-tuning of the pre-trained model of Vivit, changing the classifier of the last layer of the model; Specifically, the output layer of the classifier is changed from the original 400 categories to 7 categories suitable for the user's emotion recognition task, the backbone network is frozen during the model training process, and the parameters of the last layer are trained; at the same time, the feature vector A is sent to the audio emotion recognition model, and the CNN_LSTM network that combines the convolutional neural network (CNN) and the long short-term memory network (LSTM) is used to realize emotion recognition; The classification results are fused based on decisions to obtain the final classification results. Since the sliding window in step S2 performs the above process every 5 seconds, the personal emotion recognition results are obtained every 5 seconds, and these results are stored over time to form the first emotion trajectory.

[0027] A preferred solution is that the specific content of step S4 refers to the specific content of step S3, thereby forming the second emotion trajectory.

[0028] A preferred solution is that step S5 includes the following specific contents: Take the emotion track ten minutes before the current time, that is, the length of the first emotion track and the second emotion track are both 120, and then splice the first emotion track and the second emotion track together, so that the information of the emotion communication is interactive, forming the input of the emotion communication pattern recognition unit; In this unit, the long short-term memory network (LSTM) model is used to classify the emotional communication patterns within ten minutes, and finally the emotional communication patterns between two people are divided into four categories, namely harmony, anger, quarrel, and cold war.

[0029] A preferred solution is to establish a help-seeking decision-making unit based on the judgment of emotional communication patterns; When it is detected that two people have been in a state of disharmony for 12 hours, they will ask the server for help in a timely manner and receive suggestions on how to regulate their emotions. When it is detected that the two people are in a state of disharmony for a long time, the guardian will be asked for help and mediation will be intervened in a timely manner; When it is detected that two people are in a state of anger or cold war for a period of time, some activities to liven up the atmosphere will be pushed in a timely manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a schematic flow chart of the method of the present invention; Figure 2 is a schematic diagram of the process of the first emotion recognition unit; Figure 3 It is the experimental result table of the present invention. DETAILED DESCRIPTION

[0031] like Figure 1 and Figure 2 As shown, in this embodiment, the present invention includes a camera, a microphone, a feature extraction unit, a first emotion recognition unit, a second emotion recognition unit, and an emotion communication pattern recognition unit; The camera collects images of family members and outputs video signals; The microphone collects the voices of the family members and outputs an audio signal; The feature extraction unit receives the video signal and the audio signal, detects and identifies registered family members, extracts features of the family members, and outputs a first feature vector of a first family member and a second feature vector of a second family member; The first emotion recognition unit recognizes a first emotion trajectory of a first family member according to the first feature vector; The second emotion recognition unit recognizes a second emotion trajectory of a second family member according to the second feature vector; The emotion communication pattern recognition unit recognizes the emotion communication pattern of the two family members according to the first emotion trajectory and the second emotion trajectory.

[0032] like Figure 1 and Figure 2 As shown, in this embodiment, the emotion communication pattern recognition device also includes a help-seeking decision unit; the help-seeking decision unit tracks the emotion communication pattern of the two family members, determines whether help is needed and the method of help, and sends out a help message.

[0033] like Figure 1 and Figure 2 As shown, in this embodiment, the testing method is as follows: Step S1, the camera collects images of family members and outputs video signals; the microphone collects voices of family members and outputs voice signals; Step S2, the feature extraction unit receives the video signal and the audio signal, detects and identifies the registered family members, extracts the video features and the audio features of the family members, outputs the first feature vector of the first family member and the second feature vector of the second family member, and repeats the above process every 5 seconds; Step S3, the first emotion recognition unit receives the first feature vector, performs emotion recognition through a video emotion classification network and a speech emotion classification network, and obtains a first emotion trajectory; Step S4, the second emotion recognition unit receives the second feature vector, performs emotion recognition through the video emotion classification network and the speech emotion classification network, and obtains a second emotion trajectory; Step S5: The emotion communication pattern recognition unit receives the first emotion trajectory and the second emotion trajectory and splices the two trajectories, performs emotion communication pattern recognition through an emotion recognition classification network, and obtains the emotion communication pattern of the two family members.

[0034] like Figure 1 and Figure 2 As shown, in this embodiment, step S2 includes the following subdivision steps: Step S2-1, face detection and identification of registered family members, first use dlib (data analysis application toolkit) to perform face detection and obtain the detection frame, then use the existing face recognition network (FaceNet, DeepFace and ArcFace) to realize face recognition and complete the identification of registered family members for subsequent video signal and audio signal processing; Step S2-2, first process the video signal and the audio signal, take the current moment as the end point, take one minute of video and audio data forward, for a one-minute video, extract 32 frames at equal intervals and obtain a frame sequence, and decode the image corresponding to the frame sequence through the existing audio and video processing library av, and then use the image processor of the pre-trained model of vivit to process to obtain the feature vector V. At this time, the size of the feature vector is (3, 224, 224), and the size unit is a cubic pixel block, that is, there are three layers, each layer is a 224×224 pixel block; for a one-minute audio signal, the feature vector A can be obtained by the existing technology after preprocessing; Step S2-3: concatenate the feature vector A and the feature vector V as the first feature vector, and perform the above operation every 5 seconds.

[0035] like Figure 1 and Figure 2 As shown, in this embodiment, step S3 includes the following specific contents: The emotion recognition unit classifies emotions into seven categories, namely: neutral, happy, sad, angry, afraid, terrified, and surprised; after obtaining the first feature vector, the first emotion recognition unit sends the feature vector V to the video emotion recognition model to obtain the emotion recognition result v, wherein the video emotion recognition model is a pre-trained model of vivit (video visual transformer network, a deep learning network for video feature extraction) that is fine-tuned to change the classifier of the last layer of the model; Specifically, the output layer of the classifier is changed from the original 400 categories to 7 categories suitable for the user's emotion recognition task, the backbone network is frozen during the model training process, and the parameters of the last layer are trained; at the same time, the feature vector A is sent to the audio emotion recognition model, and the CNN_LSTM network that combines the convolutional neural network (CNN) and the long short-term memory network (LSTM) is used to realize emotion recognition; The classification results are fused based on decisions to obtain the final classification results. Since the sliding window in step S2 performs the above process every 5 seconds, the personal emotion recognition results are obtained every 5 seconds, and these results are stored over time to form the first emotion trajectory.

[0036] like Figure 1 and Figure 2 As shown, in this embodiment, the specific content of step S4 refers to the specific content of step S3, thereby forming the second emotion trajectory.

[0037] like Figure 1 and Figure 2 As shown, in this embodiment, step S5 includes the following specific contents: Take the emotion track ten minutes before the current time, that is, the length of the first emotion track and the second emotion track are both 120, the length unit is frame, and each sampling is regarded as a frame; then splice the first emotion track and the second emotion track together, so that the information of emotion communication is interactive, forming the input of the emotion communication pattern recognition unit; In this unit, the long short-term memory network (LSTM) model is used to classify the emotional communication patterns within ten minutes, and finally the emotional communication patterns between two people are divided into four categories, namely harmony, anger, quarrel, and cold war.

[0038] like Figure 1 and Figure 2 As shown, in this embodiment, based on the judgment of the emotional communication mode, a help-seeking decision unit is established; When it is detected that two people have been in a state of disharmony for 12 hours, they will ask the server for help in a timely manner and receive suggestions on how to regulate their emotions. When it is detected that the two people are in a state of disharmony for a long time, the guardian will be asked for help and mediation will be intervened in a timely manner; When it is detected that two people are in a state of anger or cold war for a period of time, some activities to liven up the atmosphere will be pushed in a timely manner.

[0039] In this embodiment, the experiments and results involved in this application are: 1. Face Detection and Face Recognition (1) Dataset The UCEC-FACE (Chinese Face Dataset for Face Recognition in Uncontrolled Classroom Environments) dataset is used. The dataset consists of several pictures, each of which contains a person's face.

[0040] (2) Experimental process l*Dataset preparation l*Convert the image to grayscale l* Feature extraction: First, face detection is performed using dlib (data analysis application toolkit) and mediapipe (media pipeline toolkit). The face detector of dlib (data analysis application toolkit) uses a method based on the histogram of directional gradients (HOG) features and a cascade classifier; the MediaPipe face detector uses a deep learning-based method to extract face features through a convolutional neural network (CNN). Then, for images with detected face borders, features are extracted using FaceNet (facial network) and VGG-Net (visual geometry group network).

[0041] l*Classification network: fully connected layer l* training: Perform kfold-10 (10 folds) cross-training in 10,000 batches for each type of feature, and stop training if the loss does not decrease after 10 trainings.

[0042] l*Evaluation indicators: accuracy and F1-Score. F1-Score refers to an indicator for measuring the performance of a classification model, which is particularly suitable for processing extremely unbalanced data sets.

[0043] (3) Experimental results (e.g. Figure 3 shown) (4) Results analysis It can be seen from the above data that good experimental results can be obtained on the UCEC-FACE dataset (a Chinese face dataset for face recognition in an uncontrolled classroom environment) by extracting features through the face detection method and face recognition method and classifying through a fully connected layer network, which provides technical support for this technical solution.

[0044] 2. Image Emotion Recognition (1) Dataset The Facial Emotion Expressions dataset is a dataset used for facial emotion expression research, which is used to analyze and identify emotional information in facial expressions. This dataset collects images containing facial expressions and related emotional labels. The emotions are divided into seven categories, namely anger, fear, fear, happiness, neutral, sadness, and surprise.

[0045] (2) Experimental process l*Dataset preparation l*Data processing: Randomly sample the dataset to increase the number of minority class samples to solve the problem of class imbalance, then process the labels and divide the dataset into training and test sets in a ratio of 6:4. Use the image processor of the vit (Visual Transformer Network, a deep learning network for image feature extraction) pre-trained model to process the images.

[0046] l*Training: Fine-tune the pre-trained model of VIT, modify its last classification layer to 7 categories, select only the backbone network of the pre-trained model of VIT, freeze the parameters of the backbone network, and then start training.

[0047] l*Evaluation index: accuracy (3) Experimental results Accuracy: 89.89% F1-Score:88.40% (4) Results analysis The accuracy on the test set reached 89.89% and the F1-Score reached 88.40%, which obviously has a very good effect and provides technical support for some functions of this technical solution.

[0048] Although the embodiments of the present invention are described with practical solutions, they do not constitute limitations on the meaning of the present invention. For those skilled in the art, it is obvious to modify the implementation scheme and combine it with other solutions based on this description.

Claims

1. A device for identifying emotional communication patterns, characterized in that: It includes a camera, a microphone, a feature extraction unit, a first emotion recognition unit, a second emotion recognition unit, and an emotion communication pattern recognition unit; The camera collects images of family members and outputs video signals; The microphone collects the voices of the family members and outputs an audio signal; The feature extraction unit receives the video signal and the audio signal, detects and identifies registered family members, extracts features of the family members, and outputs a first feature vector of a first family member and a second feature vector of a second family member; The first emotion recognition unit recognizes a first emotion trajectory of a first family member according to the first feature vector; The second emotion recognition unit recognizes a second emotion trajectory of a second family member according to the second feature vector; The emotion communication pattern recognition unit recognizes the emotion communication pattern of the two family members according to the first emotion trajectory and the second emotion trajectory.

2. The device for identifying an emotional communication pattern according to claim 1, characterized in that: The device for identifying the emotional communication pattern also includes a help-seeking decision unit; the help-seeking decision unit tracks the emotional communication pattern of the two family members, determines whether help-seeking is needed and the method of help-seeking, and sends out a help-seeking message.

3. A recognition method for an emotion communication pattern recognition device as claimed in claim 2, characterized in that: The test method is as follows: Step S1, the camera collects images of family members and outputs video signals; the microphone collects voices of family members and outputs voice signals; Step S2, the feature extraction unit receives the video signal and the audio signal, detects and identifies the registered family members, extracts the video features and the audio features of the family members, outputs the first feature vector of the first family member and the second feature vector of the second family member, and repeats the above process every 5 seconds; Step S3, the first emotion recognition unit receives the first feature vector, performs emotion recognition through a video emotion classification network and a speech emotion classification network, and obtains a first emotion trajectory; Step S4, the second emotion recognition unit receives the second feature vector, performs emotion recognition through the video emotion classification network and the speech emotion classification network, and obtains a second emotion trajectory; Step S5: The emotion communication pattern recognition unit receives the first emotion trajectory and the second emotion trajectory and splices the two trajectories, performs emotion communication pattern recognition through an emotion recognition classification network, and obtains the emotion communication pattern of the two family members.

4. A recognition method according to claim 3, characterized in that: The step S2 includes the following subdivision steps: Step S2-1, face detection and identification of registered family members, first use dlib to perform face detection, obtain the detection frame, and then use the existing face recognition network (FaceNet, DeepFace and ArcFace) to implement face recognition and complete the identification of registered family members for subsequent video signal and audio signal processing; Step S2-2, first process the video signal and the audio signal, take the current moment as the end point, take one minute of video and audio data forward, for a one-minute video, extract 32 frames at equal intervals and obtain a frame sequence, and decode the image corresponding to the frame sequence through the existing audio and video processing library av, and then use the image processor of the pre-trained model of vivit to process to obtain a feature vector V, at this time the size of the feature vector is (3, 224, 224); for a one-minute audio signal, the feature vector A can be obtained by preprocessing through the existing technology; Step S2-3: concatenate the feature vector A and the feature vector V as the first feature vector, and perform the above operation every 5 seconds.

5. A recognition method according to claim 3, characterized in that: The step S3 includes the following specific contents: the emotion recognition unit classifies emotions into 7 categories, namely: neutral, happy, sad, angry, afraid, terrified, and surprised; after the first emotion recognition unit obtains the first feature vector, the feature vector V is sent to the video emotion recognition model to obtain the emotion recognition result v, wherein the video emotion recognition model is a fine-tuning of the pre-trained model of Vivit, and the classifier of the last layer of the model is changed; Specifically, the output layer of the classifier is changed from the original 400 categories to 7 categories suitable for the user's emotion recognition task, the backbone network is frozen during the model training process, and the parameters of the last layer are trained; at the same time, the feature vector A is sent to the audio emotion recognition model, and the CNN_LSTM network that combines the convolutional neural network (CNN) and the long short-term memory network (LSTM) is used to realize emotion recognition; The classification results are fused based on decisions to obtain the final classification results. Since the sliding window in step S2 performs the above process every 5 seconds, the personal emotion recognition results are obtained every 5 seconds, and these results are stored over time to form the first emotion trajectory.

6. A recognition method according to claim 5, characterized in that: The specific content of step S4 refers to the specific content of step S3, thereby forming the second emotion trajectory.

7. A recognition method according to claim 3, characterized in that: The step S5 includes the following specific contents: Take the emotion track ten minutes before the current time, that is, the length of the first emotion track and the second emotion track are both 120, and then splice the first emotion track and the second emotion track together, so that the information of the emotion communication is interactive, forming the input of the emotion communication pattern recognition unit; In this unit, the long short-term memory network (LSTM) model is used to classify the emotional communication patterns within ten minutes, and finally the emotional communication patterns between two people are divided into four categories, namely harmony, anger, quarrel, and cold war.

8. A recognition method according to claim 3, characterized in that: Establish a help-seeking decision-making unit based on the judgment of emotional communication patterns; When it is detected that two people have been in a state of disharmony for 12 hours, they will ask the server for help in a timely manner and receive suggestions on how to regulate their emotions. When it is detected that the two people are in a state of disharmony for a long time, the guardian will be asked for help and mediation will be intervened in a timely manner; When it is detected that two people are in a state of anger or cold war for a period of time, some activities to liven up the atmosphere will be pushed in a timely manner.

Images