Emotion recognition method and device based on artificial intelligence, equipment and storage medium

Abstract

The invention provides an emotion recognition method and device based on artificial intelligence, intelligent wearable equipment and a computer readable storage medium, and belongs to the technical field of computer vision. The method comprises the following steps: collecting a video real-time frame of a target object according to a video frame rate of the intelligent wearable device; if the pixel difference between the video real-time frame and the previous recognition frame is greater than a preset difference threshold value, performing feature extraction processing on the video real-time frame through a feature extraction layer of a target emotion recognition model to obtain an image coding feature of the video real-time frame; performing prediction based on the image coding features of the video real-time frame through a classification layer of the target emotion recognition model to obtain a predicted emotion category of the video real-time frame; and determining a current emotion recognition result of the target object based on the predicted emotion category of the video real-time frame and the predicted emotion category of the forward frame of the video real-time frame. According to the invention, the calculation performance of the equipment and the emotion recognition accuracy can be effectively considered.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and specifically to an emotion recognition method, apparatus, smart wearable device, and computer-readable storage medium based on artificial intelligence. Background Art

[0002] With the continuous development of computer vision technology, computer vision technology is being applied in more and more fields. Emotion recognition based on images is an application of computer vision technology. For example, in a digital medical application scenario, the emotions of the object can be recognized based on images to provide auxiliary information for digital medical treatment. However, the image-based recognition model requires a large amount of computing resources. Therefore, it is difficult to balance the computing performance of the device and the accuracy of emotion recognition when deploying emotion recognition functions on smart wearable devices with low computing power. Summary of the invention

[0003] The present application provides an artificial intelligence-based emotion recognition method, apparatus, smart wearable device, and computer-readable storage medium, which belong to the field of computer vision technology and can effectively balance the computing performance of the device and the accuracy of emotion recognition.

[0004] In a first aspect, the present application provides an emotion recognition method based on artificial intelligence, the method comprising:

[0005] According to the video frame rate of the smart wearable device, controlling the smart wearable device to collect real-time video frames of the target object;

[0006] Performing difference detection processing on the real-time video frame and a previous recognition frame of the real-time video frame to obtain a pixel difference between the real-time video frame and the previous recognition frame;

[0007] If the pixel difference is greater than a preset difference threshold, a feature extraction process is performed on the real-time video frame through a feature extraction layer of a target emotion recognition model to obtain an image coding feature of the real-time video frame;

[0008] By using the classification layer of the target emotion recognition model, prediction is performed based on the image coding features of the real-time video frame to obtain the predicted emotion category of the real-time video frame;

[0009] Based on the predicted emotion category of the real-time video frame and the predicted emotion category of the forward frame of the real-time video frame, a current emotion recognition result of the target object is determined.

[0010] In a second aspect, the present application provides an emotion recognition device based on artificial intelligence, the emotion recognition device based on artificial intelligence comprising:

[0011] A collection unit, used to control the smart wearable device to collect real-time frames of video of the target object according to the video frame rate of the smart wearable device;

[0012] A detection unit, configured to perform a difference detection process on the real-time video frame and a previous recognition frame of the real-time video frame to obtain a pixel difference between the real-time video frame and the previous recognition frame;

[0013] A first recognition unit is configured to perform feature extraction processing on the real-time video frame through a feature extraction layer of a target emotion recognition model to obtain image coding features of the real-time video frame if the pixel difference is greater than a preset difference threshold;

[0014] The first recognition unit is further configured to perform prediction based on the image coding features of the real-time video frame through the classification layer of the target emotion recognition model to obtain a predicted emotion category of the real-time video frame;

[0015] The second recognition unit is used to determine the current emotion recognition result of the target object based on the predicted emotion category of the real-time video frame and the predicted emotion category of the forward frame of the real-time video frame.

[0016] In a third aspect, the present application also provides a smart wearable device, which includes a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and implement the artificial intelligence-based emotion recognition method when executing the computer program.

[0017] In a fourth aspect, the present application also provides a computer-readable storage medium on which a computer program is stored, and the computer program is loaded by a processor to execute the artificial intelligence-based emotion recognition method.

[0018] In this application, on the one hand, by controlling the smart wearable device to collect the real-time video frames of the target object according to the video frame rate of the smart wearable device, the corresponding video frame rate can be adapted according to the performance of the smart wearable device to collect image frames for identifying emotion categories, reducing the amount of data processing, and to a certain extent reducing the difficulty of deploying emotion recognition functions on smart wearable devices with low computing power. On the other hand, by performing difference detection processing on the real-time video frame and the previous recognition frame of the real-time video frame, the real-time video frame is further processed only when the pixel difference is greater than the preset difference threshold. Therefore, the judgment redundancy of the image frame can be reduced, and the computing performance utilization rate of the device can be improved, so that the device can concentrate more computing power on emotion recognition judgment, thereby effectively taking into account the computing performance of the device and the accuracy of emotion recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 This is a flow chart of an emotion recognition method based on artificial intelligence provided in an embodiment of the present application;

[0021] Figure 2 It is a schematic diagram of a principle framework of a target emotion recognition model provided in an embodiment of the present application;

[0022] Figure 3 This is a flowchart of an embodiment of step 103 in the embodiment of the present application;

[0023] Figure 4 It is a schematic diagram of a principle framework of a preset emotion recognition model provided in an embodiment of the present application;

[0024] Figure 5 This is a flow chart of an embodiment of the preset emotion recognition model training in the embodiment of the present application;

[0025] Figure 6 It is a schematic diagram of the structure of an embodiment of an emotion recognition device based on artificial intelligence provided in an embodiment of the present application;

[0026] Figure 7 It is a schematic block diagram of the structure of a smart wearable device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0028] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0029] In the description of the embodiments of the present application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0030] In order to enable any technician in the field to implement and use the application, the following description is given. In the following description, details are listed for the purpose of explanation. It should be understood that those of ordinary skill in the art can recognize that the application can also be implemented without using these specific details. In other examples, the known process will not be elaborated in detail to avoid unnecessary details that make the description of the embodiment of the application obscure. Therefore, the application is not intended to be limited to the embodiments shown, but is consistent with the widest range of principles and features disclosed in accordance with the embodiments of the application.

[0031] The embodiments of the present application provide an emotion recognition method, apparatus, smart wearable device and computer-readable storage medium based on artificial intelligence.

[0032] The executor of the artificial intelligence-based emotion recognition method of the embodiment of the present application may be the artificial intelligence-based emotion recognition device provided in the embodiment of the present application, or the smart wearable device provided in the embodiment of the present application, wherein the artificial intelligence-based emotion recognition device may be implemented in hardware or software.

[0033] Some embodiments of the present application are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0034] See also Figure 1 , Figure 1 1 is a flow chart of an emotion recognition method based on artificial intelligence provided in an embodiment of the present application. The emotion recognition method based on artificial intelligence includes steps 101 to 105, wherein:

[0035] 101. Control the smart wearable device to collect real-time video frames of a target object according to the video frame rate of the smart wearable device.

[0036] Among them, the smart wearable device is a wearable device equipped with a camera, for example, it can be smart glasses or smart helmets, wherein the smart glasses can be AR (augmented reality) glasses, VR (Virtual Reality) glasses, myopia glasses or sunglasses, etc. equipped with a camera and loaded with a processor.

[0037] The target object is the object in front of the smart wearable device. For example, in inspection scenarios such as airports, staff can wear smart wearable devices to collect real-time video frames of the inspected person to determine the current emotions of the inspected person and identify potential security risks. For another example, in digital education, digital medical psychological counseling, conversation and other scenarios, real-time video frames of the interlocutor can be collected to determine the interlocutor's current emotions, so as to assist the wearer of the smart wearable device to make more informed decisions based on the interlocutor's current emotions, provide support for mental health testing and intervention, and help people improve their social skills and emotional intelligence.

[0038] For example, an image of a target object opposite to the wearer can be collected, and a continuously collected sequence of images can form a video. Since the size of smart wearable devices (such as smart glasses) is small, the data processing capacity of the processor built into the smart wearable device is relatively low. In order to take into account the recognition accuracy of the target object's emotional category, the amount of data processing is reduced as much as possible to meet the application scenario requirements of the smart wearable device. In this embodiment, the corresponding video frame rate is adapted according to the performance of the smart wearable device, and the smart wearable device is controlled to collect real-time frames of the video of the target object according to the video frame rate adapted by the smart wearable device. For example, different from the traditional video sampling frame rate (such as greater than 30 frames per second), it is dynamically adjusted to a suitable video frame rate interval, such as 10 to 15 frames per second, according to the computing resources of the smart wearable device.

[0039] Furthermore, in order to reduce the power consumption of the smart wearable device, the motion state of the smart wearable device is also detected; when the smart wearable device is in a non-motion state, the smart wearable device is controlled to stop collecting video frames.

[0040] 102. Perform difference detection processing on the real-time video frame and a previous recognition frame of the real-time video frame to obtain a pixel difference between the real-time video frame and the previous recognition frame.

[0041] The last recognized frame is an image frame that is closest in time to the real-time frame of the video and has been used to recognize the predicted emotion category of the target object.

[0042] Since the data processing capability of the smart wearable device is relatively weak, some frame dropping strategies are adopted in this embodiment to perform frame dropping processing on the real-time video frame of the target object. For example, the real-time video frame and the previous recognition frame of the real-time video frame are subjected to difference detection processing to obtain the pixel difference between the real-time video frame and the previous recognition frame. When the pixel difference between the real-time video frame and the previous recognition frame is greater than the preset difference threshold, the process proceeds to step 103; otherwise, when the pixel difference between the real-time video frame and the previous recognition frame is less than or equal to the preset difference threshold, the real-time video frame is discarded. For another example, in some embodiments, some non-task images, blurred images, and occluded images are also discarded to reduce the amount of data processing as much as possible while taking into account the recognition accuracy of the target object's emotion category.

[0043] 103. If the pixel difference is greater than a preset difference threshold, feature extraction processing is performed on the real-time video frame through a feature extraction layer of a target emotion recognition model to obtain image coding features of the real-time video frame.

[0044] In order to better understand this embodiment, the target emotion recognition model in this embodiment is first introduced below. Figure 2 As shown, Figure 2 It is a schematic diagram of the principle framework of the target emotion recognition model provided in the embodiment of the present application. The target emotion recognition model may include a feature extraction layer and a classification layer. In some embodiments, the target emotion recognition model may also include a preprocessing layer. The working principles of each layer of the target emotion recognition model are similar to the working principles of each module corresponding to the preset emotion recognition model (the preprocessing layer, the feature extraction layer, and the classification layer correspond to the preprocessing module, the feature encoding module, and the first classification module, respectively). For details, please refer to the relevant instructions below and will not be repeated here.

[0045] In some embodiments, Figure 3 As shown, step 103 may specifically include the following steps 1031 to 1034:

[0046] 1031. Perform key area detection on the real-time video frame through the divided sub-layers of the feature extraction layer to obtain a face area map and a body area map of the real-time video frame.

[0047] 1032. Perform feature encoding on the face region map of the real-time video frame through the face encoding sublayer of the feature extraction layer to obtain face encoding features of the real-time video frame.

[0048] In some embodiments, in order to improve the facial feature changes of the image, the facial region map of the real-time video frame can also be processed by sequence division to obtain a multi-scale facial feature sequence of the real-time video frame; based on the multi-scale facial feature sequence of the real-time video frame, encoding is performed to obtain the facial change feature of the real-time video frame; the facial region map of the real-time video frame is encoded to obtain the facial detail feature of the real-time video frame; the facial change feature of the real-time video frame and the facial detail feature of the real-time video frame are fused to obtain the facial preliminary feature matrix of the real-time video frame; based on the facial preliminary feature matrix of the real-time video frame, the facial coding feature of the real-time video frame is obtained. In this way, the facial feature changes of the real-time video frame can be captured by combining image sequences of different time scales, avoiding misjudgment caused by facial expression fluctuations, and improving the accuracy of emotion classification.

[0049] In some embodiments, in order to improve the sensitivity of the enhanced model to the facial change area, the attention of the extracted facial coding features to the facial change area is increased to improve the accuracy of emotion category recognition. At this time, step 1032 may specifically include the following steps: first, a preliminary feature extraction process is performed on the facial area map of the real-time video frame to obtain a preliminary facial feature matrix of the real-time video frame; then, in order to improve the sensitivity of the enhanced model to the facial change area, a mapping process is performed on the preliminary facial feature matrix of the real-time video frame to obtain a facial query feature matrix, a facial key feature matrix and a facial value feature matrix of the real-time video frame; based on the facial query feature matrix and the facial key feature matrix of the real-time video frame, a facial correlation matrix of the real-time video frame is obtained; based on the facial correlation matrix of the real-time video frame and the facial value feature matrix of the real-time video frame, a weighted sum is performed to obtain a facial attention matrix of the real-time video frame; based on the facial attention matrix of the real-time video frame and the facial preliminary feature matrix of the real-time video frame, a facial coding feature of the real-time video frame is obtained.

[0050] 1033. Perform feature encoding on the posture area map of the real-time video frame through the posture encoding sublayer of the feature extraction layer to obtain the posture encoding features of the real-time video frame.

[0051] In some embodiments, in order to improve the changes in the posture characteristics of the image, the posture area map of the real-time video frame can also be processed by sequence division to obtain a plurality of scale posture feature sequences of the real-time video frame; based on the plurality of scale posture feature sequences of the real-time video frame, encoding is performed to obtain the posture change characteristics of the real-time video frame; the posture area map of the real-time video frame is encoded to obtain the posture detail characteristics of the real-time video frame; the posture change characteristics of the real-time video frame and the posture detail characteristics of the real-time video frame are fused to obtain the posture preliminary feature matrix of the real-time video frame; based on the posture preliminary feature matrix of the real-time video frame, the posture coding characteristics of the real-time video frame are obtained. In this way, the changes in the posture characteristics of the real-time video frame can be captured by combining image sequences of different time scales, avoiding misjudgment caused by posture fluctuations, and improving the accuracy of emotion classification.

[0052] In some embodiments, in order to improve the sensitivity of the enhanced model to the posture change area, the attention of the extracted posture coding features to the posture change area is increased to improve the accuracy of emotion category recognition. At this time, step 1032 may specifically include the following steps: first, a preliminary feature extraction process is performed on the posture area map of the real-time video frame to obtain a preliminary posture feature matrix of the real-time video frame; then, in order to improve the sensitivity of the enhanced model to the posture change area, a mapping process is performed on the preliminary posture feature matrix of the real-time video frame to obtain a posture query feature matrix, a posture key feature matrix and a posture value feature matrix of the real-time video frame; based on the posture query feature matrix and the posture key feature matrix of the real-time video frame, a posture correlation matrix of the real-time video frame is obtained; based on the posture correlation matrix of the real-time video frame and the posture value feature matrix of the real-time video frame, a weighted sum is performed to obtain a posture attention matrix of the real-time video frame; based on the posture attention matrix of the real-time video frame and the posture preliminary feature matrix of the real-time video frame, the posture coding features of the real-time video frame are obtained.

[0053] 1034. Fuse the facial coding features of the real-time video frame and the body coding features of the real-time video frame to obtain image coding features of the real-time video frame.

[0054] 104. Predicting based on the image coding features of the real-time video frame through the classification layer of the target emotion recognition model to obtain a predicted emotion category of the real-time video frame.

[0055] Exemplarily, after obtaining the image coding features of the real-time video frames, the image coding features of the real-time video frames are input into the classification layer of the target emotion recognition model, and the classification layer of the target emotion recognition model performs prediction based on the image coding features of the real-time video frames to obtain the predicted emotion category of the real-time video frames.

[0056] 105. Determine a current emotion recognition result of the target object based on the predicted emotion category of the real-time video frame and the predicted emotion category of the forward frame of the real-time video frame.

[0057] The forward frame of the real-time video frame is an image frame before the real-time video frame.

[0058] Exemplarily, the predicted emotion category of the image frame at each time point can be determined with reference to steps 101 to 104, and the predicted emotion category of the image frame at each time point (including the predicted emotion category of the real-time video frame and the predicted emotion category of the frame preceding the real-time video frame) can be combined to draw an emotion change curve; if the predicted emotion category within a first preset time length (such as 10 seconds) before the real-time video frame exceeds the second preset time length (such as 3 seconds) and is the same as the predicted emotion category of the real-time video frame, then the predicted emotion category of the frame preceding the real-time video frame is used as the current emotion recognition result of the target object.

[0059] In order to better understand this embodiment, the preset emotion recognition model in this embodiment is first introduced below. Figure 4 As shown, Figure 4 : is a schematic diagram of a principle framework of a preset emotion recognition model provided in an embodiment of the present application. The preset emotion recognition model may include a feature encoding module, a category encoding module, a first classification module, and a second classification module. In some embodiments, the preset emotion recognition model may also include a preprocessing module. The working principles of each module are as follows:

[0060] 1. The preprocessing module is used to preprocess the input video stream, including grayscale and normalization operations, identify and extract key frames, and input the image frames that meet the processing conditions into the feature encoding module for further processing.

[0061] 2. A feature coding module, which is used to encode each input image frame to obtain image coding features of each image frame. In some embodiments, the feature coding module includes a segmentation sublayer, a face coding sublayer, and a posture coding sublayer. The segmentation sublayer of the feature coding module can be used to perform key area detection on the image frame to obtain a face area map and a posture area map of the image frame; the face coding sublayer of the feature coding module can be used to perform feature coding on the posture area map of the image frame to obtain the posture coding features of the image frame; the posture coding sublayer of the feature coding module can be used to perform feature coding on the posture area map of the image frame to obtain the posture coding features of the image frame; the face coding features of the image frame and the posture coding features of the image frame are fused to obtain the image coding features of the image frame.

[0062] In some embodiments, the parameters of the face coding sublayer (i.e., face extraction parameters) are used to extract features from the face region map of the image frame, and a preliminary face feature matrix of the image frame is obtained as the face coding features of the image frame. The parameters of the posture coding sublayer (i.e., posture extraction parameters) are used to extract features from the posture region map of the image frame, and a preliminary posture feature matrix of the image frame is obtained as the posture coding features of the image frame.

[0063] In some embodiments, an attention mechanism may be added to the feature encoding module. In this case, the parameters of the face encoding sublayer may include face extraction parameters and face attention parameters, and the parameters of the posture encoding sublayer may include posture extraction parameters and posture attention parameters. On the one hand, the parameters of the face encoding sublayer (i.e., face extraction parameters) may be used to first extract features from the face region map of the image frame to obtain a preliminary face feature matrix of the image frame; then, the parameters of the face encoding sublayer (i.e., face attention parameters) may be used to map the preliminary face feature matrix of the image frame to obtain a face query feature matrix, a face key feature matrix, and a face value feature matrix of the image frame; based on the face query feature matrix and the face key feature matrix of the image frame, a face correlation matrix of the image frame is obtained; based on the face correlation matrix of the image frame and the face value feature matrix of the image frame, a weighted sum is performed to obtain a face attention matrix of the image frame; finally, based on the face attention matrix of the image frame and the face preliminary feature matrix of the image frame, a weighted sum is performed to obtain a face encoding feature of the image frame. On the other hand, the parameters of the posture encoding sublayer (i.e., posture extraction parameters) can be used to first perform feature extraction on the posture area map of the image frame to obtain a preliminary posture feature matrix of the image frame; then, the parameters of the posture encoding sublayer (i.e., posture attention parameters) can be used to map the preliminary posture feature matrix of the image frame to obtain a posture query feature matrix, a posture key feature matrix, and a posture value feature matrix of the image frame; based on the posture query feature matrix and the posture key feature matrix of the image frame, a posture correlation matrix of the image frame is obtained; based on the posture correlation matrix of the image frame and the posture value feature matrix of the image frame, a weighted summation is performed to obtain a posture attention matrix of the image frame; finally, based on the posture attention matrix of the image frame and the posture preliminary feature matrix of the image frame, a weighted summation is performed to obtain the posture coding features of the image frame.

[0064] 3. A category coding module is used to encode the actual emotion category of each input image frame to obtain the category coding features of each image frame.

[0065] 4. A first classification module, used to perform classification prediction based on the image coding features of each image frame to obtain a first predicted emotion category of each image frame. For example, the emotion categories may be: normal A, happy B, sad C, etc. In some embodiments, a support vector machine (SVM) or a deep neural network may be used as the first classification module to identify different emotion categories.

[0066] 5. A second classification module, used to perform classification prediction based on the category coding features of each image frame to obtain a second predicted emotion category of each image frame. In some embodiments, a support vector machine or a deep neural network can be used as the second classification module to identify different emotion categories.

[0067] For example, Figure 5 As shown, the preset emotion recognition model can be trained through the following steps 501 to 505 to obtain a trained emotion recognition model:

[0068] 501. Perform encoding processing on each processed frame of the sample video through a feature encoding module in a preset emotion recognition model to obtain image encoding features of each processed frame.

[0069] In some embodiments, before entering step 501, the sample video will be preprocessed by a preprocessing module in a preset emotion recognition model. For example, first, according to a preset frame dropping strategy (such as whether it is a non-task image, whether it is a blurred image, or whether it is an occluded image), the sample video is subjected to frame dropping processing, and it is determined whether each frame of the sample video needs to be dropped. After the frame dropping processing, a preliminary image set of the sample video is obtained to avoid the problem of misrecognition caused by non-task images, blurred images, occluded images, etc. Then, it is determined whether each image in the preliminary image set needs to be processed. If it is determined that it needs to be processed, it is added to the processing image set. Each frame in the processing image set can be used as a processing frame in step 501. For example, each image in the preliminary image set is traversed in sequence according to the acquisition time, and the pixel difference between the current traversed image frame and the previous processing frame (that is, the image frame in the preliminary image set that is closest to the current traversed image frame and has been determined to need to be processed) is obtained. If the pixel difference between the current traversed image frame and the previous processing frame is greater than a preset difference threshold, it is considered that the current traversed image frame needs to be processed, and the current traversed image frame is added to the processing image set as a processing frame.

[0070] There are many implementation methods in step 501, which include, for example:

[0071] (1) In some embodiments, the facial region map of each processing frame is feature extracted by using the parameters of the facial coding sublayer in the feature coding module (i.e., facial extraction parameters), and a preliminary facial feature matrix of each processing frame is obtained as the facial coding features of each processing frame. The posture region map of each processing frame is feature extracted by using the parameters of the posture coding sublayer in the feature coding module (i.e., posture extraction parameters), and a preliminary posture feature matrix of each processing frame is obtained as the posture coding features of each processing frame. Finally, the facial coding features of each processing frame and the posture coding features of each processing frame are fused and spliced ​​to obtain the image coding features of each processing frame.

[0072] In some embodiments, in order to improve the facial feature changes of the image, the facial region map of each processing frame can also be processed by sequence division to obtain multiple scale facial feature sequences of each processing frame; based on the multiple scale facial feature sequences of each processing frame, encoding is performed to obtain the facial change features of each processing frame; the facial region map of each processing frame is encoded to obtain the facial detail features of each processing frame; the facial change features of each processing frame and the facial detail features of each processing frame are fused to obtain the facial preliminary feature matrix of each processing frame; based on the facial preliminary feature matrix of each processing frame, the facial coding features of each processing frame are obtained. In this way, the facial feature changes of each processing frame can be captured by combining image sequences of different time scales, avoiding misjudgment caused by facial expression fluctuations, and improving the accuracy of emotion classification.

[0073] In some embodiments, in order to improve the changes in the posture characteristics of the image, the posture area map of each processing frame can also be processed by sequence division to obtain a plurality of scale posture feature sequences of each processing frame; based on the plurality of scale posture feature sequences of each processing frame, encoding is performed to obtain the posture change characteristics of each processing frame; the posture area map of each processing frame is encoded to obtain the posture detail characteristics of each processing frame; the posture change characteristics of each processing frame and the posture detail characteristics of each processing frame are fused to obtain the posture preliminary feature matrix of each processing frame; based on the posture preliminary feature matrix of each processing frame, the posture coding characteristics of each processing frame are obtained. In this way, the changes in the posture characteristics of each processing frame can be captured by combining image sequences of different time scales, avoiding misjudgment caused by posture fluctuations, and improving the accuracy of emotion classification.

[0074] (2) In some embodiments, an attention mechanism may be added to the feature encoding module. In this case, the parameters of the face encoding sublayer may include face extraction parameters and face attention parameters, and the parameters of the body encoding sublayer may include face extraction parameters and face attention parameters. On the one hand, the parameters of the face encoding sublayer (i.e., face extraction parameters) may be used to first extract features from the face region map of each processing frame to obtain a preliminary face feature matrix of each processing frame; then, the parameters of the face encoding sublayer (i.e., face attention parameters) may be used to map the preliminary face feature matrix of each processing frame to obtain a face query feature matrix, a face key feature matrix, and a face value feature matrix of each processing frame; based on the face query feature matrix and the face key feature matrix of each processing frame, a face correlation matrix of each processing frame may be obtained; based on the face correlation matrix of each processing frame and the face value feature matrix of each processing frame, a weighted sum is performed to obtain a face attention matrix of each processing frame; then, based on the face attention matrix of each processing frame and the face preliminary feature matrix of each processing frame, a weighted sum is performed to obtain a face encoding feature of each processing frame. On the other hand, the parameters of the posture encoding sublayer (i.e., posture extraction parameters) can be used to first extract features from the posture region map of each processing frame to obtain the posture preliminary feature matrix of each processing frame; then, the parameters of the posture encoding sublayer (i.e., posture attention parameters) can be used to map the posture preliminary feature matrix of each processing frame to obtain the posture query feature matrix, posture key feature matrix, and posture value feature matrix of each processing frame; based on the posture query feature matrix and posture key feature matrix of each processing frame, the posture correlation matrix of each processing frame is obtained; based on the posture correlation matrix of each processing frame and the posture value feature matrix of each processing frame, a weighted sum is performed to obtain the posture attention matrix of each processing frame; then, based on the posture attention matrix of each processing frame and the posture preliminary feature matrix of each processing frame, a weighted sum is performed to obtain the posture coding features of each processing frame. Finally, the face coding features of each processing frame and the posture coding features of each processing frame are fused and spliced ​​to obtain the image coding features of each processing frame.

[0075] 502. Based on the image coding features of each processed frame, obtain a first Gaussian distribution feature of the sample video.

[0076] The first Gaussian distribution feature refers to a result obtained by performing Gaussian distribution calculation using image coding features of each processed frame of the sample video.

[0077] 503. Perform encoding processing on the actual emotion category of each processed frame through the category encoding module in the preset emotion recognition model to obtain the category encoding features of each processed frame.

[0078] 504. Based on the category coding features of each processed frame, obtain a second Gaussian distribution feature of the sample video.

[0079] The second Gaussian distribution feature refers to a result obtained by performing Gaussian distribution calculation using the category coding features of each processed frame of the sample video.

[0080] 505. Based on a first loss between the first Gaussian distribution feature and the second Gaussian distribution feature, train the preset emotion recognition model to obtain a trained emotion recognition model.

[0081] There are many ways to implement the training of the preset emotion recognition model in step 505. For ease of understanding, in this embodiment, "after the parameters of the category encoding module, the first classification module, and the second classification module are learned and fixed, the model parameters of the feature encoding module of the preset emotion recognition module are trained and learned" is taken as an example. Exemplarily, step 505 includes the following implementation methods:

[0082] (1) Using the first loss (reference Figure 4 At this time, step 505 may specifically include: obtaining a first loss between the first Gaussian distribution feature and the second Gaussian distribution feature; adjusting parameters of a feature coding module of a preset emotion recognition model (such as parameters of a face coding sublayer of the feature coding module, such as face extraction parameters and face attention parameters, parameters of a body coding sublayer of the feature coding module, such as body extraction parameters and body attention parameters, etc.) until the training stop condition is met, thereby obtaining a trained emotion recognition model.

[0083] Among them, the category encoding module and the second classification module can be trained using sample images alone.

[0084] (2) Using the first loss and the second loss (reference Figure 4 L2 in the above). Exemplarily, at this time, step 505 may specifically include: obtaining a first loss between the first Gaussian distribution feature and the second Gaussian distribution feature; predicting based on the image coding features of each processed frame through a first classification module of a preset emotion recognition model to obtain a first predicted emotion category of each processed frame; determining a second loss of the feature coding module based on the first predicted emotion category of each processed frame and the actual predicted emotion category of each processed frame; adjusting the parameters of the feature coding module of the preset emotion recognition model (such as the parameters of the face coding sublayer of the feature coding module, such as face extraction parameters and face attention parameters, and the parameters of the body coding sublayer of the feature coding module, such as body extraction parameters and body attention parameters, etc.) based on the first loss and the second loss until the training stop condition is met, thereby obtaining a trained emotion recognition model.

[0085] (3) Using the first loss, second loss and third loss (reference Figure 4L3 in the above). Exemplarily, at this time, step 505 may specifically include: obtaining a first loss between the first Gaussian distribution feature and the second Gaussian distribution feature; predicting based on the image coding features of each processed frame through a first classification module of a preset emotion recognition model to obtain a first predicted emotion category of each processed frame; determining a second loss of the feature coding module based on the first predicted emotion category of each processed frame and the actual predicted emotion category of each processed frame; predicting based on the image coding features of each processed frame through a second classification module of a preset emotion recognition model to obtain a second predicted emotion category of each processed frame; determining a third loss of the feature coding module based on the second predicted emotion category of each processed frame and the actual predicted emotion category of each processed frame; adjusting the parameters of the feature coding module of the preset emotion recognition model (such as the parameters of the face coding sublayer of the feature coding module, such as the face extraction parameters and the face attention parameters, and the parameters of the body coding sublayer of the feature coding module, such as the body extraction parameters and the body attention parameters, etc.) based on the first loss, the second loss and the third loss until the training stop condition is met, thereby obtaining a trained emotion recognition model.

[0086] The validation set can be used to optimize the trained emotion recognition model, adjust hyperparameters such as learning rate and regularization coefficient, and evaluate the performance of the model on an independent test set to ensure the generalization ability of the model. In actual applications, user feedback and model prediction results are collected to continuously iterate and optimize the model.

[0087] Among them, the trained emotion recognition model can be compressed in at least one of the following ways to make the device more suitable for smart wearable devices with lower processing performance: a. Pruning: removing unimportant weights or neurons in the neural network to reduce the model size; b. Quantization: converting the model weights from floating point numbers to low-bit width representations (such as int8) to reduce the model size and computing requirements; c. Knowledge distillation: transferring the knowledge of a large, trained model to a smaller model.

[0088] Therefore, after obtaining the trained emotion recognition model, in some embodiments, the feature extraction module of the trained emotion recognition model can be extracted as the feature extraction layer of the target emotion recognition model, and the first classification module can be used as the classification layer of the target emotion recognition model to build the target emotion recognition model, thereby building a target emotion recognition model with a relatively low load, thereby improving the usability of the target emotion recognition model on the smart wearable device. Furthermore, the preprocessing module of the trained emotion recognition model can also be used as the preprocessing layer of the target emotion recognition model.

[0089] In some embodiments, the above steps 101 to 105 are all performed by a smart wearable device.

[0090] In some embodiments, in order to improve the availability of the emotion category recognition function on the smart wearable device and take into account the recognition accuracy of the emotion category, the image preprocessing (such as the acquisition, graying, and normalization of real-time video frames) and the simple data processing such as determining whether the real-time video frames need to be dropped can also be performed on the smart wearable device side, and the relatively complex model reasoning process (such as feature extraction and classification prediction of image frames) can be performed. That is, the target emotion recognition model can be deployed on the server, and steps 101 to 102 can be performed on the smart wearable device side, and the data obtained after processing steps 101 to 102 can be sent to the server, and the server executes steps 103 to 104 through the target emotion recognition model deployed by it, and returns the execution result (i.e., the predicted emotion category of the real-time video frame) to the smart wearable device; finally, the smart wearable device executes step 105 according to the predicted emotion category of the real-time video frame fed back by the server, and determines the current emotion recognition result of the target object. In this way, the smart wearable device can send real-time video frames to the server; call the server, perform feature extraction processing on the real-time video frames through the feature extraction layer of the target emotion recognition model, and obtain the image coding features of the real-time video frames; call the server, perform prediction based on the image coding features of the real-time video frames through the classification layer of the target emotion recognition model, and obtain the predicted emotion category of the real-time video frames.

[0091] From the above content, it can be seen that, on the one hand, by controlling the smart wearable device to collect the real-time video frames of the target object according to the video frame rate of the smart wearable device, the corresponding video frame rate can be adapted according to the performance of the smart wearable device to collect image frames for identifying emotion categories, reducing the amount of data processing, and to a certain extent reducing the difficulty of deploying emotion recognition functions on smart wearable devices with low computing power. On the other hand, by performing difference detection processing on the real-time video frame and the previous recognition frame of the real-time video frame, the real-time video frame is further processed only when the pixel difference is greater than the preset difference threshold. Therefore, the judgment redundancy of the image frame can be reduced, and the computing performance utilization rate of the device can be improved, so that the device can concentrate more computing power on the emotion recognition judgment, thereby effectively taking into account the computing performance of the device and the accuracy of emotion recognition. On the third hand, through the frame loss strategy, some non-task images, blurred images, and occluded images can be discarded, reducing the misjudgment of emotion categories caused by such images, so as to reduce the amount of data processing as much as possible while taking into account the recognition accuracy of the target object's emotion category.

[0092] Therefore, this embodiment can achieve the following benefits: 1) Improve accuracy: By dropping frames for non-person images, or blurred, occluded images, the image sampling accuracy is improved from the source, combined with the encoding of multiple scale facial feature sequences based on real-time video frames in step 1032, and the calculation of facial attention matrix, and the encoding of multiple scale body feature sequences based on real-time video frames in step 1033, and the calculation of body attention matrix, to avoid misjudgment caused by expression fluctuations; 2) Improve performance and battery life: Through sleep detection (that is, when the smart wearable device is in a non-motion state, control the smart wearable device to stop collecting video frames), model compression, sampling down-frequency (that is, control the real-time frame of the smart wearable device video according to the video frame rate of the smart wearable device), frame dropping strategy, heterogeneous preprocessing (that is, the server executes steps 103 to 104 through the target emotion recognition model deployed by it), etc., the amount of data processed by the moving picture is greatly reduced, thereby improving performance and battery life.

[0093] In addition, in order to better implement the emotion recognition method based on artificial intelligence in the embodiment of the present application, on the basis of the emotion recognition method based on artificial intelligence, the embodiment of the present application also provides an emotion recognition device based on artificial intelligence, such as Figure 6 FIG. 1 is a schematic diagram of a structure of an embodiment of an artificial intelligence-based emotion recognition device provided in an embodiment of the present application. The artificial intelligence-based emotion recognition device 600 includes:

[0094] The acquisition unit 601 is used to control the smart wearable device to acquire real-time frames of the video of the target object according to the video frame rate of the smart wearable device;

[0095] A detection unit 602 is used to perform a difference detection process on the real-time video frame and a previous recognition frame of the real-time video frame to obtain a pixel difference between the real-time video frame and the previous recognition frame;

[0096] The first recognition unit 603 is configured to perform feature extraction processing on the real-time video frame through a feature extraction layer of a target emotion recognition model to obtain image coding features of the real-time video frame if the pixel difference is greater than a preset difference threshold;

[0097] The first recognition unit 603 is further configured to perform prediction based on the image coding features of the real-time video frame through the classification layer of the target emotion recognition model to obtain a predicted emotion category of the real-time video frame;

[0098] The second recognition unit 604 is used to determine the current emotion recognition result of the target object based on the predicted emotion category of the real-time video frame and the predicted emotion category of the forward frame of the real-time video frame.

[0099] In some embodiments, the first identification unit 603 is used to:

[0100] By using the sub-layers of the feature extraction layer, key area detection is performed on the real-time video frame to obtain a face area map and a body area map of the real-time video frame;

[0101] Performing feature encoding on the face region map of the real-time video frame through the face encoding sublayer of the feature extraction layer to obtain face encoding features of the real-time video frame;

[0102] Performing feature encoding on the posture region map of the real-time video frame through the posture encoding sublayer of the feature extraction layer to obtain the posture encoding features of the real-time video frame;

[0103] The face coding features of the real-time video frame and the body coding features of the real-time video frame are fused to obtain the image coding features of the real-time video frame.

[0104] In some embodiments, the first recognition unit 603 is used to: perform sequence division processing on the face area map of the real-time video frame to obtain multiple scale facial feature sequences of the real-time video frame;

[0105] Encoding based on multiple scale facial feature sequences of the real-time video frame to obtain facial change features of the real-time video frame;

[0106] Encoding the face area map of the real-time video frame to obtain face detail features of the real-time video frame;

[0107] Performing fusion processing on the facial change features of the real-time video frame and the facial detail features of the real-time video frame to obtain a preliminary facial feature matrix of the real-time video frame;

[0108] Based on the preliminary facial feature matrix of the real-time video frame, the facial coding features of the real-time video frame are obtained.

[0109] In some embodiments, the first recognition unit 603 is used to: perform preliminary feature extraction processing on the face area map of the real-time video frame to obtain a preliminary face feature matrix of the real-time video frame;

[0110] Mapping the preliminary facial feature matrix of the real-time video frame to obtain a facial query feature matrix, a facial key feature matrix and a facial value feature matrix of the real-time video frame;

[0111] Acquire a face correlation matrix of the real-time video frame based on the face query feature matrix and the face key feature matrix of the real-time video frame;

[0112] Performing weighted summation based on the face correlation matrix of the real-time video frame and the face value feature matrix of the real-time video frame to obtain a face attention matrix of the real-time video frame;

[0113] Based on the face attention matrix of the real-time video frame and the face preliminary feature matrix of the real-time video frame, the face coding feature of the real-time video frame is obtained.

[0114] In some embodiments, the artificial intelligence-based emotion recognition device further includes a training unit (not shown in the figure) for:

[0115] By using a feature encoding module in a preset emotion recognition model, encoding is performed on each processed frame of the sample video to obtain image encoding features of each processed frame;

[0116] Based on the image coding features of each processed frame, obtaining a first Gaussian distribution feature of the sample video;

[0117] By means of a category coding module in the preset emotion recognition model, encoding is performed on the actual emotion category of each processing frame to obtain a category coding feature of each processing frame;

[0118] Based on the category coding features of each processed frame, obtaining a second Gaussian distribution feature of the sample video;

[0119] Based on a first loss between the first Gaussian distribution feature and the second Gaussian distribution feature, the feature encoding module of the preset emotion recognition model is trained to obtain a trained feature encoding module to serve as the feature extraction layer.

[0120] In some embodiments, the detection unit 602 is used to:

[0121] Detecting the motion state of the smart wearable device;

[0122] When the smart wearable device is in a non-moving state, the smart wearable device is controlled to stop collecting video frames.

[0123] In some embodiments, the first identification unit 603 is used to:

[0124] Sending the real-time video frame to a server;

[0125] Calling the server to perform feature extraction processing on the real-time video frame through the feature extraction layer of the target emotion recognition model to obtain image coding features of the real-time video frame;

[0126] The server is called to perform prediction based on the image coding features of the real-time video frame through the classification layer of the target emotion recognition model to obtain the predicted emotion category of the real-time video frame.

[0127] In specific implementation, the above units can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units can refer to the previous embodiment of the emotion recognition method based on artificial intelligence, which will not be repeated here.

[0128] See also Figure 7 , Figure 7 A schematic block diagram of the structure of a smart wearable device provided in an embodiment of the present application. The smart wearable device may be a server.

[0129] like Figure 7 As shown, the smart wearable device includes a processor, a memory, and a network interface connected via a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.

[0130] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any emotion recognition method based on artificial intelligence.

[0131] The processor is used to provide computing and control capabilities to support the operation of the entire smart wearable device.

[0132] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any emotion recognition method based on artificial intelligence.

[0133] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will appreciate that Figure 7 The structure shown in the figure is only a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the smart wearable device to which the scheme of the present application is applied. The specific smart wearable device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0134] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. Among them, the processor is used to run a computer program stored in a memory to implement the steps in the emotion recognition method based on artificial intelligence.

[0135] A person of ordinary skill in the art will appreciate that all or part of the steps in the above-mentioned artificial intelligence-based emotion recognition method can be completed by instructions, or by controlling related hardware through instructions. The instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0136] To this end, an embodiment of the present application provides a computer-readable storage medium, in which multiple computer programs are stored. The computer programs can be loaded by a processor to execute any one of the artificial intelligence-based emotion recognition methods provided in the embodiments of the present application.

[0137] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0138] In the above-mentioned artificial intelligence-based emotion recognition device and computer-readable storage medium embodiments, the description of each embodiment has its own emphasis. For parts not described in detail in a certain embodiment, reference can be made to the relevant description of other embodiments. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the above-described artificial intelligence-based emotion recognition device, computer-readable storage medium and the specific working process and beneficial effects that can be brought about can refer to the description of the artificial intelligence-based emotion recognition method in the above embodiment, and the specific details will not be repeated here.

[0139] The above is a detailed introduction to an emotion recognition method, device, smart wearable device and computer-readable storage medium based on artificial intelligence provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and its core idea of ​​the present application; at the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application. The non-company software tools or components that appear in the embodiments of the present application are only examples and do not represent actual use.

Claims

1. An emotion recognition method based on artificial intelligence, characterized in that: The method comprises: According to the video frame rate of the smart wearable device, controlling the smart wearable device to collect real-time video frames of the target object; Performing difference detection processing on the real-time video frame and a previous recognition frame of the real-time video frame to obtain a pixel difference between the real-time video frame and the previous recognition frame; If the pixel difference is greater than a preset difference threshold, a feature extraction process is performed on the real-time video frame through a feature extraction layer of a target emotion recognition model to obtain an image coding feature of the real-time video frame; By using the classification layer of the target emotion recognition model, prediction is performed based on the image coding features of the real-time video frame to obtain a predicted emotion category of the real-time video frame; Based on the predicted emotion category of the real-time video frame and the predicted emotion category of the forward frame of the real-time video frame, a current emotion recognition result of the target object is determined.

2. The emotion recognition method based on artificial intelligence according to claim 1, characterized in that: The feature extraction layer of the target emotion recognition model is used to perform feature extraction processing on the real-time video frame to obtain image coding features of the real-time video frame, including: By using the sub-layers of the feature extraction layer, key area detection is performed on the real-time video frame to obtain a face area map and a body area map of the real-time video frame; Performing feature encoding on the face region map of the real-time video frame through the face encoding sublayer of the feature extraction layer to obtain face encoding features of the real-time video frame; Performing feature encoding on the posture region map of the real-time video frame through the posture encoding sublayer of the feature extraction layer to obtain the posture encoding features of the real-time video frame; The face coding features of the real-time video frame and the body coding features of the real-time video frame are fused to obtain the image coding features of the real-time video frame.

3. The emotion recognition method based on artificial intelligence according to claim 2, characterized in that: The step of encoding the facial region map of the real-time video frame to obtain facial encoding features of the real-time video frame includes: Performing sequence division processing on the face region map of the real-time video frame to obtain multiple scale face feature sequences of the real-time video frame; Encoding based on multiple scale facial feature sequences of the real-time video frame to obtain facial change features of the real-time video frame; Encoding the face area map of the real-time video frame to obtain face detail features of the real-time video frame; Performing fusion processing on the facial change features of the real-time video frame and the facial detail features of the real-time video frame to obtain a preliminary facial feature matrix of the real-time video frame; Based on the preliminary facial feature matrix of the real-time video frame, the facial coding features of the real-time video frame are obtained.

4. The emotion recognition method based on artificial intelligence according to claim 2, characterized in that: The step of encoding the facial region map of the real-time video frame to obtain facial encoding features of the real-time video frame includes: Performing preliminary feature extraction processing on the face region map of the real-time video frame to obtain a preliminary face feature matrix of the real-time video frame; Mapping the preliminary facial feature matrix of the real-time video frame to obtain a facial query feature matrix, a facial key feature matrix and a facial value feature matrix of the real-time video frame; Acquire a face correlation matrix of the real-time video frame based on the face query feature matrix and the face key feature matrix of the real-time video frame; Performing weighted summation based on the face correlation matrix of the real-time video frame and the face value feature matrix of the real-time video frame to obtain a face attention matrix of the real-time video frame; Based on the facial attention matrix of the real-time video frame and the facial preliminary feature matrix of the real-time video frame, the facial coding features of the real-time video frame are obtained.

5. The emotion recognition method based on artificial intelligence according to claim 1, characterized in that: The feature extraction layer is obtained in the following way: By using a feature encoding module in a preset emotion recognition model, encoding is performed on each processed frame of the sample video to obtain image encoding features of each processed frame; Based on the image coding features of each processed frame, obtaining a first Gaussian distribution feature of the sample video; By means of a category coding module in the preset emotion recognition model, encoding is performed on the actual emotion category of each processing frame to obtain a category coding feature of each processing frame; Based on the category coding features of each processed frame, obtaining a second Gaussian distribution feature of the sample video; Based on a first loss between the first Gaussian distribution feature and the second Gaussian distribution feature, the feature encoding module of the preset emotion recognition model is trained to obtain a trained feature encoding module to serve as the feature extraction layer.

6. The emotion recognition method based on artificial intelligence according to claim 1, characterized in that: The method further comprises: Detecting the motion state of the smart wearable device; When the smart wearable device is in a non-moving state, the smart wearable device is controlled to stop collecting video frames.

7. The emotion recognition method based on artificial intelligence according to claim 1, characterized in that: The method further comprises: Sending the real-time video frame to a server; Calling the server to perform feature extraction processing on the real-time video frame through the feature extraction layer of the target emotion recognition model to obtain image coding features of the real-time video frame; The server is called to perform prediction based on the image coding features of the real-time video frame through the classification layer of the target emotion recognition model to obtain the predicted emotion category of the real-time video frame.

8. An emotion recognition device based on artificial intelligence, characterized in that: The emotion recognition device based on artificial intelligence includes: A collection unit, used to control the smart wearable device to collect real-time frames of video of the target object according to the video frame rate of the smart wearable device; A detection unit, configured to perform a difference detection process on the real-time video frame and a previous recognition frame of the real-time video frame to obtain a pixel difference between the real-time video frame and the previous recognition frame; A first recognition unit is configured to perform feature extraction processing on the real-time video frame through a feature extraction layer of a target emotion recognition model to obtain image coding features of the real-time video frame if the pixel difference is greater than a preset difference threshold; The first recognition unit is further configured to perform prediction based on the image coding features of the real-time video frame through the classification layer of the target emotion recognition model to obtain a predicted emotion category of the real-time video frame; The second recognition unit is used to determine the current emotion recognition result of the target object based on the predicted emotion category of the real-time video frame and the predicted emotion category of the forward frame of the real-time video frame.

9. A smart wearable device, characterized in that: The smart wearable device includes a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the artificial intelligence-based emotion recognition method as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program is loaded by a processor to execute the artificial intelligence-based emotion recognition method described in any one of claims 1 to 7.

Images