Face expression recognition method and device, computer readable storage medium, and terminal equipment

Abstract

A face expression recognition method and device, a computer readable storage medium and a terminal device are provided. The face expression recognition method comprises: obtaining a face image to be recognized, the face image comprising a target face; performing key point detection on the target face to obtain a plurality of face key points; connecting the plurality of key points to form a plurality of grid regions according to functional regions in the target face, the functional regions being regions in the face that affect the face expression, each grid region covering one functional region; inputting the plurality of grid regions and the target face into a trained classification model, so that the classification model outputs a plurality of motion unit encodings based on the plurality of grid regions; and determining a face expression of the target face by using the plurality of motion unit encodings and a mapping relationship between a motion unit encoding combination and a face expression. The technical scheme can improve the accuracy of face expression recognition.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method and apparatus for facial expression recognition, a computer-readable storage medium, and a terminal device. Background Technology

[0002] Generally speaking, common facial expressions can be categorized into seven types: calm, surprise, anger, sadness, happiness, fear, and disgust. The newly proposed facial behavior coding system associates various human emotions with numerous facial action units (AUs).

[0003] To recognize facial expressions, the existing technical solution is to use deep learning algorithms to input the entire face image into the recognition model, thereby predicting the motion unit code of the current face.

[0004] However, existing methods that input the entire face image block into the recognition model result in large prediction errors, leading to inaccurate emotion recognition. Furthermore, due to differences in individual appearance and the angle at which face images are captured, methods that directly input face images into the recognition model have poor robustness. Summary of the Invention

[0005] The technical problem solved by this invention is how to achieve accurate facial expression recognition.

[0006] To address the aforementioned technical problems, this invention provides a facial expression recognition method, comprising: acquiring a face image to be recognized, the face image including a target face; performing key point detection on the target face to obtain multiple facial key points; connecting the multiple key points according to functional regions in the target face to form multiple grid regions, the functional regions being regions in the face that affect facial expressions, each grid region covering one functional region; inputting the multiple grid regions and the target face into a trained classification model, so that the classification model outputs multiple motion unit codes based on the multiple grid regions; and determining the facial expression of the target face using the multiple motion unit codes and the mapping relationship between the combination of motion unit codes and facial expressions.

[0007] Optionally, the step of connecting the multiple key points to form multiple grid regions based on the functional regions in the target face includes: connecting adjacent key points among the multiple key points to obtain multiple regions of interest; and merging at least one region of interest located within the same functional region to obtain the grid region.

[0008] Optionally, connecting adjacent key points among the plurality of key points includes: selecting a key point located at the contour position of the target face as a starting position point; starting from the starting position point, connecting adjacent key points sequentially in order from the contour position to the center position of the target face.

[0009] Optionally, connecting adjacent key points among the plurality of key points includes: if the distance between adjacent key points is greater than a preset threshold, selecting at least one intermediate key point among the adjacent key points; and connecting the plurality of key points and adjacent key points among the selected intermediate key points.

[0010] Optionally, the step of inputting the multiple grid regions and the target face into the trained classification model includes: determining the minimum bounding rectangle region of each grid region, and inputting the minimum bounding rectangle region and the target face into the classification model.

[0011] Optionally, the target face is a face image block, and after obtaining the face image to be identified, the method further includes: performing face recognition on the face image to be identified; and extracting the region of the identified face in the face image to be identified as the face image block.

[0012] Optionally, the classification model can be trained in the following way: constructing the classification model using a neural network; constructing a training sample set, which includes face images, grid regions in the face images, and facial expressions corresponding to the face images; and training the classification model using the training sample set to obtain a trained classification model.

[0013] This invention also discloses a facial expression recognition device, comprising: an acquisition module for acquiring a face image to be recognized, the face image including a target face; a detection module for performing key point detection on the target face to obtain multiple facial key points; a grid region forming module for connecting the multiple key points to form multiple grid regions based on functional regions in the target face, the functional regions being regions in the face that affect facial expressions, each grid region covering one functional region; a classification module for inputting the multiple grid regions and the target face into a trained classification model, so that the classification model outputs multiple motion unit codes based on the multiple grid regions; and a mapping module for determining the facial expression of the target face using the multiple motion unit codes and the mapping relationship between the combination of motion unit codes and facial expressions.

[0014] This invention also discloses a computer-readable storage medium storing a computer program thereon, wherein the computer program is executed by a processor to perform the steps of the facial expression recognition method.

[0015] This invention also discloses a terminal device, including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor runs the computer program, it executes the steps of the facial expression recognition method.

[0016] Compared with the prior art, the technical solution of the embodiments of the present invention has the following beneficial effects:

[0017] In this invention, for a face image to be identified, key points in the target face can be detected, and grid regions covering the face area can be formed based on these key points. Functional regions refer to areas that influence facial expressions. In other words, by inputting the grid regions in the face image that influence facial expressions into the classification model, the model identifies motion unit codes only based on these grid regions, avoiding interference from other irrelevant areas in the face image. This ensures the accuracy of motion unit code prediction and, consequently, the accuracy of facial expression recognition.

[0018] Furthermore, in determining the region of interest, if the distance between adjacent key points is greater than a preset threshold, at least one intermediate key point is selected between the adjacent key points; the multiple key points and adjacent key points among the selected intermediate key points are then connected. By selecting intermediate key points and connecting them with other key points, the area of ​​the formed region of interest can be minimized. Since the grid region is obtained by merging multiple regions of interest, a smaller area of ​​the region of interest ensures the accuracy of the grid region determination. Attached Figure Description

[0019] Figure 1 This is a flowchart of a facial expression recognition method according to an embodiment of the present invention;

[0020] Figure 2 This is a schematic diagram of key points in a human face according to an embodiment of the present invention;

[0021] Figure 3 This is a schematic diagram of a grid region according to an embodiment of the present invention;

[0022] Figure 4 This is a schematic diagram of a region of interest according to an embodiment of the present invention;

[0023] Figure 5 This is a schematic diagram of the structure of a facial expression recognition device according to an embodiment of the present invention. Detailed Implementation

[0024] As described in the background section, existing methods that input the entire face image block into the recognition model result in large prediction errors, leading to inaccurate emotion recognition. Furthermore, due to differences in individual appearances and the varying angles from which face images are captured, methods that directly input face images into the recognition model exhibit poor robustness.

[0025] In this invention, for a face image to be identified, key points in the target face can be detected, and grid regions covering the face area are formed based on these key points. Functional regions refer to areas that influence facial expressions. In other words, by inputting the grid regions in the face image that influence facial expressions into the classification model, the model identifies motion unit codes only based on these grid regions, avoiding interference from other irrelevant areas in the face image. This ensures the accuracy of motion unit code prediction and, consequently, the accuracy of facial expression recognition.

[0026] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0027] Figure 1 This is a flowchart of a facial expression recognition method according to an embodiment of the present invention.

[0028] The facial expression recognition method of this invention can be used on the terminal device side, that is, the terminal device can execute the various steps of the above method. Specifically, the terminal device can be a mobile phone, computer, tablet computer, or other similar device.

[0029] Specifically, facial expression recognition methods may include the following steps:

[0030] Step 101: Obtain the face image to be identified, wherein the face image includes the target face;

[0031] Step 102: Perform key point detection on the target face to obtain multiple facial key points;

[0032] Step 103: Connect the multiple key points according to the functional regions in the target face to form multiple grid regions. The functional regions are the areas in the face that affect facial expressions. Each grid region covers one functional region.

[0033] Step 104: Input the multiple grid regions and the target face into the trained classification model, so that the classification model can output multiple motion unit codes based on the multiple grid regions;

[0034] Step 105: Determine the facial expression of the target face using the multiple motion unit codes and the mapping relationship between the combination of motion unit codes and facial expressions.

[0035] It should be noted that the sequence number of each step in this embodiment does not represent a limitation on the execution order of each step.

[0036] It is understood that, in specific implementations, the facial expression recognition method can be implemented using software programs, which run within a processor integrated into a chip or chip module. This method can also be implemented using a combination of software and hardware; this application does not impose any restrictions.

[0037] In this embodiment, the face image to be identified can be an image or a frame from a video. The face image includes the target face. The face image may also include other objects besides the target face, specifically the background in the image; however, this embodiment of the invention does not impose any limitations on this.

[0038] In the specific implementation of step 102, key point detection can be performed on the target face in the face image to obtain multiple facial key points of the target face. Facial key points are key feature points of the face, such as eyes, nose tip, corners of the mouth, eyebrows, and contour points of various facial components. Specifically, a facial key point detection model constructed using a neural network can be used to detect multiple facial key points. The face image is input into the facial key point detection model, which outputs multiple facial key points.

[0039] In a specific application scenario, please refer to Figure 2 The number of facial landmarks can be 68.

[0040] It should be noted that the number of facial key points can be adaptively adjusted according to the actual application environment. For example, it can be 49 or 21, etc. This application does not limit this.

[0041] In the specific implementation of step 103, some key points from multiple key points can be connected to form multiple grid regions. Each grid region covers a functional region, which refers to the area of ​​the face that affects facial expressions, such as the eyes, nose, mouth, cheeks, upper eyelids, and chin. The number of grid regions is the same as the number of functional regions. For example, when there are six functional regions, there are also six grid regions.

[0042] In a specific application scenario, please refer to Figure 3 , Figure 3 The black shaded area shown is a grid area that covers the chin of the face. This grid area is formed by connecting key points 7, 8, 9, 10, 11, 56, 57, 58, 59, and 60.

[0043] Understandable, Figure 3Only one grid area covering the chin is shown. In practical applications, multiple grid areas can be formed for each face image, covering all functional areas.

[0044] Furthermore, in the specific implementation of step 104, the grid region and the target face can be input into the classification model, which can be pre-trained. The classification model can output corresponding multiple motion unit codes. Specifically, the grid region and the target face input into the classification model can be in image format. For example, the grid region can be the image portion of the corresponding functional region in a face image, and the target face can be the image portion of a face image containing only the face, also known as a face image patch.

[0045] In practice, the motion unit code indicates a motion unit, and each motion unit can represent a movement of a facial part. For example, motion unit code AU0 represents raising the muscles of the upper lip and philtrum area, AU1 represents lowering the jaw, AU2 represents stretching the corners of the mouth, AU3 represents lowering and gathering the eyebrows, AU4 represents pulling the corners of the mouth downwards, and AU5 represents raising the outer corners of the eyebrows.

[0046] For more detailed implementations of motion unit encoding, please refer to existing technologies; they will not be elaborated here.

[0047] Furthermore, different combinations of motion unit codes can represent different facial expressions. Therefore, in the specific implementation of step 105, the facial expression of the target face can be determined based on multiple motion unit codes. Specifically, a mapping between each basic facial expression and each motion unit can be pre-established. After the classification model outputs the motion unit codes, the corresponding expression can be determined based on this mapping relationship.

[0048] This invention, through its embodiments, inputs the grid regions in a face image that affect facial expressions into a classification model. This allows the classification model to identify motion unit codes based solely on these grid regions, avoiding interference from other irrelevant regions in the face image. This ensures the accuracy of motion unit code prediction and, consequently, the accuracy of facial expression recognition.

[0049] In a non-limiting embodiment, when forming the grid region, adjacent key points among the plurality of key points are connected to obtain a plurality of Regions of Interest (ROIs); at least one ROI located within the same functional area is merged to obtain the grid region.

[0050] In this embodiment, regions of interest (ROIs) can be formed by connecting adjacent key points. A ROI is a smaller area than the grid region. By dividing ROIs and merging them into grid regions, the grid region division can be made more precise.

[0051] Furthermore, when forming the region of interest, a key point located at the contour position of the target face is selected as the starting point; starting from the starting point, adjacent key points are connected sequentially in the order from the contour position to the center position of the target face.

[0052] Continue to refer to Figure 2 and Figure 3 Starting with keypoint 7, connect keypoints 7 and 60, and then connect keypoints 8 and 59. This forms a region of interest (ROI) A. Similarly, connect keypoints 9 and 58 to form ROI B; connect keypoints 10 and 57 to form ROI C; and connect keypoints 11 and 56 to form ROI D. ROIs A, B, C, and D are then merged to form a grid area covering the chin.

[0053] Furthermore, if the distance between adjacent key points is greater than a preset threshold, at least one intermediate key point is selected between the adjacent key points; the multiple key points and the adjacent key points among the selected intermediate key points are connected.

[0054] In this embodiment of the invention, by selecting intermediate key points and connecting them with other key points, the area of ​​the formed region of interest can be minimized. Since the grid region is obtained by merging multiple regions of interest, a smaller area of ​​the region of interest ensures the accuracy of the grid region determination.

[0055] Please refer to Figure 4Starting with keypoint 7, since the distance between keypoint 7 and keypoint 60 is relatively large, an intermediate keypoint A1 can be selected between keypoints 7 and 60. Similarly, a keypoint A2 is selected between keypoints 8 and 59, a keypoint A3 between keypoints 9 and 58, a keypoint A4 between keypoints 10 and 57, and a keypoint A5 between keypoints 11 and 56. Connecting keypoints 7, A1, A2, and 8 forms region of interest e; correspondingly, keypoints A2, 8, 9, and A3 form region of interest f; and so on, forming regions of interest g, h, a, b, c, and d respectively. Merging regions of interest e, f, g, h, a, b, c, and d forms a grid area covering the chin.

[0056] In a non-limiting embodiment, Figure 1 Step 104 may include the following steps: determining the minimum bounding rectangle region of each grid region, and inputting the minimum bounding rectangle region and the target face into the classification model.

[0057] like Figure 3 and Figure 4 As shown, the grid region formed in this embodiment of the invention may be an irregular shape. However, the classification model requires the input image to be a regular shape, such as a rectangle. Therefore, the minimum bounding rectangle region of the grid region can be determined, and the minimum bounding rectangle region and the target face are input into the classification model.

[0058] Specifically, the smallest bounding rectangle region can also be an image format.

[0059] In a non-limiting embodiment, Figure 1 The steps following step 101 may include: performing face recognition on the face image to be recognized; extracting the region of the recognized face in the face image to be recognized as the face image block.

[0060] In this embodiment, a face image patch refers to the portion of the target face within a face image. The input to the classification model is this face image patch, rather than the entire face image, thereby avoiding interference from non-face portions of the face image and ensuring the accuracy of expression recognition.

[0061] Furthermore, using multiple grid regions and face image patches as input to the classification model allows the model to focus its attention on the grid regions when predicting motion units, while non-grid regions in the face image patches do not participate in the prediction of motion units, thus avoiding interference from non-grid regions. At the same time, the above attention mechanism can also improve the accuracy of prediction, thereby ensuring the accuracy of expression recognition.

[0062] In a non-limiting embodiment, the classification model is trained as follows: the classification model is constructed using a neural network; a training sample set is constructed, the training sample set including face images, grid regions in the face images, and facial expressions corresponding to the face images; the classification model is trained using the training sample set to obtain a trained classification model.

[0063] In this embodiment, the classification model can be constructed using a neural network algorithm. The training sample set can include face images, grid regions within the face images, and facial expressions corresponding to the face images, which can be pre-annotated. The classification model obtained after training the classification model using the training sample set can then be used for online facial expression recognition.

[0064] In a specific application scenario of this invention, embodiments of the invention can be used to perform facial expression recognition on passengers inside a vehicle to achieve interaction between the vehicle and the passenger. Specifically, a camera device is installed inside the vehicle to capture facial images or videos of passengers. The vehicle's control system has an facial expression recognition function, meaning it can use the technical solution of this application to perform facial expression recognition on the captured facial images, obtain the passenger's expression, and then respond according to the passenger's current mood. For example, if the passenger's expression is determined to be sad by capturing a picture of the passenger, soothing music can be played automatically.

[0065] In another specific application scenario of this invention, the embodiments of this invention can be used in situations where it is necessary to determine a user's true mood, such as assisting a lie detector in determining whether a user is lying, or assisting a doctor in a psychological clinic in determining a user's true state of mind. Since a user's micro-expressions can reflect their true state of mind, recognizing facial expressions through user facial images can be used to assist in determining their true mood.

[0066] In another specific application scenario of this invention, embodiments of this invention can be used for real-time facial expression generation of virtual avatars. Specifically, an application on a terminal device can obtain a user's facial expression by executing the steps of the method described in this application, and then generate a virtual avatar corresponding to that expression, thereby improving the user experience.

[0067] Please refer to Figure 5 This invention also discloses a facial expression recognition device. The facial expression recognition device 50 may include:

[0068] The acquisition module 501 is used to acquire a face image to be identified, wherein the face image includes the target face;

[0069] The detection module 502 is used to perform key point detection on the target face to obtain multiple facial key points;

[0070] The grid region forming module 503 is used to connect the multiple key points to form multiple grid regions based on the functional regions in the target face. The functional regions are the areas in the face that affect facial expressions, and each grid region covers one functional region.

[0071] The classification module 504 is used to input the multiple grid regions and the target face into the trained classification model, so that the classification model can output multiple motion unit codes based on the multiple grid regions;

[0072] The mapping module 505 is used to determine the facial expression of the target face by utilizing the multiple motion unit codes and the mapping relationship between the motion unit code combination and facial expressions.

[0073] This invention, through its embodiments, inputs the grid regions in a face image that affect facial expressions into a classification model. This allows the classification model to identify motion unit codes based solely on these grid regions, avoiding interference from other irrelevant regions in the face image. This ensures the accuracy of motion unit code prediction and, consequently, the accuracy of facial expression recognition.

[0074] For more information on the working principle and operation mode of the facial expression recognition device 50, please refer to [link / reference needed]. Figures 1 to 4 The relevant descriptions in the text will not be repeated here.

[0075] In specific implementations, the aforementioned facial expression recognition device may correspond to a chip in a terminal device that has facial expression recognition functionality, such as a System-On-a-Chip (SOC), a baseband chip, etc.; or it may correspond to a chip module in a terminal device that includes a facial expression recognition function; or it may correspond to a chip module with a data processing function; or it may correspond to a terminal device.

[0076] Regarding the modules / units included in the various devices and products described in the above embodiments, they can be software modules / units, hardware modules / units, or a combination of both. For example, for various devices and products applied to or integrated into a chip, all of their modules / units can be implemented using hardware methods such as circuits, or at least some modules / units can be implemented using software programs that run on a processor integrated within the chip, while the remaining (if any) modules / units can be implemented using hardware methods such as circuits; for various devices and products applied to or integrated into a chip module, all of their modules / units can be implemented using hardware methods such as circuits, and different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or different components of the chip module, or at least some modules / units can be implemented using hardware methods such as circuits. The components can be implemented using software programs that run on the processor integrated within the chip module. The remaining (if any) modules / units can be implemented using hardware methods such as circuits. For various devices and products applied to or integrated into the terminal, each of its components / units can be implemented using hardware methods such as circuits. Different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or in different components within the terminal. Alternatively, at least some modules / units can be implemented using software programs that run on the processor integrated within the terminal, while the remaining (if any) modules / units can be implemented using hardware methods such as circuits.

[0077] This invention also discloses a storage medium, which is a computer-readable storage medium storing a computer program thereon. When the computer program is executed, it can perform the steps of the aforementioned facial expression recognition method. The storage medium may include ROM, RAM, a disk, or an optical disk, etc. The storage medium may also include non-volatile memory or non-transitory memory, etc.

[0078] This invention also discloses a terminal device, which may include a memory and a processor. The memory stores a computer program that can run on the processor. When the processor runs the computer program, it can execute the steps of the aforementioned facial expression recognition method. The user device includes, but is not limited to, terminal devices such as mobile phones, computers, and tablet computers.

[0079] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article indicates that the preceding and following related objects have an "or" relationship.

[0080] In the embodiments of this application, "multiple" refers to two or more.

[0081] The descriptions of "first," "second," etc., appearing in the embodiments of this application are for illustrative purposes and to distinguish the objects being described. They have no order and do not indicate any special limitation on the number of devices in the embodiments of this application, nor do they constitute any limitation on the embodiments of this application.

[0082] In this application, the term "connection" refers to various connection methods, such as direct connection or indirect connection, to achieve communication between devices. This application does not impose any limitations on this.

[0083] It should be understood that in the embodiments of this application, the processor can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0084] It should also be understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0085] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0086] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0087] In the several embodiments provided in this application, it should be understood that the disclosed methods, apparatuses, and systems can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for example, the division of units is merely a logical functional division, and other division methods may exist in actual implementation; for example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0088] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0089] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can be physically comprised separately, or two or more units can be integrated into one unit. The integrated unit described above can be implemented in hardware or in the form of hardware plus software functional units.

[0090] The integrated unit implemented as a software functional unit described above can be stored in a computer-readable storage medium. This software functional unit, stored in a storage medium, includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute some steps of the methods described in the various embodiments of the present invention.

[0091] While the present invention has been disclosed above, it is not limited thereto. Any person skilled in the art can make various modifications and alterations without departing from the spirit and scope of the invention; therefore, the scope of protection of the present invention should be determined by the scope defined in the claims.

Claims

1. A facial expression recognition method, characterized in that, include: Acquire a face image to be identified, wherein the face image includes the target face; Key point detection is performed on the target face to obtain multiple facial key points; The multiple key points are connected to form multiple grid regions based on the functional regions in the target face. The functional regions are the areas in the face that affect facial expressions, and each grid region covers one functional region. The classification model is trained by inputting the multiple grid regions and the target face into the classification model, so that the classification model can output multiple motion unit codes based on the multiple grid regions; The facial expression of the target face is determined by utilizing the multiple motion unit codes and the mapping relationship between the combination of motion unit codes and facial expressions; The step of connecting the multiple key points to form multiple grid regions based on the functional regions in the target face includes: connecting adjacent key points among the multiple key points to obtain multiple regions of interest; At least one region of interest located within the same functional area is merged to obtain the grid region; The step of connecting adjacent key points among the plurality of key points includes: If the distance between adjacent key points is greater than a preset threshold, then at least one intermediate key point is selected between the adjacent key points. Connect intermediate key points to other key points.

2. The facial expression recognition method according to claim 1, characterized in that, The step of connecting adjacent key points among the plurality of key points includes: Select key points located at the contour position of the target face as starting position points; Starting from the initial position, connect adjacent key points sequentially in order from the outline position to the center position of the target face.

3. The facial expression recognition method according to claim 1, characterized in that, The classification model trained using the multiple grid regions and the target face input includes: Determine the minimum bounding rectangle region for each grid region, and input the minimum bounding rectangle region and the target face into the classification model.

4. The facial expression recognition method according to claim 1, characterized in that, The target face is a face image patch, and the process of acquiring the face image to be identified further includes: Perform face recognition on the face image to be identified; The region of the identified face in the face image to be identified is extracted as the face image block.

5. The facial expression recognition method according to claim 1, characterized in that, The classification model is trained using the following method: The classification model is constructed using a neural network; Construct a training sample set, which includes face images, grid regions in the face images, and facial expressions corresponding to the face images; The classification model is trained using the training sample set to obtain the trained classification model.

6. A facial expression recognition device, characterized in that, include: The acquisition module is used to acquire a face image to be identified, wherein the face image includes the target face; The detection module is used to perform key point detection on the target face to obtain multiple facial key points; The grid region forming module is used to connect multiple key points to form multiple grid regions based on the functional regions in the target face. The functional regions are the areas in the face that affect facial expressions, and each grid region covers one functional region. The classification module is used to input the multiple grid regions and the target face into the trained classification model, so that the classification model can output multiple motion unit codes based on the multiple grid regions; The mapping module is used to determine the facial expression of the target face by utilizing the multiple motion unit codes and the mapping relationship between the combination of motion unit codes and facial expressions; The step of connecting the multiple key points to form multiple grid regions based on the functional regions in the target face includes: connecting adjacent key points among the multiple key points to obtain multiple regions of interest; At least one region of interest located within the same functional area is merged to obtain the grid region; The step of connecting adjacent key points among the plurality of key points includes: If the distance between adjacent key points is greater than a preset threshold, then at least one intermediate key point is selected between the adjacent key points. Connect intermediate key points to other key points.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program, when run by a processor, performs the steps of the facial expression recognition method according to any one of claims 1 to 5.

8. A terminal device, comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, characterized in that, When the processor runs the computer program, it performs the steps of the facial expression recognition method according to any one of claims 1 to 5.

Images