A method, device, storage medium and electronic device for training a face recognition model

Abstract

This application relates to the field of image recognition technology, specifically providing a method, apparatus, storage medium, and electronic device for training a facial recognition model. The method may include: inputting an image dataset for training into a backbone network module and outputting image feature information corresponding to the image dataset; inputting the image feature information into a feature processing module and outputting image fusion features of different sizes; inputting the image fusion features of different sizes into a target detection module and outputting image detection results; optimizing an initial facial recognition model using the loss value of the image detection results determined by a target loss function to obtain a trained facial recognition model; the facial recognition model is used to recognize facial expressions in the image to be recognized. Some embodiments of this application can improve the accuracy of the trained facial recognition model.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, and more specifically, to a method, apparatus, storage medium, and electronic device for training a facial recognition model. Background Technology

[0002] Facial Expression Recognition (FAR) serves as a bridge between facial feature analysis and high-level emotion understanding.

[0003] Currently, existing facial expression recognition technologies primarily rely on models based on convolutional neural networks (CNNs) or Transformers. These techniques can capture changes in facial features to a certain extent, thereby analyzing and identifying different emotional states. However, due to the limitations of local receptive fields in capturing long-range information, CNNs struggle to effectively capture such information in certain situations, potentially leading to poor results in tasks such as facial image segmentation. On the other hand, Transformers excel in global modeling and can effectively capture long-range dependencies, but their self-attention mechanism is complex when dealing with large image sizes, potentially posing challenges, especially in tasks such as ultra-high-resolution image detection and small object detection.

[0004] Therefore, how to provide an accurate and efficient facial expression recognition solution has become an urgent technical problem to be solved. Summary of the Invention

[0005] The purpose of some embodiments of this application is to provide a method, apparatus, storage medium, and electronic device for training a facial recognition model. The technical solutions of the embodiments of this application can improve the accuracy of training the facial recognition model, thereby achieving high accuracy and efficiency in facial expression recognition, and are highly practical.

[0006] In a first aspect, some embodiments of this application provide a method for training a facial recognition model. This method trains an initial facial recognition model to obtain a trained facial recognition model. The initial facial recognition model includes a backbone network module, a feature processing module, and a target detection module. The method includes: inputting an image dataset for training into the backbone network module and outputting image feature information corresponding to the image dataset; inputting the image feature information into the feature processing module and outputting image fusion features of different sizes; inputting the image fusion features of different sizes into the target detection module and outputting image detection results; optimizing the initial facial recognition model using the loss value of the image detection results determined by a target loss function to obtain the trained facial recognition model. The target loss function includes: facial expression classification loss, facial detection and localization loss, target confidence loss, and temporal consistency loss. The facial recognition model is used to recognize facial expressions in the image to be recognized.

[0007] Some embodiments of this application train the initial facial model by inputting the image dataset into the backbone network module, feature processing module, and object detection module of the initial facial model to obtain image detection results; the initial facial model is optimized by calculating the loss value of the image detection results through the target loss function to obtain a trained facial recognition model, so as to achieve accurate recognition of facial expressions in the image to be recognized. The training effect of the facial recognition model is high and the recognition efficiency is high.

[0008] In some embodiments, the image dataset used for training is continuously updated and obtained by: training the initial face recognition model using the i-th training dataset to obtain the training result of the i-th face recognition model; where i is a positive integer; confirming that the loss value of the training result of the i-th face recognition model is greater than a set threshold, or i≤N, where N is the number of iterations; updating the i-th training dataset based on the training result of the i-th face recognition model to obtain the (i+1)-th training dataset, where the (i+1)-th training dataset is the image dataset.

[0009] Some embodiments of this application improve the accuracy of the training dataset and thus enhance the training effect of the facial recognition model by filtering and updating the training dataset each time during the training process.

[0010] In some embodiments, when i=1, the first training dataset is obtained by cropping the original facial expression dataset to obtain a standard dataset; performing data augmentation on the expression categories in the standard dataset to obtain the first training dataset; wherein, the data augmentation includes mirroring, random region cropping, and random noise addition on the facial expression images in the standard dataset.

[0011] Some embodiments of this application obtain a first training dataset by cropping and enhancing the original facial expression dataset, thereby increasing the richness of the dataset.

[0012] In some embodiments, the backbone network module includes: a preprocessing module, multiple feature extraction modules, and multiple feature fusion modules; wherein, the step of inputting the image dataset used for training into the backbone network module and outputting image feature information corresponding to the image dataset includes: the preprocessing module extracting features from the images in the image dataset to obtain facial features; the multiple feature extraction modules acquiring the temporal dynamic information corresponding to the facial features and constructing a facial feature temporal sequence; and the multiple feature fusion modules mapping the facial feature temporal sequence to different dimensions to obtain the image feature information.

[0013] Some embodiments of this application process the image dataset sequentially through the preprocessing module, feature preprocessing module, and feature fusion module in the backbone network module to obtain image feature information, thereby achieving effective data processing and providing data support for subsequent model training.

[0014] In some embodiments, the step of inputting the image feature information into the feature processing module and outputting image fusion features of different sizes includes: the feature processing module performing sampling and convolution operations on the image feature information in different dimensions to obtain the image fusion features of different sizes.

[0015] Some embodiments of this application use a feature processing module to process image feature information of different dimensions to obtain image fusion features of different sizes, which can provide data support for subsequent model training.

[0016] In some embodiments, the step of inputting the image fusion features of different sizes into the target detection module and outputting the image detection result includes: inputting the image fusion features of different sizes into the target detection layer in the target detection module respectively to obtain the image detection result, wherein the target detection layer of different sizes is used to detect image fusion features of different sizes.

[0017] In some embodiments, the target loss function is constructed as follows: obtaining the respective loss parameters and weights corresponding to the facial expression classification loss, the face detection and localization loss, the target confidence loss, and the temporal consistency loss; performing a weighted summation of the respective loss parameters and weights to obtain the target loss function; the loss value of the image detection result is obtained as follows: calculating different losses on the image detection result to obtain the respective loss values ​​corresponding to the facial expression classification loss, the face detection and localization loss, the target confidence loss, and the temporal consistency loss; and inputting the respective loss values ​​as the respective loss parameters into the target loss function to obtain the loss value.

[0018] Some embodiments of this application calculate the loss value of image detection results using a target loss function composed of four parts, which can improve the accuracy and effect of model training and ensure that the recognition accuracy of the subsequently trained facial recognition model is high.

[0019] Secondly, some embodiments of this application provide an apparatus for training a facial recognition model. This apparatus is used to train an initial facial recognition model to obtain a trained facial recognition model. The initial facial recognition model includes a backbone network module, a feature processing module, and a target detection module. The apparatus includes: an extraction module for inputting an image dataset for training into the backbone network module and outputting image feature information corresponding to the image dataset; a fusion module for inputting the image feature information into the feature processing module and outputting image fusion features of different sizes; a processing module for inputting the image fusion features of different sizes into the target detection module and outputting image detection results; and an optimization module for optimizing the initial facial recognition model using a loss value determined by a target loss function to obtain the trained facial recognition model. The target loss function includes: facial expression classification loss, facial detection and localization loss, target confidence loss, and temporal consistency loss. The facial recognition model is used to recognize facial expressions in the image to be recognized.

[0020] Thirdly, some embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the method described in any embodiment of the first aspect.

[0021] Fourthly, some embodiments of this application provide an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, can implement the method as described in any embodiment of the first aspect.

[0022] Fifthly, some embodiments of this application provide a computer program product, the computer program product including a computer program, wherein the computer program, when executed by a processor, can implement the method described in any embodiment of the first aspect. Attached Figure Description

[0023] To more clearly illustrate the technical solutions of some embodiments of this application, the accompanying drawings used in some embodiments of this application will be briefly described below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 System diagrams for training facial recognition models provided for some embodiments of this application;

[0025] Figure 2 Flowcharts of methods for training facial recognition models provided for some embodiments of this application;

[0026] Figure 3 Block diagrams of apparatus for training facial recognition models provided for some embodiments of this application;

[0027] Figure 4 A schematic diagram of an electronic device provided for some embodiments of this application. Detailed Implementation

[0028] The technical solutions of some embodiments of this application will now be described with reference to the accompanying drawings.

[0029] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0030] In related technologies, facial expression recognition plays a crucial role in human-computer interaction tasks such as emotion analysis, user behavior prediction, and emotional robotics. In the field of mental health, it provides strong technical support for applications such as autism diagnosis and mood disorder treatment. Furthermore, facial expression recognition is widely used in multimedia processing tasks such as video surveillance, driver fatigue detection, and virtual reality. Therefore, systematically studying facial expression recognition algorithms has profound application significance, not only improving the naturalness and accuracy of human-computer interaction but also providing a solid foundation for emotional intelligence in multiple fields. Currently, CNN models extract local and global features from facial images through multiple convolutional and pooling layers. These features include the shape, position, and texture information of key parts such as the eyes, mouth, and eyebrows, and their changing patterns under different emotional states provide the basis for model recognition. However, when processing complex facial expressions, CNN models may be affected by external factors such as lighting, occlusion, and angle changes, leading to a decrease in recognition accuracy. Transformer models utilize a self-attention mechanism to capture the correlation information between different regions in a facial image. This mechanism treats facial images as a sequence of pixels or feature vectors, and delves into potential facial expression features by calculating the influence weights of each element within the sequence on all other elements. The Transformer model demonstrates significant advantages in handling features with long-range dependencies. However, this approach also comes with high computational complexity and a large model parameter size, which can lead to challenges during training, including extended training time and significant demands on computational resources, thus increasing resource consumption and implementation difficulty.

[0031] In view of this, some embodiments of this application provide a method for training a facial recognition model. In this method, a pre-designed initial facial recognition model is trained using an image dataset to obtain image detection results. Then, a target loss function with four loss components, constructed according to this application, is used to calculate the loss on the image detection results. The obtained loss value is then used to optimize the initial facial recognition model, thereby obtaining a well-trained facial recognition model with higher accuracy. By training the pre-designed initial facial recognition model and optimizing it using the constructed target loss function, some embodiments of this application can achieve comprehensive training and tuning of the initial facial recognition model, improving the training effect and robustness. Subsequently, facial expression recognition can be performed using the trained facial recognition model, achieving higher accuracy and efficiency, thus improving the overall effect of facial expression recognition.

[0032] The following is in conjunction with the appendix Figure 1 The overall structure of a system for training a facial recognition model, provided by some embodiments of this application, is illustrated by way of example.

[0033] like Figure 1 As shown in the diagram, some embodiments of this application provide a system diagram for training a facial recognition model. This system may include a terminal 100 and a server 200. The terminal 100 pre-prepares an image dataset for model training, and the server 200 pre-deploys an initial facial recognition model. When the terminal 100 initiates model training, it sends the image dataset to the server 200. The server 200 uses the image dataset to train the initial facial recognition model multiple times until training stops, at which point it outputs the trained facial recognition model. This facial recognition model is deployed on the server 200 and can subsequently perform facial expression recognition on images transmitted from the terminal 100.

[0034] In some embodiments of this application, the initial facial recognition model may include a backbone network module, a feature processing module, and an object detection module. The internal component structure of the initial facial recognition model can be adjusted as needed. The terminal 100 can be a mobile terminal or a non-portable computer terminal; this application does not specifically limit the type of terminal.

[0035] The following is in conjunction with the appendix Figure 2 The present application provides an exemplary embodiment of the implementation process of training a facial recognition model by a server 200.

[0036] Please see the appendix Figure 2 , Figure 2 This document provides a flowchart of a method for training a facial recognition model, as illustrated in some embodiments of this application. Before executing the method for training the facial recognition model as described in steps S210 to S240 below, it is necessary to prepare an image dataset for training. Therefore, the process of obtaining the image dataset is first exemplarily described below.

[0037] In some embodiments of this application, the image dataset used for training is continuously updated and obtained through the following method:

[0038] S201, use the i-th training dataset to train the i-th initial face recognition model to obtain the i-th face recognition model training result; where i is a positive integer;

[0039] For example, in the model training process of this application embodiment, the model needs to be trained iteratively multiple times. The training dataset used in each training process is updated to improve the accuracy of model training. For example, the i-th training dataset is used to train the i-th initial facial model to obtain the training result. That is, when i=1, the first training dataset is used to train the first initial facial model to obtain the first facial recognition model training result. The first initial facial model is then optimized using the target loss function in S240 based on the first facial recognition model training result to obtain the second initial facial model (that is, the (i+1)-th initial facial model).

[0040] In some embodiments of this application, when i=1, the first training dataset is obtained by the following method: cropping the collected original facial expression dataset to obtain a standard dataset; performing data augmentation on the expression categories in the standard dataset to obtain the first training dataset; wherein, the data augmentation includes: mirroring, randomly cropping regions, and adding random noise to the facial expression images in the standard dataset.

[0041] For example, in this embodiment, the collected original facial expression dataset has problems such as inconsistent specifications and sizes, and complex backgrounds. To ensure the uniformity of the input samples for the initial facial recognition model, the original facial expression dataset needs to be preprocessed. For example, the OpenCV face detection tool (haarcascade_frontalface_alt.xml) is initially used to detect faces and extract face regions from all images in the original facial expression dataset. The face data is then cropped and standardized to a size of 128×128 pixels to obtain a standard dataset to adapt to the input requirements of subsequent models. For expression categories with relatively small amounts of data in the standard dataset, the following data augmentation techniques are used to expand the dataset to balance the category distribution: mirror flipping: horizontally flipping the images to increase data samples; random cropping: randomly selecting regions from the cropped images for adjustment to enhance data diversity; adding noise: adding random noise to the images to simulate image quality changes under complex scenes, etc. The first training dataset for training is obtained through the above operations. It should be understood that the processing method of the original facial expression dataset can be adjusted, and this embodiment is not limited to this.

[0042] S202, confirm that the loss value of the i-th face recognition model training result is greater than the set threshold, or i≤N, where N is the number of iterations;

[0043] For example, in this embodiment, if the loss value of the i-th face recognition model training result does not meet the set threshold condition or the iteration number has not been reached, the i-th training dataset needs to be updated. If the loss value of the i-th face recognition model training result meets the set threshold condition, the i-th initial face model is used as the trained face recognition model. The set threshold can be A; a loss value meeting the set threshold condition means the loss value is less than or equal to A, and conversely, a loss value not meeting the set threshold condition means the loss value is greater than A. The value of A or N can be set according to the actual application scenario, and this application is an embodiment and is not specifically limited here.

[0044] S203, based on the training result of the i-th face recognition model, update the i-th training dataset to obtain the (i+1)-th training dataset, wherein the (i+1)-th training dataset is the image dataset.

[0045] For example, in this embodiment, the iteration phase can be divided into two iteration phases according to the model training phase: the first iteration and the second iteration, and a dual-layer iteration strategy can be used for further optimization. It should be noted that the update methods for the training dataset differ between these two iterations. In the first iteration, based on the i-th face recognition model training result, misclassified or low-confidence training data samples are automatically selected, re-labeled, and added to the (i+1)-th training dataset for continued training and model fine-tuning. In the second iteration, new sample data is generated using the pre-trained model. After selecting high-confidence data from the i-th training dataset, it is combined with the new sample data to form the (i+1)-th training dataset, expanding the dataset size and optimizing the global feature extraction capability of the face recognition model.

[0046] To improve the quality of the training dataset, this application employs iterative semi-supervised annotation, where high-quality labeled data is the prerequisite and foundation for automatic facial expression recognition. To address this issue, an iterative semi-supervised annotation strategy (i.e., the first and second layer iterations mentioned above) is used. A pre-trained model is introduced during the annotation process, combining model predictions with a small amount of manual annotation. Through continuous iteration, the annotation of a large number of facial expressions is achieved, thus continuously updating the training dataset during training. In the second layer iteration, based on the recognized image detection results, training images (i.e., the training dataset) are added for fine-tuning. The goal of this stage is to increase the accuracy of future facial expression recognition using a wider range of training images.

[0047] Since this application iteratively updates the training image dataset, the training process from S210 to S240 needs to be executed for each i-th training dataset until the loss value of the model trained in a certain iteration is less than or equal to a set threshold, at which point the training ends. The following describes the training process of an initial face recognition model using a specific training dataset.

[0048] In some embodiments of this application, the method for training a facial recognition model may include:

[0049] S210, the image dataset used for training is input into the backbone network module, and the image feature information corresponding to the image dataset is output.

[0050] For example, in some embodiments of this application, the i-th training dataset is input into the backbone network module of the i-th initial face recognition model, and the backbone network module performs feature extraction and processing on the images in the i-th training dataset to obtain image feature information.

[0051] In some embodiments of this application, the backbone network module includes: a preprocessing module, multiple feature extraction modules, and multiple feature fusion modules; S210 may include: the preprocessing module extracting features from the images in the image dataset to obtain facial features; the multiple feature extraction modules acquiring the temporal dynamic information corresponding to the facial features and constructing a facial feature temporal sequence; the multiple feature fusion modules mapping the facial feature temporal sequence to different dimensions to obtain the image feature information.

[0052] For example, in this embodiment, the feature extraction module in the backbone network module can use an ODSSBblock (Orthogonal Dynamic State Space Block) module or other types of feature processing modules, and the feature fusion module can use a VCM (Vision Clue Merge) module or other modules for fusing features. Specifically, the preprocessing module can first extract features (e.g., facial features) from the image in the i-th training dataset using a feature extraction algorithm to obtain facial features. Then, the feature extraction module can obtain the temporal information of these facial features and construct a temporal sequence of facial features according to their chronological order. Finally, the feature fusion module maps the temporal sequence of facial features to different spatial or temporal dimensions to obtain image feature information. This enhances local feature representation and subsequent detection accuracy.

[0053] S220, the image feature information is input into the feature processing module, and image fusion features of different sizes are output.

[0054] For example, in some embodiments of this application, the feature processing module can process image feature information of different dimensions to obtain image fusion features of different sizes.

[0055] For example, in some embodiments of this application, S220 may include: the feature processing module performs sampling and convolution operations on the image feature information in different dimensions to obtain the image fusion features of different sizes.

[0056] For example, in some embodiments of this application, the feature processing module can perform upsampling and / or downsampling operations on image feature information of different dimensions, and convolve the sampling results with the sampling results of other dimensions to obtain image fusion features. For example, if there is image feature information of three dimensions, the result of downsampling the image feature information of the first dimension is convolved with the result of downsampling the image feature information of the second dimension to obtain image fusion features at a set size. For example, the image feature information of the first dimension can also be downsampled directly to output image fusion features of a set size. The set size can be arbitrarily set according to actual needs, such as 80×80, 40×40, etc. It is understood that which one, two, or three of the upsampling, downsampling, and convolution operations are performed on the image feature information can be flexibly set according to the actual application scenario, and this application embodiment does not make specific limitations here.

[0057] S230, the image fusion features of different sizes are input into the target detection module, and the image detection result is output.

[0058] For example, in some embodiments of this application, the target detection module includes a sub-module for detecting image fusion features of different sizes, with one sub-module corresponding to each size, so as to obtain image detection results for image fusion features of different sizes.

[0059] In some embodiments of this application, S230 may include: inputting the image fusion features of different sizes into the target detection layer in the target detection module to obtain the image detection result, wherein the target detection layer of different sizes is used to detect image fusion features of different sizes.

[0060] For example, in some embodiments of this application, image detection results of the corresponding size are obtained by inputting the fusion features of images of different sizes into a sub-module of the corresponding size (as a specific example of an object detection layer).

[0061] S240, the initial face recognition model is optimized using the loss value of the image detection result determined by the target loss function to obtain the trained face recognition model. The target loss function includes: facial expression classification loss, face detection and localization loss, target confidence loss, and temporal consistency loss. The face recognition model is used to recognize facial expressions in the image to be recognized.

[0062] For example, in this embodiment, to reflect the accuracy of model training, a target loss function containing multiple losses is designed. The target loss function calculates the loss between the image detection results and the facial recognition samples in the training dataset, obtaining a loss value. This loss value is then used to optimize the initial facial recognition model for the i-th iteration. This process is repeated until a well-trained facial recognition model that meets a set threshold condition is obtained. In the application stage of the facial recognition model, the image to be recognized can be input into the model, and the facial expression recognition result can be output; for example, the facial expression recognition result can be categorized as happy, sad, or melancholic.

[0063] To achieve higher accuracy in model optimization, in some embodiments of this application, the target loss function is constructed as follows: obtaining the respective loss parameters and weights corresponding to the facial expression classification loss, the facial detection and localization loss, the target confidence loss, and the temporal consistency loss; and performing a weighted summation of the respective loss parameters and weights to obtain the target loss function.

[0064] For example, in some embodiments of this application, the target loss function consists of four parts.

[0065] The first part is the facial expression classification loss L. cls It can use Cross-Entropy Loss or Focal Loss to handle the class imbalance problem. In facial expression multi-class classification loss... cls The following formula can be used for calculation:

[0066]

[0067] Where N is the number of samples in the training dataset (usually the number of pixels, feature points, or other spatial dimensions in a video frame), C is the number of facial expression categories, and y i,c Let i be the real facial expression label of the i-th sample. To predict probabilities.

[0068] The second part is the face detection and localization loss L. loc It can use the CIoU loss of YOLOv8 for face detection tasks, and the calculation formula is as follows:

[0069] Lloc =1-CIoU(b p b t )

[0070] Among them, b p b is the predicted bounding box in face detection. t The CIoU() function is used to calculate b, which is the pre-annotated ground truth bounding box in face detection. p and b t Similarity between them.

[0071] The third part is the target confidence loss L. obj This is used to measure whether the predicted bounding box contains a valid facial expression target (i.e., the difference between the confidence score of the predicted bounding box and the true expression label), and the calculation formula is as follows:

[0072]

[0073] Among them, y i y is the ground truth label for the i-th sample, taking a value of 0 or 1. i =1 indicates that the sample is a positive sample (i.e., it contains the target); y i =0 indicates that the sample is a negative sample (i.e., it does not contain the target). Let be the confidence score for the i-th sample, representing the probability that the model believes the sample contains the target; its value ranges from [0,1]. The log() function is used to calculate the difference between the predicted probability and the true label.

[0074] Part Four is the time consistency loss L time It can be used to add temporal consistency loss to video or continuous frame expression recognition tasks in the training dataset, ensuring the smoothness of the model's predictions for consecutive frames. It is used to measure the difference between the model's outputs at adjacent time steps, and its calculation formula is as follows:

[0075]

[0076] Among them, f t f represents the model's output at time step t (e.g., feature map, optical flow, depth map, etc.). t-1 This represents the model's output at time step t-1. ||f t -f t-1 || 2 To calculate f t and f t-1 The square of the Euclidean distance (L2 norm) between them represents the difference between the two.

[0077] The weighted sum of the above four losses is used as the final fusion loss function (as a specific example of the target loss function):

[0078] L total =λ loc L loc +λ cls L cls +λ obj L obj +λ time L time

[0079] Where, λ loc λ represents the loss weight for face detection and localization loss. cls The loss weights for facial expression classification loss, λ obj λ is the loss weight for the target confidence loss. time The loss weight is used to determine the time consistency loss. The value of the loss weight can be set according to the actual situation, and this application does not impose specific limitations on it.

[0080] In some embodiments of this application, S240 may include: calculating different losses on the image detection results to obtain various loss values ​​corresponding to the facial expression classification loss, the facial detection and localization loss, the target confidence loss, and the temporal consistency loss; and inputting the various loss values ​​as the respective loss parameters into the target loss function to obtain the loss value.

[0081] For example, in some embodiments of this application, the above four parts of loss are calculated by comparing the image detection results with real face detection data in the training dataset, and then input into the fusion loss function to obtain the loss value. This loss value can be used to optimize the i-th initial face recognition model, ultimately resulting in a face recognition model with higher accuracy after training.

[0082] As can be seen from the above embodiments of this application, this application can improve the accuracy and efficiency of facial recognition, and is especially suitable for facial expression recognition scenarios that require efficient and real-time processing, and has high practicality.

[0083] Please refer to Figure 3 , Figure 3 The diagram shows a block diagram of an apparatus for training a facial recognition model according to some embodiments of this application. It should be understood that this apparatus for training a facial recognition model corresponds to the method embodiments described above and is capable of performing the various steps involved in the method embodiments. The specific functions of this apparatus for training a facial recognition model can be found in the description above; detailed descriptions are omitted here to avoid repetition.

[0084] Figure 3The apparatus for training a facial recognition model includes at least one software functional module that can be stored in a memory or embedded in the apparatus in the form of software or firmware. This apparatus is used to train an initial facial recognition model to obtain a trained facial recognition model. The initial facial recognition model includes a backbone network module, a feature processing module, and a target detection module. The apparatus includes: an extraction module 310, used to input the image dataset for training into the backbone network module and output image feature information corresponding to the image dataset; a fusion module 320, used to input the image feature information into the feature processing module and output image fusion features of different sizes; a processing module 330, used to input the image fusion features of different sizes into the target detection module and output image detection results; and an optimization module 340, used to optimize the initial facial recognition model using the loss value of the image detection results determined by a target loss function to obtain the trained facial recognition model. The target loss function includes: facial expression classification loss, facial detection and localization loss, target confidence loss, and temporal consistency loss. The facial recognition model is used to recognize facial expressions in the image to be recognized.

[0085] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the device described above can be referred to the corresponding process in the aforementioned method, and will not be elaborated further here.

[0086] Some embodiments of this application also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can perform the operation of any of the methods corresponding to the methods provided in the above embodiments.

[0087] Some embodiments of this application also provide a computer program product, which includes a computer program, wherein when the computer program is executed by a processor, it can implement the operation of any of the methods corresponding to the above embodiments provided in the above embodiments.

[0088] like Figure 4 As shown, some embodiments of this application provide an electronic device 400, which includes a memory 410, a processor 420, and a computer program stored in the memory 410 and executable on the processor 420. When the processor 420 reads the program from the memory 410 via a bus 430 and executes the program, it can implement the methods of any of the above embodiments.

[0089] Processor 420 can process digital signals and may include various computing architectures. For example, it may be a complex instruction set computer architecture, a reduced instruction set computer architecture, or an architecture that implements multiple instruction set combinations. In some examples, processor 420 may be a microprocessor.

[0090] Memory 410 can be used to store instructions executed by processor 420 or data related to the execution of instructions. These instructions and / or data may include code for implementing some or all of the functions of one or more modules described in the embodiments of this application. The processor 420 of this disclosure embodiment can be used to execute instructions in memory 410 to implement the methods shown above. Memory 410 includes dynamic random access memory, static random access memory, flash memory, optical memory, or other memories well known to those skilled in the art.

[0091] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application. It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0092] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0093] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A method for training a facial recognition model, characterized in that, The method is used to train an initial facial recognition model to obtain a trained facial recognition model. The initial facial recognition model includes: a backbone network module, a feature processing module, and an object detection module. The method includes: The image dataset used for training is input into the backbone network module, which outputs image feature information corresponding to the image dataset. The image dataset is continuously updated during training based on an iterative semi-supervised annotation strategy. The iteration phase is divided into two iteration phases according to the model training phase: the first iteration and the second iteration. In the first iteration, based on the i-th face recognition model training result, misclassified or low-confidence training data samples are automatically selected, re-labeled, and added to the (i+1)-th training dataset for further fine-tuning of the model. In the second iteration, new sample data is generated using the pre-trained model, and high-confidence data is selected from the i-th training dataset and combined with the new sample data to form the (i+1)-th training dataset. The image feature information is input into the feature processing module, which outputs image fusion features of different sizes. The image fusion features of different sizes are input into the target detection module, and the image detection results are output. The initial face recognition model is optimized using the loss value of the image detection result determined by the objective loss function to obtain the trained face recognition model. The objective loss function includes: facial expression classification loss, face detection and localization loss, target confidence loss, and temporal consistency loss. The face recognition model is used to recognize facial expressions in the image to be recognized.

2. The method as described in claim 1, characterized in that, The image dataset used for training is continuously updated using the following method: The initial face recognition model is trained using the i-th training dataset to obtain the training result of the i-th face recognition model; where i is a positive integer. Confirm that the loss value of the i-th face recognition model training result is greater than a set threshold, or i ≤ N, where N is the number of iterations; The i-th training dataset is updated based on the i-th face recognition model training result to obtain the (i+1)-th training dataset, which is the image dataset.

3. The method as described in claim 2, characterized in that, When i=1, the first training dataset was obtained using the following method: The original facial expression dataset was cropped to obtain a standard dataset; Data augmentation is performed on the facial expression categories in the standard dataset to obtain the first training dataset; wherein, the data augmentation includes: mirroring, randomly cropping regions, and adding random noise to the facial expression images in the standard dataset.

4. The method according to any one of claims 1-3, characterized in that, The backbone network module includes: a preprocessing module, multiple feature extraction modules, and multiple feature fusion modules; The step of inputting the image dataset used for training into the backbone network module and outputting image feature information corresponding to the image dataset includes: The preprocessing module extracts features from the images in the image dataset to obtain facial features; The multiple feature extraction modules acquire the temporal dynamic information corresponding to the facial features and construct a temporal sequence of facial features. The multiple feature fusion modules map the facial feature time sequence to different dimensions to obtain the image feature information.

5. The method as described in claim 4, characterized in that, The step of inputting the image feature information into the feature processing module and outputting image fusion features of different sizes includes: The feature processing module performs sampling and convolution operations on the image feature information in different dimensions to obtain the image fusion features of different sizes.

6. The method according to any one of claims 1-3 and 5, characterized in that, The step of inputting the image fusion features of different sizes into the target detection module and outputting the image detection result includes: The image fusion features of different sizes are respectively input into the target detection layer in the target detection module to obtain the image detection result. The target detection layer of different sizes is used to detect image fusion features of different sizes.

7. The method according to any one of claims 1-3 and 5, characterized in that, The target loss function is constructed as follows: Obtain the respective loss parameters and weights for the facial expression classification loss, the facial detection and localization loss, the target confidence loss, and the temporal consistency loss; The target loss function is obtained by weighted summation of each loss parameter and each loss weight. The loss value of the image detection result is obtained by the following method; Different losses are calculated on the image detection results to obtain the respective loss values ​​for the facial expression classification loss, the facial detection and localization loss, the target confidence loss, and the temporal consistency loss. The loss values ​​are input as loss parameters into the target loss function to obtain the loss values.

8. An apparatus for training a facial recognition model, characterized in that, The apparatus is used to train an initial facial recognition model to obtain a trained facial recognition model, wherein the initial facial recognition model includes: a backbone network module, a feature processing module, and an object detection module; the apparatus is used to perform the method as described in claim 1, including: The extraction module is used to input the image dataset used for training into the backbone network module and output image feature information corresponding to the image dataset; The fusion module is used to input the image feature information into the feature processing module and output image fusion features of different sizes; The processing module is used to input the image fusion features of different sizes into the target detection module and output the image detection results; An optimization module is used to optimize the initial face recognition model using the loss value of the image detection result determined by the target loss function, so as to obtain the trained face recognition model. The target loss function includes: facial expression classification loss, face detection and localization loss, target confidence loss and temporal consistency loss. The face recognition model is used to recognize facial expressions in the image to be recognized.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, performs the method as described in any one of claims 1-7.

10. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored on the memory and running on the processor, wherein the computer program is executed by the processor to perform the method as claimed in any one of claims 1-7.

Images