Methods, apparatus, equipment and storage media for training face recognition models

Abstract

This application relates to the field of face recognition technology, and provides a method, apparatus, device, and storage medium for training a face recognition model. The method includes: acquiring unobstructed and occluded face images of the same user; extracting full-face features, partial face feature maps from the unobstructed face image, and full-face features and partial face feature maps from the occluded face image; constructing a target loss function based on the full-face features, partial face feature maps, and occluded face feature maps, and calculating the target loss; and updating the model parameters of the face recognition model based on the target loss to obtain the target face recognition model. This scheme improves the recognition accuracy of occluded face images by guiding the updating of model parameters through the target loss.

Description

Technical Field

[0001] This application relates to the field of face recognition technology, and in particular to a method, apparatus, device and storage medium for training a face recognition model. Background Technology

[0002] Facial recognition is a biometric technology that identifies individuals based on their facial features. Currently, facial recognition technology is widely used in production, daily life, and other fields.

[0003] Typically, facial recognition technology uses image acquisition devices such as cameras or webcams to capture clear and complete images or video streams of human faces. The facial recognition device can then automatically detect and track faces in the images and perform facial recognition on the detected faces.

[0004] However, in real life, due to external environmental factors and users' personal habits, the images captured by image acquisition devices are not always complete and clear. When there are facial occlusions, such as wearing masks or hats, the accuracy of facial recognition is low. Summary of the Invention

[0005] This application provides a method, apparatus, device, and storage medium for training a face recognition model, which can solve the technical problem of how to improve the accuracy of face recognition under occlusion conditions.

[0006] In a first aspect, embodiments of this application provide a method for training a face recognition model, including:

[0007] Obtain both unobstructed and occluded facial images of the same user. Using both unobstructed and occluded facial images of the same user as training data allows the facial recognition model to learn more comprehensive facial features.

[0008] Extract full-face features, partial-face feature maps of unoccluded face images, full-face features of occluded face images, and partial-face feature maps of occluded face images.

[0009] A target loss function is constructed based on the full-face features of an unoccluded face image, partial face feature maps of an unoccluded face image, and full-face features and partial face feature maps of an occluded face image, and the target loss is calculated. Combining the differences between the full-face features and partial face features of unoccluded and occluded face images to jointly construct the target loss function can make the target function more accurate.

[0010] Based on the target loss, the model parameters of the face recognition model are updated to obtain the target face recognition model. The target face recognition model achieves a preset accuracy threshold for recognizing occluded face images. By iteratively updating the parameters of the face recognition model, it can continuously learn the impact of occlusion on face recognition, thereby improving the accuracy of the face recognition model for occluded face images.

[0011] Secondly, embodiments of this application provide an apparatus for training a face recognition model, which has the function of implementing the method in the first aspect or any possible implementation thereof. Specifically, the apparatus includes units for implementing the method in the first aspect or any possible implementation thereof.

[0012] In one embodiment, the device includes:

[0013] The acquisition unit is used to acquire both unobstructed and occluded face images of the same user.

[0014] The extraction unit is used to extract full-face features of an unoccluded face image, partial face feature maps of an unoccluded face image, full-face features of an occluded face image, and partial face feature maps of an occluded face image.

[0015] The processing unit is used to construct a target loss function and calculate the target loss based on the full-face features of the unoccluded face image, the partial face feature map of the unoccluded face image, the full-face features of the occluded face image, and the partial face feature map of the occluded face image.

[0016] The processing unit is also used to update the model parameters of the face recognition model according to the target loss until the face recognition model's recognition accuracy for occluded face images reaches a preset accuracy threshold.

[0017] Thirdly, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it causes the computer device to implement any of the implementation methods of the first aspect described above.

[0018] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a computer device, causes the computer device to implement the method of any of the implementations of the first aspect described above.

[0019] Fifthly, embodiments of this application provide a computer program product that, when run on a computer device, causes the computer device to execute any of the implementation methods of the first aspect described above.

[0020] The beneficial effects of this application embodiment compared with the prior art are as follows: by using both unoccluded and occluded face images as training datasets and constructing a corresponding target loss function, the target loss is calculated based on the difference between the features of occluded and unoccluded face images, and the model parameters are continuously updated. This allows the face recognition model to learn more comprehensive face features and continuously learn the impact of occlusion on face recognition. Face recognition can be performed using only some face features, thus improving the accuracy of the face recognition model in recognizing occluded face images. Attached Figure Description

[0021] Figure 1 This is a scene diagram of a captured face image provided in an embodiment of this application.

[0022] Figure 2 This is a flowchart illustrating a method for training a face recognition model provided in an embodiment of this application.

[0023] Figure 3 This is a flowchart illustrating another method for training a face recognition model provided in an embodiment of this application.

[0024] Figure 4 This is a schematic diagram of the structure of a face recognition model training device provided in an embodiment of this application.

[0025] Figure 5 This is a schematic diagram of the structure of the computer device provided in the embodiments of this application. Detailed Implementation

[0026] Facial recognition is a technology that uses computer vision and pattern recognition to analyze and compare facial images or videos to automatically identify and verify facial identities. The core principle of facial recognition is to extract feature information from facial images, such as facial contours, eyes, and mouth, and then compare these features with pre-stored facial feature templates to determine if a match exists.

[0027] For example, in a company's time and attendance system, the company pre-collects facial images of each employee and stores them in the attendance machine's facial image database. Each day, when employees arrive and leave work, they stand or approach the attendance machine. After collecting the facial image, the machine extracts a set of feature vectors from the employee's face image. These feature vectors uniquely represent the face's characteristics. The attendance machine then compares the extracted facial features with pre-stored facial feature templates. Based on the similarity or distance obtained from the comparison, the attendance machine can determine whether the face matches the stored facial feature template. If the similarity or distance exceeds a preset threshold, the match is considered successful; otherwise, it is considered a failed match.

[0028] For example, in a residential access control system, the property management company collects facial images of each resident beforehand and stores them in the access control system's facial image database. Whenever a resident returns home, the access control system compares the collected facial image with the pre-stored facial feature templates. Based on the similarity or distance obtained from the comparison, the attendance machine determines whether the face matches the stored facial feature template. If the similarity or distance exceeds a preset threshold, the match is considered successful; otherwise, it is considered a failure.

[0029] Based on the above application scenarios, it is clear that facial recognition methods require capturing facial images at the current moment. Due to objective environmental factors and user habits, not all captured facial images are complete faces. Typically, facial images used for facial recognition suffer from partial occlusion, inconsistent brightness, or incompleteness. For example, under lighting conditions, parts of the face may be dark and difficult to recognize; hats, masks, glasses, or other items may obscure the face; and improper shooting angles or distances may result in incomplete facial images. Therefore, facial recognition suffers from relatively low accuracy.

[0030] The following is combined Figure 1 Let's explain in detail the different types of facial images that may be captured in facial recognition.

[0031] Figure 1 This is a scene diagram of a captured face image provided in an embodiment of this application.

[0032] like Figure 1 As shown, Figure 1 Part A in the diagram represents a user's complete face, and part B represents the user's face when wearing a hat. Figure 1 In the diagram, part C represents a user's complete face, and part D represents the user's face when wearing a mask; Figure 1 In the diagram, part E represents a user's complete face, and part F represents the user's face while wearing sunglasses. Under the same conditions, it's understandable that the accuracy rate is generally higher when recognizing faces in parts A, C, and E; and lower when recognizing faces in parts B, D, and F.

[0033] To address the aforementioned technical issues, this application proposes a method for training a face recognition model. By calculating three different recognition losses during the face recognition process, the training process of the face recognition model is guided, and the model parameters of the face recognition model are updated to improve the recognition accuracy of occluded face images.

[0034] To further illustrate the technical solution of this application, specific embodiments are described below.

[0035] Figure 2 This is a flowchart illustrating a method for training a face recognition model provided in an embodiment of this application.

[0036] like Figure 2 As shown, the above method includes the following steps S201 to S207.

[0037] S201. Obtain the unobstructed and obstructed face images of the same user.

[0038] A facial image is a visual representation of a human face captured by a camera or other image acquisition device. It is a two-dimensional image composed of information such as the shape, texture, and color of the face.

[0039] In facial recognition, the face image is the most basic input data, carrying facial feature information such as key points, contours, and textures. These feature vectors can be used as input to facial recognition algorithms for feature matching and recognition.

[0040] An unobstructed face image refers to a complete, intact, and visible face image. For example, Figure 1 The facial images in parts A, C, and E.

[0041] An occluded face image is an image in which one or more parts of a face are obscured, making it difficult to visualize. Compared to an unoccluded face image, an occluded face image is an incomplete face image. This can refer to a face image of someone wearing a mask, someone wearing glasses, etc. For example, Figure 1 The image contains facial images in parts B, D, and F. During training, corresponding labels can be used to indicate whether a given facial image is an occluded face image.

[0042] The unobstructed and occluded face images of the same user mentioned here can also be understood as training samples. In other words, when training a face recognition model, the unobstructed and occluded face images of the same user can be combined to train the face recognition model.

[0043] S202. Extract the full-face features of an unobstructed face image, the partial face feature map of an unobstructed face image, the full-face features of an occluded face image, and the partial face feature map of an occluded face image.

[0044] Full-face features refer to the feature vector extracted from a face image that contains information about the entire face. It is a set of numerical features extracted from the entire face image, used to uniquely represent the face.

[0045] As an example rather than a limitation, full-face features include information such as the shape, texture, and color of the face, which can be used for feature matching and recognition in face recognition algorithms.

[0046] A partial facial feature map refers to a feature map extracted from a facial image that contains only a portion of the facial information. It is a feature image extracted from a facial image to represent a specific part of the facial features.

[0047] As an example and not a limitation, a partial facial feature map may include feature images of parts of a face such as the eyes, nose, and mouth.

[0048] Both full-face features and partial-face feature maps are numerical feature vectors or feature images extracted from face images, and both can be used for face detection or face recognition.

[0049] It can be understood that the full-face features of an unoccluded face image are the feature vectors of the entire face information in the unoccluded face image. The partial face feature map of an unoccluded face image is a feature map containing only a portion of the face information of the unoccluded face.

[0050] Correspondingly, the full-face feature map of an occluded face image is the feature vector of the entire face information in the occluded face image. The partial face feature map of an occluded face image is the feature map that contains the partial face information of the occluded face.

[0051] In one implementation, a face detection algorithm (such as a deep learning face detector) is used to locate face regions in the image, the detected faces are aligned, and full-face features are extracted from the aligned face image. The full-face feature extraction methods here can include Local Binary Patterns (LBP), Principal Component Analysis (PCA), and face descriptors (such as deep learning models like Open Face, FaceNet, and Deep Face).

[0052] In one implementation, face detection algorithms (such as deep learning face detectors) are used to locate face regions in the image, and key point detection algorithms such as multi-task convolutional neural networks (MTCNN) are used to locate key points of the face, such as eyebrows, eyes, nose, and mouth. Based on the location information of the key points, the face image is cropped into partial face feature maps.

[0053] S203. Construct a target loss function based on the full-face features of the unobstructed face image, the partial face feature map of the unobstructed face image, the full-face features of the occluded face image, and the partial face feature map of the occluded face image, and calculate the target loss.

[0054] In face recognition, the loss function is used to measure the difference between the model's prediction and the true label. Common loss functions include: Cross-Entropy Loss, Triplet Loss, Cosine Loss, and so on.

[0055] The role of the loss function is to guide the model's learning process. By minimizing the loss function, the model's parameters are optimized, enabling the model to better fit the training data and improve the accuracy of face recognition. In other words, the loss function can be viewed as the objective function for model optimization; by continuously optimizing the loss function, the model's recognition ability is improved.

[0056] Loss refers to the difference or error between the model's prediction and the true label. During training, the model reduces the difference between the prediction and the true label by minimizing the loss function, thereby improving the model's performance.

[0057] The target loss function here refers to the loss function that can be used to identify occluded face images. The target loss is the loss function that can be used to identify occluded face images.

[0058] In one implementation, during training, a training dataset containing unoccluded and occluded face images is prepared. Each sample has corresponding full-face features and partial face feature maps. The full-face features of the unoccluded face image, the partial face feature maps of the unoccluded face image, the full-face features of the occluded face image, and the partial face feature maps of the occluded face image are represented as feature vectors and used as input values ​​for the target loss function. The target function is implemented mathematically, and the loss value of each sample is calculated. Alternatively, a batch calculation method can be used to calculate the loss value of multiple samples simultaneously. Finally, the loss values ​​of all samples are summarized to obtain the target loss value.

[0059] S204. Based on the target loss, update the model parameters of the face recognition model to obtain the target face recognition model. The target face recognition model achieves a preset accuracy threshold for recognizing occluded face images.

[0060] In one implementation, the face recognition model updated based on the target loss can be based on the gradient descent algorithm. The basic idea of ​​gradient descent is to guide the direction of updating the model parameters by calculating the gradient (derivative) of the target loss function with respect to the model parameters. By iteratively updating the model parameters, gradient descent can gradually minimize the model's loss function.

[0061] In one example, the parameters of the face recognition model are initialized, such as weights and biases. Using samples from the training dataset, the gradient of the target loss function with respect to the model parameters is calculated. The gradient represents the rate of change of the target function at the current parameter values ​​and guides the direction of parameter updates. Based on the direction and magnitude of the gradient, the model parameters are updated. A learning rate is typically used to control the step size of parameter updates to avoid updates being too fast or too slow. The above method is repeated until a stopping condition is met, such as reaching a certain number of iterations or the value of the target loss function converging to a small threshold.

[0062] By iteratively updating the model parameters, the gradient descent algorithm can gradually adjust the model's parameters, thereby improving the model's performance in face recognition tasks. The final parameters of the target face recognition model are designed to achieve a preset accuracy threshold for recognizing occluded faces.

[0063] It is understandable that the standards for facial image recognition rate differ in different application scenarios. A preset accuracy threshold can be set according to actual needs, and no limit is imposed here.

[0064] By using both unoccluded and occluded face images as training datasets and constructing corresponding target loss functions, the target loss is calculated based on the differences between the features of occluded and unoccluded face images, and the model parameters are continuously updated. This allows the face recognition model to learn more comprehensive face features and continuously learn the impact of occlusion on face recognition. Face recognition can be performed using partial face features, thus improving the accuracy of the face recognition model in recognizing occluded face images.

[0065] The following describes the process of constructing the target loss function.

[0066] In one implementation, the target loss function includes a mean squared error (MSE) loss function, and the target loss includes the MSE loss. Based on the full-face features of the unoccluded face image and the full-face features of the occluded face image, the MSE loss function is constructed, and the MSE loss is calculated. This method can be seen as a specific example of step S203 described above.

[0067] The Mean Squared Error (MSE Loss) function here can be torch.nn.MSELoss.

[0068] In one example, suppose that the full-face features of an unoccluded face image are extracted using an unoccluded face recognition model, and the full-face features of an occluded face image are extracted using an occluded face recognition model. The full-face features of the unoccluded and occluded face images are of the same dimension, and their shapes can be represented by [B, D], where B is the batch size and D is the feature dimension. The difference between the two types of full-face features is used as the input to the MSE Loss function. By calculating the MSE Loss function, which is the squared difference between the predicted and true values, the mean squared error loss is obtained.

[0069] It's understandable that using the mean squared error (MSE) loss to guide the parameter updates of an occluded face recognition model allows it to learn the discrimination capabilities of an unoccluded face recognition model. In other words, the unoccluded face recognition model has already learned to distinguish faces with high accuracy. By using the differences between the two types of full-face features as input to calculate the MSE loss, and then using this loss to guide the parameter updates of the occluded face recognition model, it can gradually learn the discrimination capabilities of the unoccluded model, thereby improving the accuracy of the occluded face recognition model.

[0070] In one implementation, the target loss function includes a partial face attention transfer loss function, and the target loss includes the partial face attention transfer loss. Based on the partial face feature maps of the unoccluded face image and the partial face feature maps of the occluded face image, a partial face attention transfer loss function is constructed, and the partial face attention transfer loss is calculated. This method can be seen as a specific example of step S203 above.

[0071] The term "partial face" as used here refers to the portion of a face remaining after it has been covered. For example, when a mask is worn on the lower half of a person's face, "partial face" refers to the upper half of the face not covered by the mask. The specific area of ​​"partial face" can be determined based on the actual situation and is not limited here.

[0072] In facial recognition, partial occlusion of the face by items such as hats, masks, and glasses can make it difficult for facial recognition models to correctly identify the occluded portion of the face. Partial Attention Transfer Loss introduces an attention mechanism to help the facial recognition model focus on both occluded and unoccluded areas, thereby improving its ability to recognize occluded faces.

[0073] In one example, assume that the partial face feature maps of an unoccluded face image are extracted using an unoccluded face recognition model, and the partial face feature maps of an occluded face image are extracted using an occluded face recognition model. The partial face feature maps of the unoccluded and occluded face images are of the same dimension, and their shapes can be [B, C, H, W], where B is the batch size, C is the number of feature map channels, H is the height of the feature map, and W is the width of the feature map. The partial face attention transfer loss is calculated using the following formula (1) for these two half-face feature maps.

[0074] Lat=mean(||norm(M fm )-norm(N fm )||2), (1)

[0075] Where Lat represents half-face attention shift loss; ||·||2 refers to the 2-norm rule in mathematics; M fm With N fm These represent two 2D vectors transformed from two 4D half-face feature maps. First, the two feature maps are calculated using ||·||², and then the average value is calculated along the channel direction. The shape is [B, H, W]. Then, they are converted into 2D vectors with the shape [B, H*W]. They represent the eigenvector M respectively fm With N fm Perform normalization, such as M fm Normalization can be performed using torch.nn.functional.normalize(M fm The expression is used to calculate the mean; mean represents the average of a 2-dimensional vector, which is a 1-dimensional constant.

[0076] By introducing a partial face attention transfer loss, the occluded face recognition model focuses its attention on the unoccluded facial regions during training, learning from the unoccluded face recognition model its feature representation capabilities for the unoccluded facial regions. In other words, the occluded face recognition model can better learn how to focus on both occluded and unoccluded regions, thereby improving its ability to recognize occluded faces.

[0077] In one implementation, the target loss function includes an occluded full-face classification loss function, and the target loss includes the occluded full-face classification loss; based on the full-face features of the occluded face image, an occluded full-face classification loss function is constructed, and the occluded full-face classification loss is calculated. This method can be seen as a specific example of step S203 above.

[0078] In one example, the loss function for classifying occluded full faces could be a spaced cross-entropy loss (SoftMaxLoss) function, such as the ArcFace Loss function.

[0079] The ArcFace Loss function is an improved classification loss function specifically designed for face recognition tasks. Its main calculation principle is as follows:

[0080] The output vector of the face recognition model is normalized to a unit length. A linear transformation maps the normalized output to a high-dimensional feature space; this mapping is called "feature extraction." The feature vector is then multiplied by the weight vector to obtain a score for each category. For the true label category, a large positive number (called the "margin") is subtracted from its score, while for other categories, a smaller positive number is subtracted. The adjusted scores are then normalized to obtain the probability distribution for each category.

[0081] This can also be understood as follows: during the training of an occluded face recognition model, by using the full-face features of the occluded face image as the input value of the ArcFace Loss function to calculate the occluded full-face classification loss, the set of faces of the same type (which can also be understood as the faces of the same user) can be compacted, while the sets of faces of different types (which can also be understood as the faces of different users) are spaced far apart, so that the face recognition model can better distinguish different faces in the feature space, thereby improving the accuracy of face recognition.

[0082] In one implementation, the weight values ​​corresponding to the mean squared error loss, partial face attention transfer loss, and occluded full face classification loss are calculated respectively; the target loss is calculated based on the values ​​of the mean squared error loss, partial face attention transfer loss, and occluded full face classification loss and the corresponding weight values.

[0083] In one example, the target loss is calculated using the following formula (2).

[0084] L=a*Lmse + b*Lat + c*Lcls, (2)

[0085] Where L represents the target loss, Lmse represents the mean squared error loss, Lat represents the partial face attention transfer loss, Lcls represents the occluded full face classification loss, a represents the weight of the mean squared error loss, b represents the weight of the partial face attention transfer loss, and c represents the weight of the occluded full face classification loss.

[0086] As an example rather than a limitation, a can take the value 100, b can take the value 10, and c can take the value 1.

[0087] When multiple loss functions exist, some may be more important than others. By setting different weights, we can ensure that important loss functions have a greater impact on the training of the face recognition model, and also balance the contributions of different loss functions, preventing any one loss function from dominating the entire training process. This helps stabilize the training process and improve the model's generalization performance.

[0088] In one implementation, the full-face features and partial-face feature maps of an unoccluded face image are extracted using a pre-trained unoccluded face recognition model. Similarly, the full-face features and partial-face feature maps of an occluded face image are extracted using a pre-trained occluded face recognition model.

[0089] A pre-trained unoccluded face recognition model refers to a face recognition model trained on a large-scale dataset to recognize unoccluded faces. This model uses a large number of unoccluded face images for training to learn facial features and representations. Pre-trained unoccluded face recognition models are typically trained using deep learning methods, such as convolutional neural networks (CNNs). During training, the model uses a network structure composed of multiple convolutional layers, pooling layers, and fully connected layers to extract features from the input face images using a feature extraction network and map them into a high-dimensional feature space. By learning these feature representations, the unoccluded face recognition model can classify and recognize corresponding faces in face recognition tasks. The pre-trained unoccluded face recognition model does not update parameters in the face recognition model training method mentioned in this application.

[0090] Accordingly, the pre-trained occluded face recognition model is designed to recognize occluded faces, and the parameters of the pre-trained occluded face recognition model can be updated in the face recognition model training method mentioned in this application.

[0091] In one implementation, an occlusion template is used to occlude an unoccluded face image to obtain an occluded face image. The occlusion template is used to occlude any one or more parts of the unoccluded face image.

[0092] As an example rather than a limitation, the occlusion template can include any template that can cover a person's face, such as a hat, mask, scarf, or glasses.

[0093] When masking, masking templates can cover both key facial features, such as eyebrows, glasses, and mouth, and non-key facial features, such as cheeks. Masking templates can be used individually or in combination; for example, a mask template and an eyewear template can be used simultaneously to mask a face. Specific masking methods can be combined according to actual needs and are not limited here.

[0094] The following is combined Figure 3 This paper provides an overview of a method for training a face recognition model.

[0095] Figure 3 This is a flowchart illustrating another method for training a face recognition model provided in an embodiment of this application.

[0096] like Figure 3 As shown, Figure 3 The steps include S301 to S308.

[0097] S301. Input the unobstructed face image into the unobstructed face recognition model.

[0098] like Figure 3 As shown, an unobstructed face image is a fully visible face image.

[0099] S302. Input the occluded face image into the occluded face recognition model.

[0100] Combination Figure 1 As can be seen, the occluded facial images can be images of people wearing masks, glasses, etc.

[0101] S303, Extract full-face features without occlusion and partial face feature map without occlusion.

[0102] As can be seen from the above, here we can use feature extraction networks in the unobstructed face recognition model to extract unobstructed full-face features and unobstructed partial face feature maps.

[0103] S304. Extract full-face features with occlusion and partial-face feature maps with occlusion.

[0104] As can be seen from the above, here we can use feature extraction networks in occluded face recognition models to extract full-face features and partial-face feature maps with occlusion.

[0105] S305. Calculate the root mean square error loss.

[0106] As mentioned above, the mean squared error loss is calculated here by utilizing the difference between the unobstructed full-face features and the occluded full-face features.

[0107] S306. Calculate the partial face attention shift loss.

[0108] As mentioned above, this method calculates the partial face attention transfer loss by utilizing the difference between the feature maps of the unoccluded and occluded parts of the face.

[0109] S307. Calculate the classification loss for occluded full faces.

[0110] As can be seen from the above, the occluded full-face classification loss is calculated here by using the full-face features of the occluded face image and the partial face feature map of the occluded face image.

[0111] S308. Calculate the target loss and update the model parameters.

[0112] By combining the three types of losses and assigning different weights to each loss, the weighted value of the three losses is calculated to obtain the target loss. Then, the parameters of the occluded face recognition model are updated based on the target loss.

[0113] By using the above method, a batch of paired unoccluded and occluded face images are input into the corresponding face recognition model. Different loss functions are calculated for each model, and the target loss is obtained to guide the training of the occluded face recognition model and update the parameters of the occluded face recognition model until a preset accuracy threshold is reached, which can improve the recognition accuracy of occluded faces.

[0114] As discussed above, when the recognition accuracy on an occluded face image reaches a preset accuracy threshold, a target face recognition model can be obtained and put into practical use. Using this target face recognition model, occluded faces can be identified in various situations.

[0115] For example, during transportation, when a user is wearing a mask for identity verification, the image acquisition device will capture the user's face image while wearing a mask and input it into the target face recognition model. The target face recognition model can directly recognize the user's face image while wearing a mask and make an identity judgment without the user needing to remove the mask.

[0116] For example, in a payment scenario, when a user wears glasses to make a facial recognition payment, the image acquisition device captures the user's face image while wearing glasses and inputs it into the target facial recognition model. The target facial recognition model can directly recognize the user's face image while wearing glasses and make an identity determination without the user needing to remove their glasses.

[0117] It's understandable that when training a face recognition model, a portion of unoccluded face data and a portion of occluded face data generated from this unoccluded face data are used. For example, 50% unoccluded face data and 50% occluded face data. The specific data ratio can be set according to the actual situation and is not limited here. Using occluded face data to train the model can improve the target face recognition model's ability to recognize occluded faces. On the other hand, using unoccluded face data also allows the target face recognition model to recognize faces in normal situations, enabling more accurate recognition when encountering unoccluded faces. In summary, by using both unoccluded and occluded face data for training, the target face recognition model can accurately recognize faces in both situations.

[0118] It is understood that the target face recognition model mentioned in this application can be applied in other scenarios that require face recognition, such as surveillance management, criminal investigation, mobile payment, image retrieval, transportation, electronic device unlocking and other technical fields, which will not be elaborated here.

[0119] The methods of the embodiments of this application have been described above with reference to the accompanying drawings. It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially, these steps are not necessarily executed in the order shown in the figures. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the steps or stages of other steps. The apparatus of one embodiment of this application will now be described with reference to the accompanying drawings. For brevity, appropriate omissions will be made in the following description of the apparatus; relevant content can be referred to in the relevant descriptions of the methods above, and will not be repeated.

[0120] Figure 4 The application embodiment provides a schematic diagram of the structure of a device for training a face recognition model.

[0121] like Figure 4 As shown, the device 1000 includes the following units.

[0122] The acquisition unit 1001 is used to acquire unobstructed and occluded face images of the same user.

[0123] Extraction unit 1002 is used to extract full-face features of an unobstructed face image, partial face feature maps of an unobstructed face image, full-face features of an occluded face image, and partial face feature maps of an occluded face image.

[0124] The processing unit 1003 is used to construct a target loss function and calculate the target loss based on the full-face features of the unoccluded face image, the partial face feature map of the unoccluded face image, the full-face features of the occluded face image, and the partial face feature map of the occluded face image.

[0125] The processing unit 1003 is also used to update the model parameters of the face recognition model according to the target loss until the face recognition model's recognition accuracy for occluded face images reaches a preset accuracy threshold.

[0126] In one implementation, the acquisition unit 1001 can also be used to execute the methods in steps S301 and S302.

[0127] In one implementation, the extraction unit 1002 can also be used to execute the methods in steps S303 and S304.

[0128] In one implementation, the processing unit 1003 can also be used to execute the methods in steps S305 to S308.

[0129] In one implementation, the device 1000 further includes a storage unit that can be used to store instructions and / or data, thereby implementing the methods described in the above embodiments.

[0130] It should be noted that the information interaction and execution process between the above-mentioned units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0131] Figure 5 This is a schematic diagram of the structure of the computer device provided in an embodiment of this application. Figure 5 As shown, the computer device 3000 of this embodiment includes: at least one processor 3100 ( Figure 5 (Only one is shown) a processor, a memory 3200, and a computer program 3210 stored in the memory 3200 and executable on at least one processor 3100, wherein when the processor 3100 executes the computer program 3210, the computer device performs the steps described in the above embodiments.

[0132] The processor 3100 can be a Central Processing Unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0133] In some embodiments, memory 3200 may be an internal storage unit of computer device 3000, such as a hard disk or RAM of computer device 3000. In other embodiments, memory 3200 may be an external storage device of computer device 3000, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., provided on computer device 3000. Furthermore, memory 3200 may include both internal and external storage units of computer device 3000. Memory 3200 is used to store operating system, application programs, boot loader data, and other programs, such as program code for computer programs. Memory 3200 may also be used to temporarily store data that has been output or will be output.

[0134] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units is merely an example. In practical applications, the above functions can be assigned to different functional units or modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0135] This application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a computer device, it enables the computer device to perform the steps described in the above-described method embodiments.

[0136] This application provides a computer program product that, when run on a computer device, enables the computer device to implement the methods described above.

[0137] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it enables a computer device to implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0138] It should be understood that the sequence numbers of the steps in the above embodiments do not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. In the description, specific details such as particular system structures and technologies are set forth for illustrative purposes rather than for limiting purposes, so as to provide a thorough understanding of the embodiments of this application. However, those skilled in the art should understand that this application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of this application with unnecessary details.

[0139] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0140] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0141] Furthermore, in the description of this application and the appended claims, the terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0142] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0143] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0144] In the embodiments provided in this application, it should be understood that the disclosed apparatus, computer equipment, and methods can be implemented in other ways. For example, the apparatus and computer equipment embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0145] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for training a face recognition model, characterized in that, include: Obtain unobstructed and obstructed face images of the same user; Extract the full-face features of the unobstructed face image, the partial face feature map of the unobstructed face image, the full-face features of the occluded face image, and the partial face feature map of the occluded face image. The full-face features of the unobstructed face image and the full-face features of the occluded face image are of the same dimension, and the partial face feature maps of the unobstructed face image and the partial face feature maps of the occluded face image are of the same dimension. A target loss function is constructed based on the full-face features of the unoccluded face image, the partial face feature map of the unoccluded face image, the full-face features of the occluded face image, and the partial face feature map of the occluded face image, and the target loss is calculated. The target loss function includes a partial face attention transfer loss function, and the target loss includes a partial face attention transfer loss; or, the target loss function includes an occluded full-face classification loss function, and the target loss includes an occluded full-face classification loss. Based on the target loss, the model parameters of the face recognition model are updated to obtain the target face recognition model. The target face recognition model achieves a preset accuracy threshold for recognizing the occluded face image.

2. The method according to claim 1, characterized in that, The target loss function includes the mean squared error loss function, and the target loss includes the mean squared error loss; The step of constructing a target loss function based on the full-face features of the unobstructed face image, the partial face feature map of the unobstructed face image, the full-face features of the occluded face image, and the partial face feature map of the occluded face image, and calculating the target loss, includes: Based on the full-face features of the unobstructed face image and the full-face features of the occluded face image, the mean squared error loss function is constructed, and the mean squared error loss is calculated.

3. The method according to claim 2, characterized in that, The target loss function includes a partial face attention transfer loss function, and the target loss includes a partial face attention transfer loss; The step of constructing a target loss function based on the full-face features of the unobstructed face image, the partial face feature map of the unobstructed face image, the full-face features of the occluded face image, and the partial face feature map of the occluded face image, and calculating the target loss, includes: Based on the partial facial feature maps of the unobstructed face image and the partial facial feature maps of the occluded face image, a partial face attention transfer loss function is constructed, and the partial face attention transfer loss is calculated.

4. The method according to claim 3, characterized in that, The target loss function includes an occluded full-face classification loss function, and the target loss includes an occluded full-face classification loss. The step of constructing a target loss function based on the full-face features of the unobstructed face image, the partial face feature map of the unobstructed face image, the full-face features of the occluded face image, and the partial face feature map of the occluded face image, and calculating the target loss, includes: Based on the full-face features of the occluded face image and the partial face feature map of the occluded face image, an occluded full-face classification loss function is constructed, and the occluded full-face classification loss is calculated.

5. The method according to claim 4, characterized in that, The method further includes: Calculate the weight values ​​corresponding to the mean squared error loss, the partial face attention transfer loss, and the occluded full face classification loss, respectively. The target loss is calculated based on the values ​​of the mean squared error loss, the partial face attention transfer loss, and the occluded full face classification loss, as well as the corresponding weight values ​​of the mean squared error loss, the partial face attention transfer loss, and the occluded full face classification loss.

6. The method according to claim 1, characterized in that, The full-face features and partial-face feature maps of the unobstructed face image are extracted by a pre-trained unobstructed face recognition model. The full-face features and partial-face feature maps of the occluded face image are extracted using a pre-trained occluded face recognition model.

7. The method according to any one of claims 1-6, characterized in that, The method further includes: The occluded face image is obtained by occluding the unoccluded face image using an occlusion template, wherein the occlusion template is used to occlude any one or more parts of the unoccluded face image.

8. An apparatus for training a face recognition model, characterized in that, include: The acquisition unit is used to acquire both unobstructed and occluded face images of the same user. An extraction unit is used to extract the full-face features of the unobstructed face image, the partial face feature map of the unobstructed face image, the full-face features of the occluded face image, and the partial face feature map of the occluded face image. The full-face features of the unobstructed face image and the full-face features of the occluded face image are of the same dimension, and the partial face feature map of the unobstructed face image and the partial face feature map of the occluded face image are of the same dimension. The processing unit is configured to construct a target loss function based on the full-face features of the unoccluded face image, the partial face feature map of the unoccluded face image, the full-face features of the occluded face image, and the partial face feature map of the occluded face image, and to calculate the target loss. The target loss function includes a partial face attention transfer loss function, and the target loss includes a partial face attention transfer loss; or, the target loss function includes an occluded full-face classification loss function, and the target loss includes an occluded full-face classification loss. The processing unit is further configured to update the model parameters of the face recognition model according to the target loss, until the face recognition model achieves a preset accuracy threshold for recognizing the occluded face image.

9. A computer device, characterized in that, The device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the computer device performs the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a computer device, implements the method as described in any one of claims 1-7.

Images