Face identity recognition model training method, face identity recognition method and device

Abstract

The application discloses a face identity recognition model training method, a face identity recognition method and device, and relates to the field of artificial intelligence. The method comprises the following steps: performing feature extraction on a sample face image through a first feature extraction network to obtain a first sample face feature; based on the first sample face feature, extracting a sample occlusion region feature of a face occlusion region in the sample face image through a second feature extraction network; performing feature purification processing on the first sample face feature based on the sample occlusion region feature to obtain a second sample face feature; performing feature extraction on the first sample face feature, the second sample face feature and the sample occlusion region feature through a third feature extraction network to obtain a third sample face feature, wherein the third feature extraction network adopts a self-attention mechanism; and training a face identity recognition model based on the third sample face feature. The embodiment scheme of the application is helpful to improve the accuracy of face identity recognition in a face occlusion scene.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a training method for a facial recognition model, a facial recognition method, and an apparatus. Background Technology

[0002] As facial recognition technology continues to mature, more and more devices are adopting this biometric technology for privacy protection. For example, mobile phones with facial recognition capabilities can unlock and make payments based on a captured face.

[0003] However, users may wear items such as glasses, scarves, and masks in their daily lives, which can obstruct the face area and affect the accuracy of facial recognition. Summary of the Invention

[0004] This application provides a method for training a face recognition model, a face recognition method, and an apparatus. The technical solution is as follows:

[0005] On one hand, embodiments of this application provide a method for training a facial recognition model, the method comprising:

[0006] The first feature extraction network is used to extract features from the sample face image to obtain the first sample face features;

[0007] Based on the first sample face features, the second feature extraction network extracts the sample occlusion region features of the face occlusion region in the sample face image;

[0008] Based on the features of the occlusion region of the sample, the first sample face features are subjected to feature purification processing to obtain the second sample face features, wherein the feature purification processing is used to eliminate the interference of the face occlusion region.

[0009] Based on the face features of the first sample, the face features of the second sample, and the occlusion region features of the sample, features are extracted through a third feature extraction network to obtain the face features of the third sample. The third feature extraction network adopts a self-attention mechanism.

[0010] A face recognition model is trained based on the face features of the third sample. The face recognition model consists of the first feature extraction network, the second feature extraction network, and the third feature extraction network.

[0011] On the other hand, embodiments of this application provide a facial recognition method, the method comprising:

[0012] The first feature extraction network of the face recognition model is used to extract features from the target face image to obtain the first face feature;

[0013] Based on the first facial features, the occlusion region features of the occlusion region in the target facial image are extracted through the second feature extraction network of the facial identity recognition model;

[0014] The first face feature is purified based on the occlusion region features to obtain the second face feature, wherein the feature purification process is used to eliminate interference from the occlusion region of the face.

[0015] Based on the first face feature, the second face feature, and the occlusion region feature, the third face feature is obtained by extracting features through the third feature extraction network of the face recognition model. The third feature extraction network adopts a self-attention mechanism.

[0016] Facial identity recognition is performed based on the aforementioned third facial features;

[0017] The facial recognition model is trained using the method described above.

[0018] On the other hand, embodiments of this application provide a training device for a facial recognition model, the device comprising:

[0019] The feature extraction module is used to extract features from the sample face image through the first feature extraction network to obtain the first sample face features;

[0020] The feature extraction module is used to extract the sample occlusion region features of the face occlusion region in the sample face image based on the first sample face features and through the second feature extraction network.

[0021] The feature extraction module is used to perform feature purification processing on the first sample face features based on the features of the sample occlusion region to obtain the second sample face features, wherein the feature purification processing is used to eliminate the interference of the face occlusion region.

[0022] The feature extraction module is used to extract features based on the first sample face features, the second sample face features, and the sample occlusion region features, through a third feature extraction network to obtain the third sample face features. The third feature extraction network adopts a self-attention mechanism.

[0023] The training module is used to train a face recognition model based on the face features of the third sample. The face recognition model consists of the first feature extraction network, the second feature extraction network, and the third feature extraction network.

[0024] On the other hand, embodiments of this application provide a facial recognition device, the device comprising:

[0025] The feature extraction module is used to extract features from the target face image through the first feature extraction network of the face recognition model to obtain the first face features;

[0026] The feature extraction module is used to extract the occlusion region features of the occlusion region in the target face image based on the first face features and through the second feature extraction network of the face recognition model.

[0027] The feature extraction module is used to perform feature purification processing on the first face feature based on the occlusion region feature to obtain the second face feature, wherein the feature purification processing is used to eliminate the interference of the face occlusion region.

[0028] The feature extraction module is used to extract features based on the first face feature, the second face feature, and the occlusion region feature, through the third feature extraction network of the face recognition model to obtain the third face feature. The third feature extraction network adopts a self-attention mechanism.

[0029] The identification module is used to perform facial identity recognition based on the third facial features;

[0030] The facial recognition model is trained using the apparatus described above.

[0031] On the other hand, embodiments of this application provide a computer device, the computer device including a processor and a memory; the memory stores at least one instruction, the at least one instruction being executed by the processor to implement the training method of the face recognition model as described above, or the face recognition method.

[0032] On the other hand, embodiments of this application provide a computer-readable storage medium storing at least one piece of program code, which is loaded and executed by a processor to implement the training method for the face recognition model as described above, or the face recognition method.

[0033] On the other hand, embodiments of this application provide a computer program product including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform a training method for a face recognition model, or a face recognition method, provided in various optional implementations of the above aspects.

[0034] In this embodiment of the application, during the training and application of the face recognition model, features are extracted from the occluded area of ​​the face, and the extracted occluded area features are used to refine the face features. Then, a self-attention mechanism is used to further extract features from the face features before refinement, the face features after refinement, and the occluded area features. Finally, the extracted face features are used for face recognition, which can eliminate the interference caused by face occlusion to the recognition and help improve the accuracy of face recognition. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 A flowchart illustrating a training method for a face recognition model provided in an exemplary embodiment of this application is shown;

[0037] Figure 2 A flowchart illustrating a training method for a face recognition model provided in another exemplary embodiment of this application is shown;

[0038] Figure 3 This is a structural diagram of a face recognition model illustrated in an exemplary embodiment of this application;

[0039] Figure 4 This is a structural diagram of the attention module shown in an exemplary embodiment of this application;

[0040] Figure 5 This is a flowchart illustrating a loss determination process in an exemplary embodiment of this application;

[0041] Figure 6 A flowchart of a face recognition method provided in an exemplary embodiment of this application is shown;

[0042] Figure 7 This is a schematic diagram illustrating an implementation of a facial recognition process provided in an exemplary embodiment of this application;

[0043] Figure 8 This is a structural block diagram of a training device for a facial recognition model provided in an exemplary embodiment of this application;

[0044] Figure 9 This is a structural block diagram of a face recognition device provided in an exemplary embodiment of this application;

[0045] Figure 10This is a structural block diagram of a computer device provided in an exemplary embodiment of this application. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0047] This application provides an end-to-end model training scheme. The face recognition model trained using this scheme consists of a first feature extraction network, a second feature extraction network, and a third feature extraction network. The first feature extraction network extracts complete face features, the second feature extraction network extracts occluded region features, and the occluded region features are used to refine the complete face features. The third feature extraction network performs further feature extraction based on the complete face features, the occluded region features, and the refined face features.

[0048] By leveraging feature extraction and self-attention mechanisms, the model trained using the scheme provided in this application can extract more accurate facial features for face recognition, reduce the interference of face occlusion on face recognition, and improve the accuracy of face recognition in face occlusion scenarios.

[0049] The face recognition model training method provided in this application can be used in electronic devices with neural network model training capabilities, such as workstations, servers, cloud computing platforms, etc.; while the face recognition method provided in this application can be used in electronic devices with face recognition requirements, such as smartphones, tablets, personal computers, attendance devices, security devices, servers, etc., and this application does not limit this application. For ease of description, the following embodiments use computer devices as examples of execution subjects, but this does not constitute a limitation.

[0050] Please refer to Figure 1 This document illustrates a flowchart of a training method for a face recognition model provided in an exemplary embodiment of this application. The method may include the following steps:

[0051] Step 102: Extract features from the sample face image using the first feature extraction network to obtain the first sample face features.

[0052] In some embodiments, the sample face image belongs to pre-constructed training data, which may include real faces, real occluded faces (such as faces wearing masks, sunglasses, or scarves), and synthetic occluded faces (occluders synthesized on top of real faces). Using the above training data for model training helps improve the model's generalization performance and enables the model to be used for both unoccluded and occluded face recognition, thus improving the model's versatility.

[0053] For each sample face image in the training data, the computer device extracts features through a first feature extraction network to obtain the first sample face features of a complete face. However, if there are occluded areas in the sample face image, the face features extracted by the first feature extraction network will be affected by the occluded areas.

[0054] In some embodiments, the first feature extraction network may adopt a general feature extraction network such as Mobile FaceNet or ResNet (i.e., a network that is not specially trained for occluded faces). The network architecture of the first feature extraction network is not limited in this application embodiment.

[0055] Optionally, to improve the quality of subsequent feature extraction, before feature extraction, the computer device first performs face region detection on the sample face image using an occluded face detection model, thereby extracting faces from the sample face image based on the face regions. Furthermore, since the faces in the sample face image may not be frontal, the computer device needs to perform affine transformations (using 5-point, 68-point, or 106-point affine transformations) on the extracted faces to achieve face correction. The occluded face detection model can be a facebox, insightface, etc., and this embodiment does not limit the specific model used.

[0056] Step 104: Based on the face features of the first sample, extract the occlusion region features of the face occlusion region in the sample face image through the second feature extraction network.

[0057] When there are obstructions, the obstructions can cause some facial features to be missing, or even interfere with the facial features in the unobstructed areas, resulting in poor quality of the extracted facial features. If the extracted facial features are used directly for facial recognition, the accuracy of facial recognition will be poor.

[0058] In this embodiment, the computer device further extracts sample features of the occluded area in the sample face image through a second feature extraction network. Since the first sample face features contain occlusion information, the computer device extracts features of the occluded area based on the first sample face features.

[0059] Step 106: Based on the features of the occluded region of the sample, perform feature purification processing on the face features of the first sample to obtain the face features of the second sample. The feature purification processing is used to eliminate the interference of the face occlusion region.

[0060] Furthermore, the computer equipment uses the features of the occlusion region of the sample to extract features from the first sample face features, in order to eliminate the interference of the occlusion region on the first sample face features, and obtain the second sample face features. The feature size of the second sample face features is the same as that of the first sample face features.

[0061] Step 108: Based on the face features of the first sample, the face features of the second sample, and the features of the sample occlusion region, feature extraction is performed through a third feature extraction network to obtain the face features of the third sample. The third feature extraction network adopts a self-attention mechanism.

[0062] To further improve the quality of facial features, in some embodiments, the computer device employs a self-attention mechanism to extract features based on the first sample facial features, the second sample facial features, and the features of the sample occlusion region, to obtain the third sample facial features.

[0063] Because the refined facial features and occlusion features are used to promote the training process of the third sample facial features, the third feature extraction network can pay more attention to the unoccluded facial regions. By being able to obtain the contour features of the occluded regions from the occlusion features, the quality of the extracted third sample facial features can be further improved.

[0064] Step 110: Train a face recognition model based on the face features of the third sample. The face recognition model consists of a first feature extraction network, a second feature extraction network, and a third feature extraction network.

[0065] After feature extraction is completed through the above steps, in one possible implementation, the computer device performs facial recognition based on the facial features of the third sample, and trains the first feature extraction network, the second feature extraction network, and the third feature extraction network based on the facial recognition results to achieve end-to-end facial recognition model training.

[0066] In some embodiments, each sample face image in the training data is assigned a face identity label, wherein different face identity labels are used to identify different face objects. For example, real occluded faces, real faces, and synthetic occluded faces corresponding to the same object have the same face identity label; real occluded faces corresponding to different objects have different face identity labels.

[0067] In another possible implementation, the computer device trains a face recognition model based on the third-sample face features corresponding to each sample face image and the face identity label. During model training, the goal is to reduce the feature distance between the third-sample face features corresponding to sample face images with the same face identity label, and to increase the feature distance between the third-sample face features corresponding to sample face images with different face identity labels.

[0068] In summary, in the embodiments of this application, during the training process of the facial recognition model, features are extracted from the occluded areas of the face, and the extracted occluded area features are used to refine the facial features. Then, through a self-attention mechanism, further feature extraction is performed on the facial features before refinement, the facial features after refinement, and the occluded area features. Finally, the extracted facial features are used for facial recognition, which can eliminate the interference caused by facial occlusion to identity recognition and help improve the accuracy of facial recognition.

[0069] In one possible implementation, the second feature extraction network employs a segmentation approach, segmenting the occluded face region from the sample face image based on the first sample face features. Then, it extracts features from the occluded face region to obtain the sample occluded region features, thereby improving the feature extraction quality of the occluded face region. Please refer to [reference needed]. Figure 2 This illustrates a flowchart of a training method for a face recognition model provided in another exemplary embodiment of this application. The method may include the following steps:

[0070] Step 201: Extract features from the sample face image using the first feature extraction network to obtain the first sample face features.

[0071] The implementation method for this step can refer to step 102, and will not be described in detail here.

[0072] Indicative, such as Figure 3 As shown, the computer device extracts features from the sample face image 31 through the first feature extraction network 321 to obtain the first sample face features 322.

[0073] In some embodiments, the first sample facial feature is a two-dimensional feature that includes spatial location information and identity information, and can include occlusion information of occluded areas. It should be noted that the last fully convolutional layer and fully connected layer of the first feature extraction network are not used here because the last fully connected layer focuses more on identity information, while the features extracted by the last convolutional layer include occlusion information and identity information, but lose spatial location information.

[0074] Step 202: Based on the facial features of the first sample, the occlusion region is segmented through a segmentation sub-network to obtain the sample occlusion region image of the face occlusion region.

[0075] The second feature extraction network in this embodiment consists of a segmentation subnetwork and a feature extraction subnetwork. The segmentation subnetwork is used to segment the occluded region to obtain the sample occluded region image in the sample face image; the feature extraction subnetwork is used to extract features from the segmented sample occluded region image.

[0076] In some embodiments, the sample occlusion region image is the same size as the sample face image.

[0077] In some embodiments, the sample occlusion region image is a binary image, wherein pixels with a value of 1 correspond to occlusion regions, and pixels with a value of 0 should be non-occlusion regions.

[0078] Indicative, such as Figure 3 As shown, the computer device inputs the first sample face feature 322 into the segmentation sub-network 331 to obtain the sample occlusion region image 332, wherein the black area in the sample occlusion region image 332 corresponds to the face occlusion region.

[0079] Optionally, the segmentation sub-network can be FCN (Full Convolutional Networks), SegNet (Segment Networks), dilated convolutional networks, etc., and the embodiments of this application do not limit it to this.

[0080] It should be noted that in this embodiment, the face features of the first sample are directly used when segmenting the occluded area image of the sample, without the need to extract features from the face image of the sample, which improves the segmentation efficiency of the occluded area.

[0081] Step 203: Extract features from the sample occlusion area image through the feature extraction sub-network to obtain the sample occlusion area features.

[0082] Furthermore, the computer device inputs the sample occlusion region image into the feature extraction subnetwork to obtain the sample occlusion region features of the face occlusion region. These sample occlusion region features can characterize the spatial and contour features of the face occlusion region.

[0083] In some embodiments, to facilitate subsequent feature purification, the feature size of the sample occlusion region is consistent with the feature size of the first sample face feature.

[0084] Indicative, such as Figure 3 As shown, the computer device extracts features from the sample occlusion region image 332 through the feature extraction sub-network 333 to obtain the sample occlusion region features 334.

[0085] In one possible implementation, the feature extraction subnetwork and the segmentation subnetwork may adopt an encoding / decoding network structure, which is not limited in this embodiment of the application.

[0086] Step 204: Based on the features of the occluded region of the sample, perform feature purification processing on the face features of the first sample to obtain the face features of the second sample.

[0087] Since the feature size of the first sample face feature is the same as that of the sample occlusion region feature, in some embodiments, the computer device determines the second sample face feature based on the product of the sample occlusion region feature and the first sample face feature.

[0088] Indicative, such as Figure 3 As shown, the computer device multiplies the sample occlusion region feature 334 with the first sample face feature 322 to obtain the purified second sample face feature 34.

[0089] Step 205: Input the face features of the first sample, the face features of the second sample, and the occlusion region features of the sample into the multi-head attention layer of the ViT network. The multi-head attention layer is used to perform self-attention processing on the features input from each input head.

[0090] In some embodiments, the third feature extraction network employs a ViT (Vision Transformer) network. (Illustrative example, such as...) Figure 4 As shown, the attention module in the ViT network consists of a normalization layer 41, a multi-head attention layer 42, a normalization layer 43, and an MLP (Multi-Layer Perceptron) layer 44. The original input is added to the output of the multi-head attention layer 42 and then input to the normalization layer 43; the input to the normalization layer 43 is added to the output of the MLP layer 44 and then output as the final result.

[0091] In this embodiment of the application, the input of the attention module in the ViT network is the extracted first sample face features, second sample face features, and sample occlusion region features, and the output is the third sample face features.

[0092] In some embodiments, the input to the multi-head attention layer includes Q (Query), K (Key), and V (Value). The first sample face features, the second sample face features, and the sample occlusion region features are assigned to Q, K, and V of the multi-head attention layer, wherein Q, K, and V are all the same, or Q, K, and V are all different, or Q, K, and V are partially the same.

[0093] In one possible implementation, the first sample face feature, the second sample face feature, and the sample occlusion region feature are each assigned a value to QKV; or, the first sample face feature and the second sample face feature are simultaneously assigned a value to QKV; or, the first sample face feature and the second sample face feature are simultaneously assigned a value to QKV, and the sample occlusion region feature is simultaneously assigned a value to QK. This application does not limit the combination of values ​​assigned to the multi-head attention layer.

[0094] Step 206: Obtain the third sample face features output by the ViT network.

[0095] Indicative, such as Figure 3 As shown, the computer device inputs the first sample face feature 322, the sample occlusion region feature 334 and the second sample face feature 34 into the ViT network 35 to obtain the third sample face feature 36.

[0096] Step 207: Train a face recognition model based on the face features of the third sample. The face recognition model consists of a first feature extraction network, a second feature extraction network, and a third feature extraction network.

[0097] In some embodiments, the sample face images used for model training are all labeled with facial identity tags. For example... Figure 5 As shown, the process of training a model based on the extracted features and facial identity labels may include the following steps.

[0098] Step 207A: Determine the first feature distance based on the third sample face features corresponding to the sample face images with the same identity labels.

[0099] For sample face images with the same identity label, i.e., sample face images corresponding to the same object, the computer device calculates a first feature distance between the corresponding third sample face features of the sample face images. Optionally, this first feature distance can be an L1 distance or an L2 distance, which is not limited in this embodiment.

[0100] The larger the feature distance, the greater the difference in facial identity represented by the facial features of the third sample (i.e., the lower the probability of belonging to the same object); the smaller the feature distance, the smaller the difference in facial identity represented by the facial features of the third sample (i.e., the higher the probability of belonging to the same object).

[0101] In some embodiments, to reduce the influence of spatial information in the features, the computer device processes the third sample face features through a fully connected layer, thereby retaining only the identity information in the features, and then calculates the first feature distance between the processed third sample face features. (Illustrative example, such as...) Figure 3As shown, the computer device performs fully connected processing on the extracted third sample face features 36 to obtain face identity features 37 used to calculate feature distance.

[0102] Step 207B: Determine the second feature distance based on the third sample face features corresponding to sample face images with different identity labels.

[0103] For sample face images with different identity labels, i.e., sample face images corresponding to different objects, the computer device calculates a second feature distance between the sample face images and the corresponding third sample face features. Optionally, this second feature distance can be an L1 distance or an L2 distance, which is not limited in this embodiment.

[0104] Similar to the first feature distance calculation process, in some embodiments, the computer device processes the third sample face features through a fully connected layer, thereby retaining only the identity information in the features, and then calculates the second feature distance between the processed third sample face features.

[0105] Step 207C: Determine the face recognition loss based on the first feature distance and the second feature distance.

[0106] Furthermore, the computer device determines the face recognition loss based on the first feature distance and the second feature distance. Specifically, when training the model using the face recognition loss, the computer device aims to minimize the first feature distance and maximize the second feature distance.

[0107] Step 207D: Train a face identity recognition model based on face recognition loss.

[0108] In some embodiments, the computer device employs gradient descent or backpropagation algorithms to update the parameters of the first feature extraction network, the second feature extraction network, and the third feature extraction network. The computer device stops model training when training completion conditions are met (e.g., loss convergence or reaching the required number of training epochs).

[0109] In some embodiments, to accelerate training, when sample face images are labeled with face occlusion regions, the computer device can determine the occlusion region segmentation loss based on the sample occlusion region image and the face occlusion region label, and then use this loss as part of the model loss for model training.

[0110] In this embodiment, a segmentation approach is used to segment the face occlusion region, and features are extracted from the segmented face occlusion region. Then, the extracted face occlusion region features are used to refine the face features, which helps to reduce the interference of occlusion on the feature extraction of the non-occluded region and improve the accuracy of face features.

[0111] Furthermore, in this embodiment, multi-head attention is achieved using a ViT network. This improves attention to the unoccluded face region and allows for the extraction of contour features of the occluded region from the features of the occluded object, thereby further improving the quality of the extracted face features.

[0112] Please refer to Figure 6 This document illustrates a flowchart of a face recognition method provided in an exemplary embodiment of this application. The method may include the following steps:

[0113] Step 602: Extract features from the target face image using the first feature extraction network of the face recognition model to obtain the first face features.

[0114] The facial recognition model was trained using the model training method provided in the above embodiments.

[0115] In some embodiments, when the facial recognition model is applied to a smartphone, when identity verification is required via face (such as unlocking the phone, facial payment, etc.), the smartphone captures the target face image through the camera, runs the facial recognition model through the NPU, and extracts features from the target face image through the first feature extraction network to obtain the first face feature.

[0116] Since the acquired image may contain images other than faces, in some embodiments, before extracting face features, the computer device performs face detection on the target image to obtain face detection results, which are used to indicate the location of faces in the target image.

[0117] Furthermore, the computer device extracts the target face image from the target image based on the face detection results; then, based on the key point recognition results of the target face image, it performs an affine transformation on the target face image to obtain an aligned target face image.

[0118] Indicative, such as Figure 7 As shown, the computer device first performs face detection on the target image 71 using the face detection model 72, obtaining the face region 73 in the target image 71. The computer device then performs cropping and affine transformation on the face region 73 to obtain the target face image 74. In subsequent processes, the computer device performs identity recognition on the target face image 74 using the face recognition model 75 to obtain the identity recognition result.

[0119] Step 604: Based on the first face feature, extract the occlusion region feature of the occlusion region in the target face image through the second feature extraction network of the face identity recognition model.

[0120] When there are obstructions, the obstructions can cause some facial features to be missing, or even interfere with the facial features in the unobstructed areas, resulting in poor quality of the extracted facial features. If the extracted facial features are used directly for facial recognition, the accuracy of facial recognition will be poor.

[0121] In this embodiment, the computer device further extracts occlusion region features from the target face image using a second feature extraction network. Since the first face feature contains occlusion information, the computer device extracts features from the occluded face region based on the first face feature.

[0122] Step 606: Based on the occlusion region features, perform feature purification processing on the first face features to obtain the second face features, wherein the feature purification processing is used to eliminate interference from the occlusion region of the face.

[0123] In some embodiments, the computer device multiplies the occlusion region features with the first facial features to obtain the purified second facial features.

[0124] Step 608: Based on the first face feature, the second face feature, and the occlusion region feature, the third face feature is obtained by extracting features through the third feature extraction network of the face identity recognition model. The third feature extraction network adopts a self-attention mechanism.

[0125] Furthermore, the computer device inputs the first face feature, the second face feature, and the occlusion region feature into the third feature extraction network, which performs multi-head self-attention processing on the above features to obtain the final third face feature used for identity recognition.

[0126] Step 610: Perform facial identity recognition based on third-party facial features.

[0127] In some embodiments, the computer device pre-stores the target facial features of the target object. The computer device determines whether the face in the target face image is the face of the target object by comparing the target facial features with third facial features.

[0128] In summary, in the application of the facial recognition model in this embodiment, features are extracted from the occluded area of ​​the face, and the extracted occluded area features are used to refine the facial features. Then, a self-attention mechanism is used to further extract features from the unrefined facial features, the refined facial features, and the occluded area features. Finally, the extracted facial features are used for facial recognition, which can eliminate the interference caused by facial occlusion to the recognition and help improve the accuracy of facial recognition.

[0129] Please refer to Figure 8This diagram illustrates a structural block diagram of a training apparatus for a face recognition model according to an embodiment of this application. The apparatus includes:

[0130] The feature extraction module 801 is used to extract features from the sample face image through the first feature extraction network to obtain the first sample face features;

[0131] The feature extraction module 801 is used to extract the sample occlusion region features of the face occlusion region in the sample face image based on the first sample face features and through the second feature extraction network.

[0132] The feature extraction module 801 is used to perform feature purification processing on the first sample face features based on the features of the sample occlusion region to obtain the second sample face features, wherein the feature purification processing is used to eliminate the interference of the face occlusion region.

[0133] The feature extraction module 801 is used to extract features based on the first sample face features, the second sample face features, and the sample occlusion region features, through a third feature extraction network to obtain the third sample face features. The third feature extraction network adopts a self-attention mechanism.

[0134] Training module 802 is used to train a face recognition model based on the face features of the third sample. The face recognition model is composed of the first feature extraction network, the second feature extraction network and the third feature extraction network.

[0135] Optionally, the second feature extraction network consists of a segmentation subnetwork and a feature extraction subnetwork;

[0136] The feature extraction module 801 is used for:

[0137] Based on the first sample face features, the occlusion region is segmented through the segmentation sub-network to obtain a sample occlusion region image of the face occlusion region.

[0138] The feature extraction subnetwork is used to extract features from the image of the sample occluded region to obtain the features of the sample occluded region.

[0139] Optionally, the sample occlusion region image has the same size as the sample face image, and the feature size of the sample occlusion region is the same as the feature size of the first sample face feature.

[0140] Optionally, the third feature extraction network is a ViT network;

[0141] The feature extraction module 801 is used for:

[0142] The first sample face features, the second sample face features, and the sample occlusion region features are input into the multi-head attention layer of the ViT network. The multi-head attention layer is used to perform self-attention processing on the features input from each input head.

[0143] Obtain the third sample face features output by the ViT network.

[0144] Optionally, the first sample face features, the second sample face features, and the sample occlusion region features are assigned to Q, K, and V of the multi-head attention layer, wherein Q, K, and V are all the same, or Q, K, and V are all different, or Q, K, and V are partially the same.

[0145] Optionally, the feature extraction module 801 is used for:

[0146] The second sample face features are determined based on the product of the occlusion region features of the sample and the face features of the first sample.

[0147] Optionally, the training module 802 is used for:

[0148] Based on the third sample face features corresponding to the sample face images with the same identity labels, a first feature distance is determined;

[0149] The second feature distance is determined based on the third sample face features corresponding to the sample face images with different identity labels;

[0150] The face recognition loss is determined based on the first feature distance and the second feature distance;

[0151] The facial recognition model is trained based on the facial recognition loss.

[0152] Optionally, the device further includes:

[0153] The preprocessing module is used to perform face detection on the sample image and obtain face detection results, which are used to indicate the position of the face in the sample image.

[0154] Extract the sample face image from the sample image based on the face detection results;

[0155] Based on the key point recognition results of the sample face image, an affine transformation is performed on the sample face image to obtain the aligned sample face image.

[0156] Please refer to Figure 9 This diagram illustrates a structural block diagram of a facial recognition device according to an embodiment of this application. The device includes:

[0157] The feature extraction module 901 is used to extract features from the target face image through the first feature extraction network of the face recognition model to obtain the first face features;

[0158] The feature extraction module 901 is used to extract the occlusion region features of the occlusion region in the target face image based on the first face features and through the second feature extraction network of the face identity recognition model.

[0159] The feature extraction module 901 is used to perform feature purification processing on the first face feature based on the occlusion region feature to obtain the second face feature, wherein the feature purification processing is used to eliminate the interference of the face occlusion region.

[0160] The feature extraction module 901 is used to extract features based on the first face features, the second face features, and the occlusion region features, through the third feature extraction network of the face recognition model to obtain the third face features. The third feature extraction network adopts a self-attention mechanism.

[0161] The identification module 902 is used to perform facial identity recognition based on the third facial features;

[0162] The facial recognition model is trained using the device described above.

[0163] Optionally, the device further includes:

[0164] The preprocessing module is used to perform face detection on the target image and obtain face detection results, which are used to indicate the position of faces in the target image.

[0165] Extract the target face image from the target image based on the face detection results;

[0166] Based on the key point recognition results of the target face image, an affine transformation is performed on the target face image to obtain the aligned target face image.

[0167] Please refer to Figure 10 This illustration shows a structural block diagram of a computer device provided in an exemplary embodiment of this application. The computer device may include one or more components such as a processor 1210 and a memory 1220.

[0168] Processor 1210 may include one or more processing cores. Processor 1210 connects to various parts of the electronic device using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 1220, and by calling data stored in memory 1220. Optionally, processor 1210 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). Processor 1210 may integrate one or more of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), Neural-network Processing Unit (NPU), and modem. Specifically, the CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required to be displayed on the touch screen; the NPU is used to implement Artificial Intelligence (AI) functions; and the modem is used to handle wireless communication. It is understandable that the aforementioned modem may not be integrated into the processor 1210, but may be implemented as a separate chip.

[0169] The memory 1220 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 1220 may include a non-transitory computer-readable storage medium. The memory 1220 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 1220 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the various method embodiments described below, etc.; the data storage area may store data created according to the use of the computer device (such as audio data, telephone directory, etc.).

[0170] In addition, those skilled in the art will understand that the structure of the computer device shown in the above figures does not constitute a limitation on the computer device. The computer device may include more or fewer components than shown, or combine certain components, or have different component arrangements. For example, the computer device also includes radio frequency circuits, input units, sensors, audio circuits, speakers, microphones, power supplies, etc., which will not be described in detail here.

[0171] This application also provides a computer-readable storage medium storing at least one instruction, which is executed by a processor to implement the training method for the face recognition model or the face recognition method as described in the above embodiments.

[0172] This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the training method for the face recognition model or the face recognition method provided in the above embodiments.

[0173] Those skilled in the art will recognize that the functions described in the embodiments of this application in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of a computer program from one place to another. Storage media can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0174] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A training method for a facial recognition model, characterized in that, The method includes: The first feature extraction network is used to extract features from the sample face image to obtain the first sample face features; Based on the first sample face features, the second feature extraction network extracts the sample occlusion region features of the face occlusion region in the sample face image; Based on the features of the occlusion region of the sample, the first sample face features are subjected to feature purification processing to obtain the second sample face features, wherein the feature purification processing is used to eliminate the interference of the face occlusion region. Based on the face features of the first sample, the face features of the second sample, and the occlusion region features of the sample, features are extracted through a third feature extraction network to obtain the face features of the third sample. The third feature extraction network adopts a self-attention mechanism. A face recognition model is trained based on the face features of the third sample. The face recognition model consists of the first feature extraction network, the second feature extraction network, and the third feature extraction network.

2. The method according to claim 1, characterized in that, The second feature extraction network consists of a segmentation subnetwork and a feature extraction subnetwork; The step of extracting sample occlusion region features of the face occlusion region in the sample face image based on the first sample face features and using a second feature extraction network includes: Based on the first sample face features, the occlusion region is segmented through the segmentation sub-network to obtain a sample occlusion region image of the face occlusion region. The feature extraction subnetwork is used to extract features from the image of the sample occluded region to obtain the features of the sample occluded region.

3. The method according to claim 2, characterized in that, The sample occlusion region image has the same size as the sample face image, and the feature size of the sample occlusion region is the same as the feature size of the first sample face feature.

4. The method according to claim 1, characterized in that, The third feature extraction network is a ViT network; The step of extracting features from the first sample face features, the second sample face features, and the occlusion region features of the samples using a third feature extraction network to obtain the third sample face features includes: The first sample face features, the second sample face features, and the sample occlusion region features are input into the multi-head attention layer of the ViT network. The multi-head attention layer is used to perform self-attention processing on the features input from each input head. Obtain the third sample face features output by the ViT network.

5. The method according to claim 4, characterized in that, The first sample face features, the second sample face features, and the sample occlusion region features are assigned to Q, K, and V of the multi-head attention layer, wherein Q, K, and V are all the same, or Q, K, and V are all different, or Q, K, and V are partially the same.

6. The method according to claim 1, characterized in that, The step of refining the features of the first sample face features based on the occlusion region features to obtain the second sample face features includes: The second sample face features are determined based on the product of the occlusion region features of the sample and the face features of the first sample.

7. The method according to claim 1, characterized in that, The training of the facial recognition model based on the facial features of the third sample includes: Based on the facial features of the third sample corresponding to the sample facial images with the same identity label, a first feature distance is determined; The second feature distance is determined based on the third sample face features corresponding to the sample face images with different identity labels; The face recognition loss is determined based on the first feature distance and the second feature distance; The facial recognition model is trained based on the facial recognition loss.

8. The method according to any one of claims 1 to 7, characterized in that, The method further includes: Face detection is performed on the sample image to obtain face detection results, which are used to indicate the location of faces in the sample image; Extract the sample face image from the sample image based on the face detection results; Based on the key point recognition results of the sample face image, an affine transformation is performed on the sample face image to obtain the aligned sample face image.

9. A facial recognition method, characterized in that, The method includes: The first feature extraction network of the face recognition model is used to extract features from the target face image to obtain the first face feature; Based on the first facial features, the occlusion region features of the occlusion region in the target facial image are extracted through the second feature extraction network of the facial identity recognition model; The first face feature is purified based on the occlusion region features to obtain the second face feature, wherein the feature purification process is used to eliminate interference from the occlusion region of the face. Based on the first face feature, the second face feature, and the occlusion region feature, the third face feature is obtained by extracting features through the third feature extraction network of the face recognition model. The third feature extraction network adopts a self-attention mechanism. Facial identity recognition is performed based on the aforementioned third facial features; The facial recognition model is trained using the method described in any one of claims 1 to 8.

10. The method according to claim 9, characterized in that, The method further includes: Face detection is performed on the target image to obtain face detection results, which are used to indicate the position of faces in the target image; Extract the target face image from the target image based on the face detection results; Based on the key point recognition results of the target face image, an affine transformation is performed on the target face image to obtain the aligned target face image.

11. A training device for a facial recognition model, characterized in that, The device includes: The feature extraction module is used to extract features from the sample face image through the first feature extraction network to obtain the first sample face features; The feature extraction module is used to extract the sample occlusion region features of the face occlusion region in the sample face image based on the first sample face features and through the second feature extraction network. The feature extraction module is used to perform feature purification processing on the first sample face features based on the features of the sample occlusion region to obtain the second sample face features, wherein the feature purification processing is used to eliminate the interference of the face occlusion region. The feature extraction module is used to extract features based on the first sample face features, the second sample face features, and the sample occlusion region features, through a third feature extraction network to obtain the third sample face features. The third feature extraction network adopts a self-attention mechanism. The training module is used to train a face recognition model based on the face features of the third sample. The face recognition model consists of the first feature extraction network, the second feature extraction network, and the third feature extraction network.

12. A facial recognition device, characterized in that, The device includes: The feature extraction module is used to extract features from the target face image through the first feature extraction network of the face recognition model to obtain the first face features; The feature extraction module is used to extract the occlusion region features of the occlusion region in the target face image based on the first face features and through the second feature extraction network of the face recognition model. The feature extraction module is used to perform feature purification processing on the first face feature based on the occlusion region feature to obtain the second face feature, wherein the feature purification processing is used to eliminate the interference of the face occlusion region. The feature extraction module is used to extract features based on the first face feature, the second face feature, and the occlusion region feature, through the third feature extraction network of the face recognition model to obtain the third face feature. The third feature extraction network adopts a self-attention mechanism. The identification module is used to perform facial identity recognition based on the third facial features; The facial recognition model is trained using the device described in claim 11.

13. A computer device, characterized in that, The computer device includes a processor and a memory; the memory stores at least one instruction, which is executed by the processor to implement the training method of the face recognition model as described in any one of claims 1 to 8, or to implement the face recognition method as described in claim 9 or 10.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one piece of program code, which is loaded and executed by a processor to implement the training method of the face recognition model as described in any one of claims 1 to 8, or to implement the face recognition method as described in claim 9 or 10.

15. A computer program product, characterized in that, The computer program product includes computer instructions stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to perform a training method for a face recognition model as described in any one of claims 1 to 8, or to perform a face recognition method as described in claim 9 or 10.

Images