A face recognition method for people wearing masks based on multi-branch network and image restoration

By constructing a face recognition data set for wearing masks and using five-branch networks and face repair technology, the problem of low facial recognition accuracy in wearing masks is solved, and the recognition accuracy rate of more than 80% is achieved.

CN113822157BInactive Publication Date: 2025-02-28BEIJING UNIV OF TECH
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202110952447.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-08-19
Publication Date
2025-02-28
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing facial recognition system has significantly reduced its recognition accuracy when facing faces wearing masks, especially when the mask covers a large proportion, and it is almost impossible to recognize.

Method used

A new method for constructing face recognition data sets with masks is proposed, using unobstructed face images to generate face images with masks, and through five-branch face recognition network (QMFRN) and multi-branch network based on metric learning, combined with face repair technology, occlusion information is fully utilized to improve recognition rate.

Benefits of technology

Through this method, the accuracy of facial recognition of masks with more than 80% is achieved, which significantly improves the accuracy of facial recognition of masks with masks.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113822157B_ABST
    Figure CN113822157B_ABST
Patent Text Reader

Abstract

A method for face recognition with masks based on a multi-branch network and image restoration belongs to the field of occluded face recognition. The existence of occlusion itself has a greater impact on the face recognition effect than the features lost by the occlusion, so it is usually discarded. The present invention proposes to use the features of occlusion information to improve the recognition rate. The present invention proposes a five-branch face recognition network QMFRN with masks, designs a new loss function, takes the features of the face image with masks as the elements of the loss function, and uses the features of the mask to reduce the inter-class distance and increase the inter-class distance. First, the unobstructed face image is used to generate the face image with masks, and then the face image with masks is used to generate the repaired unobstructed face image. The construction of this data set only needs to collect unobstructed faces, and does not need to collect faces with masks, so the data acquisition is very convenient.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the field of occluded face recognition and proposes a face recognition method for a person wearing a mask. Background Art

[0002] Facial recognition technology is of great significance to maintaining social order and public safety, and can be completed without the active cooperation of the object to be identified. The industry already has many mature facial recognition algorithm models that have reached commercial standards, and the facial recognition rate under ideal conditions is close to 100%. Due to the low quality requirements for facial images and low acquisition costs, facial recognition systems have been widely used in many scenarios in daily life.

[0003] However, the face images collected under real conditions may be occluded, resulting in the loss of facial features, which greatly reduces the accuracy of existing face recognition systems. Existing experiments show that the greater the proportion of face occlusion, the lower the accuracy of face recognition. When using cosine similarity to classify and identify the extracted facial features, if the occlusion ratio exceeds 50%, the recognition rate will drop to 60%; if the occlusion ratio exceeds 75%, it is almost impossible to recognize.

[0004] The research on occluded face recognition is currently in its early stages and has great potential in practical applications. In special periods such as the COVID-19 pandemic, in order to reduce people's contact and reduce the possibility of COVID-19 transmission, masks must be worn when going out, and this kind of occlusion is very common. People need to take off their masks when passing through the face recognition system, which increases the risk of infection. However, when the mask is not taken off, the occlusion ratio of the mask is generally between 30-40%, which is a large occlusion. Existing face recognition systems usually have difficulty in correctly identifying faces, so we need to study face recognition technology for people wearing masks.

[0005] The current occluded face recognition technology usually finds the occluded area first and removes the occlusion, and then recognizes part of the face. This method eliminates the interference of occlusion on facial features, but part of the facial features are still lost, and the accuracy is not improved enough. Another novel idea is to regard the occluded face recognition problem as an image restoration problem, repair the missing facial features, and use the restored face image for face recognition, but the accuracy of this method still needs to be improved. Summary of the invention

[0006] The present invention proposes a new method for constructing a mask-wearing face recognition dataset. It can be automatically constructed by simply inputting an unobstructed face. First, an unobstructed face image is used to generate a mask-wearing face image, and then a repaired unobstructed face image is generated using the mask-wearing face image. The construction of this dataset only requires the collection of unobstructed faces, and does not require the collection of faces wearing masks, so data acquisition is very convenient.

[0007] The existence of occlusion itself has a greater impact on the face recognition effect than the features lost by the occlusion, so it is usually discarded. The patent of this invention does not simply discard the occlusion, but proposes to use the features of the occlusion information to improve the recognition rate. The present invention proposes a five-branch masked face recognition network QMFRN (quintuplets masked face recognition network), designs a new loss function, takes the features of the masked face image as the elements of the loss function, and uses the features of the mask to reduce the distance between classes and increase the distance between classes.

[0008] The invention patent also combines the multi-branch network based on metric learning and the face recognition method based on face restoration for the first time, and uses the face restoration image as an unobstructed positive example of the multi-branch network. The invention patent also provides a method for verifying the effect of the network, using the face restoration and recognition network to verify the recognition results of the multi-branch network.

[0009] The steps of the face recognition method for wearing a mask proposed by the present invention are as follows:

[0010] Step 1: Input the unobstructed face dataset and generate the masked face dataset

[0011] It includes the following sub-steps:

[0012] Step 1.1: Use the deep alignment network DAN to detect facial key points.

[0013] Step 1.2: Calculate the size of the mask based on the key points and calculate the offset angle using the vertical line formed by the bridge of the nose and the chin.

[0014] Step 1.3: Scale the mask image to the corresponding size, add it to the area corresponding to the face, and rotate it to the corresponding angle.

[0015] Step 2: Use the masked face dataset to generate a repaired face dataset.

[0016] It includes the following sub-steps:

[0017] Step 2.1: Use the object detection and localization method to find the mask on the face, remove the mask, treat it as the missing area, and fill the area with a color.

[0018] Step 2.2: Use the EdgeConnect model to perform face restoration on the image processed in step 2.1.

[0019] Step 3: Use the unobstructed face images, the masked face images, and the restored face images after removing the mask as the training set for occluded face recognition.

[0020] Step 4: Use the data set in step 3 to construct a quintuple, propose a metric loss function based on the quintuple, and use the loss function to train a five-branch masked face recognition network with shared parameters. The backbone network can be ResNet or VGG.

[0021] It includes the following sub-steps:

[0022] Step 4.1: Take an unobstructed image in the dataset as the anchor, take the restored face image of the same person after removing the mask as the PU (unobstructed positive sample), and take the face image of the same person with a mask as the PM (occluded positive sample); take the face image of another person after removing the mask as the NU (unobstructed negative sample), and the face image of the same person with a mask as the NM (occluded negative sample); these five images constitute a set of five-tuples.

[0023] Step 4.2: Train a parameter-sharing five-branch face recognition network for wearing masks, taking the five-tuple as input, and each branch gets its own embedding, which is recorded as f(Anchor), f(PU), f(PM), f(NU), and f(NM). The loss function proposed by the present invention is as follows:

[0024]

[0025] m is the minimum distance between the distance between the anchor and the positive example and the distance between the anchor and the negative example. When the difference between the distance between the anchor and the negative example and the distance between the anchor and the positive example is less than m, the loss is 0; when the difference between the distance between the anchor and the negative example and the distance between the anchor and the positive example is greater than m, the loss is the value in brackets.

[0026] These embeddings are closer to the embeddings from the same identity in the latent space than the embeddings from different identities. Instead of simply discarding the occlusion, the network makes full use of the information of the occluder mask to help recognize the face, improving the recognition rate of the face wearing the mask.

[0027] Step 5: Use the unobstructed face images as a dataset and use the FaceNet model to train an unobstructed face detection network.

[0028] Step 6: Use the QMFRN network trained in step 2 to identify the face wearing a mask. Use the face restoration network EdgeConnect and the unobstructed face detection network FaceNet to identify the face wearing a mask to verify whether the recognition result of the QMFRN network is correct. The recognition rate of the present invention is above 80%. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 :The present invention constructs a process flow diagram of a face recognition dataset for people wearing masks

[0030] Figure 2 : Schematic diagram of the steps of the face recognition method wearing a mask proposed by the present invention. DETAILED DESCRIPTION

[0031] In order to facilitate non-technical personnel to better understand the method proposed in the present invention, the present invention is described in detail below with reference to examples and drawings. The following description describes many specific details, but is only for the convenience of example and is not intended to limit the scope of protection of the present invention.

[0032] (1) The input is an unobstructed face dataset. First, the deep alignment network DAN is used to detect facial key points. The key point detection process is as follows:

[0033] Initially, the grayscale image I of the original image is input into the feedforward neural network to obtain the offset, and added to the initial evaluation S0 of the key point to predict the new key point position S1. The transformation matrix T2 from S0 to S1 is calculated. Then it enters the convolutional network to obtain the transformed image T2(I), the key point heat map H2 and the output F2 of the fc1 layer of the feedforward network. These three elements are used as the input of the feedforward network in the next stage.

[0034] The key point heat map is obtained by the following formula:

[0035]

[0036] Where (x, y) is the position of the pixel, and si is the coordinate of the i-th key point of Tt(St-1). The closer the pixel position is to the key point, the larger the value of H(x, y), and the farther away, the smaller the value.

[0037] Then we enter the iteration phase, add Tt(St-1) to the output of the feedforward network, and then do an inverse transformation of Tt to get St. Then calculate the transformation matrix Tt+1 from St-1 to St. Finally, St is input into the convolutional network, and the transformed image Tt+1(I), the key point heat map Ht+1 and the output Ft+1 of the fc1 layer are calculated as the input of the next stage.

[0038] The transformed image Tt+1(I), the key point heat map Ht+1 and the output feature Ft+1 of the fully connected layer fc1 are taken as the input of the next stage to calculate the new St+1.

[0039] After the iteration is completed, 68 facial key points are obtained. Then the Euclidean distance between key points 2 and 16 on both sides of the face is used to calculate the width of the mask, the key point 28 at the bridge of the nose and the key point 9 directly below the face are used to calculate the height of the mask, and the offset angle of the mask is calculated in combination with the vertical line. Finally, the mask image is scaled to the corresponding size and added to the corresponding area of ​​the face, and rotated by the corresponding angle to obtain the face image wearing the mask.

[0040] (2) Remove the mask from the face image of the person wearing the mask, fill the area with white, and record the image as Igt. Use the EdgeConnect model to perform face restoration on the image Igt, which is divided into two parts: edge image generation and image completion.

[0041] Edge map generation: The image mask is M (missing areas are 1, non-missing areas are 0), the grayscale image of the image is Igray, and the initial edge map Cgt is calculated using the Canny operator. The Canny algorithm is currently the most effective edge detection algorithm, which uses multiple stages to complete edge detection.

[0042] The first step of Canny edge detection is to use Gaussian filtering to reduce noise. Use the Gaussian kernel to scan the entire image, multiply the image pixel by the corresponding element of the Gaussian kernel each time, and then sum it up. The sum is used as the pixel at the position corresponding to the center of the Gaussian kernel. The Gaussian kernel is obtained by the following formula:

[0043]

[0044] (x, y) is the coordinate of the pixel point of the Gaussian kernel, and the coordinate of the center point is (0, 0). σ is the variance of the Gaussian function, which is used to control the degree of filtering. For such images, when σ is 1.2-1.5, there are few false edges and the detected edges are close to the actual edges, and the edge detection effect is optimal. Within this range, σ can take any value. After calculating the Gaussian kernel, the Gaussian kernel is normalized.

[0045] The second step is to calculate the gradient. Use the Soble horizontal operator and vertical operator to scan the entire image, perform a one-dimensional convolution operation with the input image, and obtain dx and dy. Use dx and dy to calculate the magnitude and angle of the gradient. Finally, the gradient value and angle of each pixel are obtained.

[0046] The third step is to perform non-maximum suppression. The calculated gradient becomes coarser, so it is necessary to perform non-maximum suppression on the amplitude along the gradient direction. The Canny algorithm only sets four axes: horizontal, vertical, positive 45°, and negative 45°. The gradient only has eight directions: 0, 45, 135, 180, 225, 270, and 315. The pixel's gradient angle is close to the axis, and it is compared with the two nearest points on this axis. If the center pixel is the local maximum, it is considered to be an edge pixel, otherwise it is set to 0.

[0047] The fourth step is to use double threshold selection for further screening, setting a high threshold high and a low threshold low. Points less than the low threshold are directly discarded, and points greater than the high threshold are edge points. If the point in the middle of the double threshold is connected to an edge point, then this point is an edge point, otherwise it is discarded. When the threshold is set too high, some detailed edges will be lost; when the threshold is set too low, some messy and irrelevant edges will also be retained. For this type of image, the best value of low is 100-120, and the value of high is 3 times that of low. Any value can be taken within this range.

[0048] After using the canny algorithm to obtain the edge map Cgt, the masked grayscale map I'gray is obtained by the Hadamard product of Igray and (1-M), and the masked edge map is obtained by the Hadamard product of C'gt and (1-M). I'gray, C'gt and M are input into the generator G1 to obtain the predicted edge map Cpred. Cgt and Cpred conditioned on Igray are used as the input of the discriminator D1 to predict whether the edge map is real.

[0049] Image completion: The incomplete color image I`gt is obtained by the Hadamard product of Igt and (1-M), and the composite edge graph Ccomp is obtained by the Hadamard product of Cpred and M plus C`gt. I`gt and Ccomp are input into the generator G2 to obtain the repaired image Ipred.

[0050] (3) Unobstructed face images, masked face images, and restored face images after removing masks are used as training sets for occluded face recognition.

[0051] (4) Use the training set in (3) to construct a quintuple, propose a metric loss function based on the quintuple, and use the loss function to train a five-branch mask face recognition network with shared parameters, where each branch shares the same network structure. The network uses Resnet50 as the backbone network. First, the last FC layer of Resnet50 is removed, and a Dropout layer is added to reduce overfitting. Then an adaptive mean pooling layer is input to reduce parameters and speed up training. Then the flatten function is used to compress the 2D feature map into one dimension. Then a linear layer is input to convert the input 2048-bit embeddings into 512-bit embeddings.

[0052] It includes the following sub-steps:

[0053] Step 1: Take an unobstructed image in the dataset as the anchor, take the restored face image of the same person after removing the mask as the PU (unobstructed positive sample), and take the face image of the same person with a mask as the PM (occluded positive sample); take the restored face images of different people after removing the mask as the NU (unobstructed negative sample), and their face images with masks as the NM (occluded negative sample); these five images constitute a set of quintuples.

[0054] Step 2: Train a parameter-sharing five-branch face recognition network for wearing masks, using Resnet50 as the backbone network. The structure of the network takes a five-tuple as input, and each branch gets its own embedding, which is recorded as f(Anchor), f(PU), f(PM), f(NU), and f(NM). The loss function proposed in the present invention is as follows:

[0055]

[0056] m is the minimum distance between the distance between the anchor and the positive example and the distance between the anchor and the negative example. When the difference between the distance between the anchor and the negative example and the distance between the anchor and the positive example is less than or equal to m, the loss is 0; when the difference between the distance between the anchor and the negative example and the distance between the anchor and the positive example is greater than m, the loss is the value in brackets.

[0057] These embeddings are closer to the embeddings from the same identity in the latent space than the embeddings from different identities. Instead of simply discarding the occlusion, the network makes full use of the information of the occluder mask to help recognize the face, improving the recognition rate of the face wearing the mask.

[0058] (5) Use the unobstructed face images as a dataset and use the FaceNet model to train an unobstructed face detection network.

[0059] The input of the FaceNet model is a triplet of images, namely anchor, positive and negative. Positive and anchor are two images of the same person, while negative is an image of another person. Each triplet enters a backbone network (GoogLeNet is selected as the backbone network here) to output 128-dimensional features, and then L2 regularization is performed to obtain the embeddings of the anchor, positive and negative facial features, respectively denoted as f(a), f(p) and f(n). Finally, f(a), f(p) and f(n) are used as parameters and trained using the triplet loss function. The goal of training is to minimize the loss function. The formula of the triplet loss function is as follows:

[0060]

[0061] α is the minimum distance between the distance between the anchor and the positive and the distance between the anchor and the negative. When the difference between the distance between the anchor and the negative and the distance between the anchor and the positive is less than or equal to α, the loss is 0; when the difference between the distance between the anchor and the negative and the distance between the anchor and the positive is greater than α, the loss is the value in brackets.

[0062] (6) Use the trained QMFRN network to identify the face Im wearing a mask, and the recognition result is Rq. Input Im into the face restoration network EdgeConnect to obtain the restored image Ip, and then use the unobstructed face detection network FaceNet to perform face recognition on Ip to obtain the recognition result Rf, which is used to verify whether the recognition result Rq of the QMFRN network in step 1 is correct.

Claims

1. A face recognition method for people wearing masks based on a multi-branch network and image restoration, characterized in that The following steps are involved: Step 1: Input the unobstructed face dataset and generate the masked face dataset It includes the following sub-steps: Step 1.1: Use the deep alignment network DAN to detect facial key points; Step 1.2: Calculate the size of the mask based on the key points, and calculate the offset angle using the vertical line formed by the bridge of the nose and the chin; Step 1.3: Scale the mask image to the corresponding size, add it to the area corresponding to the face, and rotate it to the corresponding angle; Step 2: Use the masked face dataset to generate a repaired face dataset; It includes the following sub-steps: Step 2.1: Use the object detection and positioning method to find the mask on the face, remove the mask, treat it as the missing area, and fill the area with a color; Step 2.2: Use the EdgeConnect model to perform face restoration on the image processed in step 2.1; Step 3: Use the unobstructed face images, the masked face images, and the restored face images after removing the mask as the training set for occluded face recognition; Step 4: Use the data set in step 3 to construct a quintuple, propose a metric loss function based on the quintuple, and use the loss function to train a five-branch mask face recognition network with shared parameters, and select resnet or vgg as the backbone network; It includes the following sub-steps: Step 4.1: Take an unobstructed image in the dataset as the anchor, take the restored face image of the same person after removing the mask as the PU (unobstructed positive sample), and take the face image of the same person wearing a mask as the PM (occluded positive sample); take the restored face image of another person after removing the mask as the NU (unobstructed negative sample), and the face image of the same person wearing a mask as the NM (occluded negative sample); these five images constitute a set of five-tuples; Step 4.2: Train a parameter-sharing five-branch face recognition network for wearing masks, taking the five-tuple as input. Each branch gets its own embedding, which are recorded as f(Anchor), f(PU), f(PM), f(NU), and f(NM). The loss function is as follows: m is the minimum distance between the distance between the anchor and the positive example and the distance between the anchor and the negative example; when the difference between the distance between the anchor and the negative example and the distance between the anchor and the positive example is less than m, the loss is 0; when the difference between the distance between the anchor and the negative example and the distance between the anchor and the positive example is greater than m, the loss is the value in brackets; Step 5: Use the unobstructed face images as a dataset and use the FaceNet model to train an unobstructed face detection network; Step 6: Use the QMFRN network trained in step 2 to recognize faces wearing masks.

2. The method according to claim 1, characterized in that: (1) The input is an unobstructed face dataset. First, the deep alignment network DAN is used to detect facial key points. The key point detection process is as follows: Initially, the grayscale image I of the original image is input into the feedforward neural network to obtain the offset, and added to the initial evaluation S0 of the key point to predict the new key point position S1; the transformation matrix T2 from S0 to S1 is calculated; then it enters the convolutional network to obtain the transformed image T2(I), the key point heat map H2 and the output F2 of the fc1 layer of the feedforward network; these three elements are used as the input of the feedforward network in the next stage; The key point heat map is obtained by the following formula: Where (x, y) is the position of the pixel, si is the coordinate of the i-th key point of Tt(St-1); Then we enter the iteration phase, add Tt(St-1) to the output of the feedforward network, and then do an inverse transformation of Tt to get St; Then the transformation matrix Tt+1 from St-1 to St is calculated; finally, St is input into the convolutional network to calculate the transformed image Tt+1(I), the key point heat map Ht+1 and the output Ft+1 of the fc1 layer as the input of the next stage; The transformed image Tt+1(I), the key point heat map Ht+1 and the output feature Ft+1 of the fully connected layer fc1 are used as the input of the next stage to calculate the new St+1; After the iteration is completed, 68 facial key points are obtained; then the Euclidean distance between key points 2 and 16 on both sides of the face is used to calculate the width of the mask, the key point 28 at the bridge of the nose and the key point 9 directly below the face are used to calculate the height of the mask, and the offset angle of the mask is calculated in combination with the vertical line; the mask image is scaled to the corresponding size and added to the corresponding area of ​​the face, and rotated by the corresponding angle to obtain the face image wearing the mask; (2) Remove the mask from the face image of the person wearing the mask, fill the area with white, and record the image as Igt; Use the EdgeConnect model to perform face restoration on the image Igt, which is divided into two parts: edge image generation and image completion; Edge map generation: the image mask is M, the missing area is 1, and the non-missing area is 0; the grayscale image of the image is Igray, and the canny operator is used to calculate the initial edge map Cgt; The first step of Canny edge detection is to use Gaussian filtering to reduce noise; scan the entire image with a Gaussian kernel, multiply the image pixel by the corresponding element of the Gaussian kernel each time, and sum the sum as the pixel at the position corresponding to the center of the Gaussian kernel; the Gaussian kernel is obtained by the following formula: (x, y) is the coordinate of the pixel point of the Gaussian kernel, and the coordinate of the center point is (0, 0); σ is the variance of the Gaussian function, which is used to control the degree of filtering; for such images, when σ is 1.2-1.5, there are few false edges and the detected edges are close to the actual edges, and the edge detection effect is optimal. Within this range, σ can take any value; after calculating the Gaussian kernel, the Gaussian kernel is normalized; The second step is to calculate the gradient. Use the Soble horizontal operator and vertical operator to scan the entire image, perform a one-dimensional convolution operation with the input image, and obtain dx and dy. Use dx and dy to calculate the magnitude and angle of the gradient, and obtain the gradient value and angle of each pixel. The third step is to perform non-maximum suppression. The calculated gradient becomes coarser, so it is necessary to perform non-maximum suppression on the amplitude along the gradient direction. The canny algorithm only sets four axes: horizontal, vertical, positive 45°, and negative 45°. The gradient has only eight directions: 0, 45, 135, 180, 225, 270, and 315. The pixel gradient angle is close to which axis, and is compared with the two nearest points on this axis. If the center pixel is the local maximum, it is considered to be an edge pixel, otherwise it is set to 0. The fourth step is to use double threshold selection for further screening, setting a high threshold high and a low threshold low; points less than the low threshold are directly discarded, and points greater than the high threshold are edge points; for this type of image, the best value of low is 100-120, and the value of high is 3 times of low. After using the canny algorithm to obtain the edge map Cgt, the masked grayscale map I`gray is obtained by the Hadamard product of Igray and (1-M), and the masked edge map is obtained by the Hadamard product of C`gt and (1-M); I`gray, C`gt and M are input into the generator G1 to obtain the predicted edge map Cpred; Cgt and Cpred conditioned on Igray are used as the input of the discriminator D1 to predict whether the edge map is real; Image completion: The incomplete color image I`gt is obtained by the Hadamard product of Igt and (1-M), and the composite edge graph Ccomp is obtained by the Hadamard product of Cpred and M plus C`gt; I`gt and Ccomp are input into the generator G2 to obtain the repaired image Ipred; (3) Unobstructed face images, masked face images, and restored face images after removing masks are used as training sets for occluded face recognition; (4) Use the training set in (3) to construct a quintuple, propose a metric loss function based on the quintuple, and use the loss function to train a five-branch mask face recognition network with shared parameters, where each branch shares the same network structure; the network uses Resnet50 as the backbone network, first removes the last FC layer of Resnet50, and adds a Dropout layer to reduce overfitting; then inputs an adaptive mean pooling layer to reduce parameters and speed up training; then uses the flatten function to compress the 2D feature map into one dimension; then inputs a linear layer to convert the input 2048-bit embeddings into 512-bit embeddings; It includes the following sub-steps: Step 1: Take an unobstructed image in the dataset as the anchor, take the restored face image of the same person after removing the mask as the PU (unobstructed positive sample), and take the face image of the same person wearing a mask as the PM (occluded positive sample); take the restored face images of different people after removing the mask as the NU (unobstructed negative sample), and their face images wearing masks as the NM (occluded negative sample); these five images constitute a set of five-tuples; Step 2: Train a parameter-sharing five-branch face recognition network for wearing masks, using Resnet50 as the backbone network. The structure of this network takes a five-tuple as input, and each branch gets its own embedding, recorded as f(Anchor), f(PU), f(PM), f(NU), and f(NM). The loss function is as follows: m is the minimum distance between the distance between the anchor and the positive example and the distance between the anchor and the negative example; when the difference between the distance between the anchor and the negative example and the distance between the anchor and the positive example is less than or equal to m, the loss is 0; when the difference between the distance between the anchor and the negative example and the distance between the anchor and the positive example is greater than m, the loss is the value in brackets; (5) Using unobstructed face images as a dataset, use the FaceNet model to train an unobstructed face detection network; The input of the FaceNet model is a triplet of images, namely anchor, positive and negative. Positive and anchor are two images of the same person, while negative is an image of another person. Each triplet enters a backbone network to output 128-dimensional features. GoogLeNet is selected as the backbone network, and then L2 regularization is performed to obtain the embeddings of anchor, positive and negative facial features, respectively recorded as f(a), f(p) and f(n). Finally, f(a), f(p) and f(n) are used as parameters and trained using the tripletloss loss function. The training goal is to minimize the loss function. The formula of the tripletloss loss function is as follows: α is the minimum distance between the distance between the anchor and the positive and the distance between the anchor and the negative; when the difference between the distance between the anchor and the negative and the distance between the anchor and the positive is less than or equal to α, the loss is 0; when the difference between the distance between the anchor and the negative and the distance between the anchor and the positive is greater than α, the loss is the value in brackets; (6) Use the trained QMFRN network to recognize the face Im wearing a mask, and the recognition result is Rq.

Citation Information

Patent Citations

  • Facial recognition method and system for face masks

    CN111460962A

  • Local occlusion face recognition method, system and device fused with eye recognition model

    CN112613414A