Face recognition method for people wearing masks, access control device, electronic device and readable medium

Abstract

The present invention relates to the field of face recognition technology, and more specifically to a method for recognizing faces wearing masks, an access control device, an electronic device, and a readable medium. The method comprises: establishing and expanding a sample library of faces not wearing masks; constructing a YOLOv5-MD model for detecting faces wearing masks, and performing training and testing; determining whether a face to be detected is wearing a mask based on the trained YOLOv5-MD; and if the face to be detected is wearing a mask, generating a mask detection result graph for the face to be detected; extracting mask features of the face to be detected, and adding the mask worn by the face to be detected to the faces in the sample library; and spatially mapping the facial features and mask features of the face to be detected wearing a mask with those of the faces in the sample library, thereby generating identification information for the face wearing a mask. The present invention improves the accuracy of face recognition for faces wearing masks and is applicable to face recognition for faces wearing masks at high-traffic entrances and exits, such as school gates and community gates.

Description

Technical Field

[0001] The present invention relates to the field of face recognition technology, and in particular to a face recognition method for a person wearing a mask, an access control device, an electronic device and a readable medium. Background Art

[0002] The novel coronavirus can be transmitted through airborne droplets, aerosols, and other carriers. Wearing a mask in public places is currently an important and effective measure to prevent the spread of the virus and will be a trend for some time after the epidemic ends. However, mask obstruction can cause the loss of intrinsic facial structure, directly affecting facial feature extraction and causing access control systems based on facial recognition to malfunction. Requiring the removal of masks for detection and recognition in crowded places like schools and residential areas would not only increase the risk of cross-infection, but also undermine the systematic and intelligent nature of automatic detection, prolonging detection time, which is unacceptable at high-traffic entrances and exits. Therefore, the rapid and accurate detection and recognition of faces wearing masks is an urgent problem to be solved.

[0003] The invention patent with authorization number CN109101923B provides a method and device for detecting whether a person is wearing a mask. The invention performs threshold segmentation on the acquired target face image to obtain a binary image of the target face, and then calculates the connected domain of the lower half of the binary image, and counts the width and height of the outer rectangular box of the largest connected domain. If the width and height of the outer rectangular box meet the preset conditions, it is considered that the person is wearing a mask. Otherwise, the target face image is converted from RGB space image to YC space image. r C b Space, combined with the binary image to obtain the back-projected face image F b , F b Divided into the upper half of the face image F b1 and the lower half face image F b2 , respectively count F b1 and F b2 The number of pixels n1 and n2 whose pixel values ​​exceed the preset threshold is used to determine whether n1 and n2 meet the preset conditions. If they do, it is considered that the person is not wearing a mask, thus automatically, quickly and accurately detecting whether the person is wearing a mask. However, the method used in this patent can only be used to determine whether the target face is wearing a mask. It cannot realize face recognition of specific groups of people at high-traffic entrances and exits such as schools and residential areas.

[0004] "An Improved YOLOv3-Based Mask Wearing Detection Algorithm" proposes an improved YOLOv3 mask wearing detection method. This method adds the SPP (spatial pyramid pooling) structure to the original YOLOv3 structure to address the issue of inconsistent image sizes input to the neural network. It then uses the K-Means algorithm to cluster the constructed mask data, re-acquire the detection prior bounding box, and appropriately distribute it based on the image size to improve the model's accuracy. Finally, the original IOU Loss loss function is replaced with CIOU Loss to address the difficulty in determining the gradient descent direction during bounding box regression and reduce the loss of accuracy during detection. Although this method has achieved certain results in mask wearing detection, it cannot realize face recognition, and the model is large, leaving much room for improvement in detection accuracy. Summary of the Invention

[0005] The Summary of the Present Invention is intended to briefly introduce concepts that will be described in detail in the Detailed Description of the Present Invention. The Summary of the Present Invention is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0006] In order to solve the technical problem of low accuracy in face recognition of people wearing masks, the present invention proposes a face recognition method for people wearing masks, an access control device, an electronic device and a readable medium.

[0007] In a first aspect, the present invention provides a method for face recognition of a person wearing a mask, the method comprising:

[0008] Establish a facial sample library of people not wearing masks for specific groups according to the required requirements;

[0009] Expand the sample library of faces without masks and build a database, where the database contains samples of faces with and without masks;

[0010] Build a YOLOv5-MD model for face detection of people wearing masks based on YOLOv5, and train and test the model using the database.

[0011] The face image to be detected is passed into the trained mask-wearing face detection model YOLOv5-MD to determine whether the face to be detected is wearing a mask. If the face to be detected is wearing a mask, a mask detection result image of the face to be detected is generated. If the face to be detected is not wearing a mask, a prompt message indicating whether the face to be detected is wearing a mask or not is generated;

[0012] According to the obtained detection result image of the face to be detected wearing a mask, the mask features of the face to be detected are extracted through the mask migration algorithm, the mask worn by the face to be detected is added to the sample library face, and an image of the face wearing a mask in the sample library is generated;

[0013] The generated mask-wearing face detection result image and the mask-wearing face image in the sample library are passed into the improved FaceNet face recognition network. The facial features and mask features of the mask-wearing face to be detected and the mask-wearing faces in the sample library are spatially mapped. The Euclidean distance is calculated through triples to generate the recognition information of the mask-wearing face.

[0014] Furthermore, the mask migration algorithm is used to expand the sample library of faces without masks, including:

[0015] Input the face image of the person wearing a mask and the face image in the sample library;

[0016] Determine the key points of the nose, eyebrows, eyes and facial contours of the face images in the sample library;

[0017] Determine the height and center line of the mask size through the key points of the nose, eyebrows, eyes and bottom of the face;

[0018] Extract the mask features of the face wearing a mask, divide the mask into two equal parts on the left and right, and adjust the mask size according to the determined height and center line of the mask;

[0019] Rotate the mask, the angle is the rotation angle of the center line relative to the y-axis, place the adjusted mask at the appropriate position on the face, complete the mask migration, and output the sample library face image wearing the mask.

[0020] Furthermore, the database is divided into training set, validation set and test set in a ratio of 7:2:1 as the model training set.

[0021] Furthermore, the construction of the mask-wearing face detection model YOLOv5-MD based on YOLOv5 includes:

[0022] Expand the thickness of shallow cross-stage local network modules in the backbone network of the YOLOv5 object detection model to extract detailed features of faces wearing masks at shallow layers.

[0023] The CA attention mechanism is introduced. The CA attention mechanism is used in the feature extraction process of the backbone network of YOLOv5 to enhance the extraction of facial features of people wearing masks, and obtain feature maps with attention weights in the width and height directions.

[0024] The obtained feature maps are weighted fused through the BiFPN feature fusion network to generate feature maps of 16×16, 32×32, and 64×64 sizes for multi-scale detection of the head part.

[0025] Furthermore, the database is used to train and test the constructed YOLOv5-MD model for detecting people wearing masks, including:

[0026] Set the training parameters of the YOLOv5-MD model for face detection with masks based on YOLOv5;

[0027] The training set of the database is fed into the constructed YOLOv5-MD model for face detection of people wearing masks based on YOLOv5, and trained according to the set parameters;

[0028] The trained model is verified based on the validation set of the database to obtain the average error and accuracy change, and the corresponding parameters are adjusted to optimize the model to achieve the optimal state;

[0029] The test set of the database is sent to the YOLOv5-MD model that has reached the optimal state for performance testing, and is evaluated based on the mAP accuracy mean and FPS detection speed indicators to obtain the final mask-wearing face detection model.

[0030] Furthermore, the generated mask-wearing face detection result image and the sample library mask-wearing face image are fed into the improved FaceNet face recognition network, and the facial features and mask features of the mask-wearing face to be detected and the sample library mask-wearing face are spatially mapped. The Euclidean distance is calculated through the triples to generate the recognition information of the mask-wearing face, including:

[0031] Replace the feature extraction network in FaceNet with the CSPNet feature extraction network of YOLOv5-MD;

[0032] The masked face images obtained from YOLOv5-MD and the masked face images generated by the sample library through the mask transfer algorithm are fed into the improved FaceNet feature extraction network for feature extraction and feature fusion;

[0033] By applying L2 normalization to the extracted features, the feature vectors of the face wearing a mask to be detected and the face wearing a mask in the sample library, as well as the feature vector of the mask, are obtained;

[0034] The obtained feature vectors of the masked face to be detected and the masked faces in the sample library, as well as the mask feature vectors, are calculated using triples to calculate the Euclidean distance. The triples include three samples: Anchor, Positive, and Negative. Anchor and Positive are face images of the same person, while Anchor and Negative are face images of different people. The judgment condition is that the square of the Euclidean distance between the feature vectors of the same face is α less than the square of the Euclidean distance between the feature vectors of different face images. α represents the boundary value between positive and negative samples.

[0035] Determine whether the calculated Euclidean distance meets the judgment conditions. If so, they are the same person. If not, they are not the same person, and generate recognition information of the face wearing a mask.

[0036] Furthermore, the determination condition is:

[0037]

[0038]

[0039]

[0040] in, represents the i-th Anchor image, represents the i-th Positive image, Represents the i-th Negative image, Represents the feature vector of the i-th Anchor image, Represents the feature vector of the i-th Positive image, represents the feature vector of the i-th Negative image, ||f(x)|| represents the distance length and α represents the boundary value between positive and negative samples, τ represents the set of possible triplets in the training set, and R d represents a d-dimensional Euclidean space.

[0041] In a second aspect, the present invention provides an access control device, comprising:

[0042] The image acquisition module includes a display module for acquiring the face image to be detected and displaying the recognition result;

[0043] The masked face detection module is used to determine whether a person is wearing a mask. The face image to be detected is fed into the trained YOLOv5-MD model for masked face detection based on YOLOv5. If the person is not wearing a mask, the display module will show that the result has failed. If the person is wearing a mask, the result image of the face to be detected wearing a mask is fed into the feature extraction module.

[0044] The feature extraction module is used to extract the mask features of the face image to be detected obtained by the face detection module wearing a mask through the mask migration algorithm, and add the mask worn by the person to be detected to the face sample library;

[0045] The face recognition module is used to identify facial features and mask features of the face image with a mask obtained by the face detection module and the face image with a mask obtained by the feature extraction module through the face recognition network FaceNet, perform L2 normalization operation, and compare the face to be detected with the face in the sample library by calculating the Euclidean distance;

[0046] The control module includes a gate module, which is used to provide feedback on the results of the face recognition module. If the comparison is successful, the control module controls the gate module to open the gate, and the display module displays that passage is allowed. If the comparison fails, the control module controls the gate module not to open the gate, and the display module displays that passage is prohibited.

[0047] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program that can be stored in the memory and run on the processor. The computer program running on the processor can implement the above-mentioned method for face recognition of people wearing masks.

[0048] In a fourth aspect, the present invention provides a readable medium, which implements the above-mentioned method for face recognition of a person wearing a mask when the computer program stored in the readable medium is executed by a processor.

[0049] The present invention has the following beneficial effects:

[0050] First, because the present invention constructs a new mask-wearing face detection model YOLOv5-MD based on deep regression neural network by introducing CA attention mechanism and improving BiFPN feature fusion network, it can effectively extract shallow detail information and deep global features of faces wearing masks, and improve the detection accuracy of mask features.

[0051] Second, because the present invention uses a mask migration algorithm to design a sample library face optimization mechanism, the spatial distance between the faces in the sample library and the faces to be detected is reduced, thereby improving the accuracy of face recognition in the improved FaceNet network.

[0052] Third, because the present invention organically integrates the feature extraction network of the mask-wearing face detection model YOLOv5-MD and the face recognition network FaceNet, it avoids repeated feature extraction, reduces computational complexity and reduces model parameters.

[0053] Fourth, the face recognition method for people wearing masks proposed in the present invention can be deployed on the proposed access control device and can be widely used at entrances and exits with large traffic flow such as schools and communities, and can quickly and accurately recognize faces of people wearing masks. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0055] Figure 1 This is a flow chart of a face recognition method for a person wearing a mask according to the present invention;

[0056] Figure 2 A flowchart of building a database based on the mask migration algorithm provided by the present invention;

[0057] Figure 3 This is a structural diagram of the YOLOv5-MD network for face detection using masks based on YOLOv5 provided by the present invention;

[0058] Figure 4 An improved FaceNet face recognition network flow chart provided by the present invention;

[0059] Figure 5 This is a flow chart of a face recognition access control device for people wearing masks provided by the present invention;

[0060] Figure 6 A structural diagram of an electronic device provided by the present invention;

[0061] Figure 7 This is another flow chart of a method for face recognition while wearing a mask according to the present invention. DETAILED DESCRIPTION

[0062] To further illustrate the technical means and effects employed by the present invention to achieve its intended objectives, the following, in conjunction with the accompanying drawings and preferred embodiments, describes in detail the specific implementations, structures, features, and effects of the technical solutions proposed by the present invention. In the following description, references to "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.

[0063] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0064] The present invention provides a method for face recognition of a person wearing a mask, the method comprising the following steps:

[0065] Establish a facial sample library of people not wearing masks for specific groups according to the required requirements;

[0066] Expand the sample library of faces without masks and build a database, where the database contains samples of faces with and without masks;

[0067] Build a YOLOv5-MD model for face detection of people wearing masks based on YOLOv5, and train and test the model using the database.

[0068] The face image to be detected is passed into the trained mask-wearing face detection model YOLOv5-MD to determine whether the face to be detected is wearing a mask. If the face to be detected is wearing a mask, a mask detection result image of the face to be detected is generated. If the face to be detected is not wearing a mask, a prompt message indicating whether the face to be detected is wearing a mask or not is generated;

[0069] According to the obtained detection result image of the face to be detected wearing a mask, the mask features of the face to be detected are extracted, the mask worn by the face to be detected is added to the face in the sample library, and an image of the face wearing a mask in the sample library is generated;

[0070] The generated mask-wearing face detection result image and the mask-wearing face image in the sample library are passed into the improved FaceNet face recognition network. The facial features and mask features of the mask-wearing face to be detected and the mask-wearing faces in the sample library are spatially mapped. The Euclidean distance is calculated through triples to generate the recognition information of the mask-wearing face.

[0071] The following is a detailed explanation of each of the above steps:

[0072] refer to Figure 1 , shows the process of some embodiments of a face recognition method for wearing a mask of the present invention. The face recognition method for wearing a mask based on YOLOv5 and FaceNet includes the following steps:

[0073] Step S1: Establish a face sample library of people without masks for a specific group of people according to the required requirements.

[0074] In some embodiments, a facial sample library of people not wearing masks can be established for specific groups of people according to required requirements.

[0075] Among them, specific groups refer to the students and faculty of a school, the owners and property staff of a community, and other people in high-traffic entrances and exits who need to wear masks and undergo facial recognition.

[0076] Step S2: Expand the sample library of faces without masks and build a database.

[0077] In some embodiments, the sample library of faces without masks can be expanded to build a database.

[0078] Among them, the constructed database contains face samples with and without masks.

[0079] As an example, the non-mask face sample library established in step S1 can be expanded by using a mask migration algorithm based on a generative adversarial network or manually adding simulated masks to construct a database that includes face samples with and without masks.

[0080] As another example, Figure 2 As shown, the mask migration algorithm based on the generative adversarial network is used to expand the sample library of unmasked faces established in step S1 to construct a database, which may include the following steps:

[0081] Step 201: input a face image of a person wearing a mask and a face image in a sample library.

[0082] Step 202: Determine the key points of the nose, eyebrows, eyes, and facial contour of the face image in the sample library.

[0083] For example, the nose, eyebrows, eyes, and facial contour key points of the face image in the sample library included in step 201 may be determined.

[0084] Step 203, determine the height and center line of the mask size through the key points of the nose, eyebrows, eyes and bottom of the face.

[0085] For example, the height and center line of the mask size can be determined through the key points of the nose, eyebrows, eyes, and bottom of the face included in step 202.

[0086] Step 204: extract the mask features of the face of the person wearing the mask, divide the mask into two equal parts on the left and right sides, and adjust the size of the mask according to the determined height and center line of the mask.

[0087] For example, the mask features of the person wearing the mask can be extracted, the mask can be divided into two equal parts on the left and right sides, and the size of the mask can be adjusted according to the height and center line of the mask size determined in step 203.

[0088] Step 205: rotate the mask, the angle being the rotation angle of the center line relative to the y-axis, and place the adjusted mask at the appropriate position on the face to complete the mask migration and output the sample library face image of the person wearing the mask.

[0089] Among them, the output facial images of the sample library of people wearing masks and the sample library of people not wearing masks can form a database.

[0090] For example, the mask can be rotated, the angle of which is the rotation angle of the center line relative to the y-axis, and the mask adjusted in step 204 is placed at the appropriate position on the face to complete the mask migration and output the sample library face image wearing the mask.

[0091] Optionally, the database can be divided into the training set, validation set, and test set of the model in a ratio of 7:2:1. That is, 70% of the face sample data is randomly selected as the training set of the model, 20% of the face sample data is randomly selected as the validation set of the model, and 10% of the face sample data is randomly selected as the test set of the model.

[0092] The database was constructed to enable feature learning for the masked face detection model. Currently, most face detection datasets, such as CelebA, MTFL, and IMDB-WIKI, are based on unmasked faces. The world's largest publicly available dataset of masked faces, RWMF (Real World Masked Face), contains 5,000 masked faces and 90,000 normal faces from 525 individuals. The relatively small number of masked face images compared to normal faces results in poor detection performance for the YOLOv5-MD detection model, thus affecting the final face recognition results. Therefore, a mask migration algorithm based on a generative adversarial network and the manual addition of simulated masks were employed to expand the masked face category. The labels of the original sample library and the expanded database were then processed into the format required by YOLO for subsequent training and testing.

[0093] Step S3: construct a YOLOv5-MD model for detecting people wearing masks based on YOLOv5, and train and test the constructed YOLOv5-MD model for detecting people wearing masks based on YOLOv5 through a database.

[0094] In some embodiments, a YOLOv5-MD model for detecting faces wearing masks based on YOLOv5 can be constructed, and the constructed YOLOv5-MD model for detecting faces wearing masks based on YOLOv5 can be trained and tested through a database.

[0095] As an example, first of all, the YOLO series target detection model combines multiple methods such as residual networks, feature pyramids, and multi-feature fusion networks, and has good detection accuracy and recognition speed. The improved basic model YOLOv5 uses CSPDarknet as the backbone network. CSPDarknet can solve the problem of gradient information duplication in network optimization in the backbone network of large convolutional neural network frameworks, reducing the number of model parameters and FLOPS values; secondly, the network structure of the YOLOv5 target detection model is simple, far surpassing YOLOv3 in flexibility and speed, and has the advantage of easy and rapid model deployment; finally, in view of the problem that wearing a mask on the face will cause the loss of the inherent structure of the face, which directly affects the extraction of facial features, thereby affecting the detection effect and subsequent face recognition effect, the YOLOv5 target detection model is selected for improvement. The masked face detection model YOLOv5-MD based on YOLOv5 includes: introducing the CA attention mechanism in the backbone network; using the improved BiFPN feature fusion network; the detection layer Head part uses 3 scale feature maps as the target prediction layer, and the masked face detection model YOLOv5-MD based on YOLOv5 is as follows: Figure 3 As shown in the figure, building a YOLOv5-MD model for face detection of people wearing masks based on YOLOv5 can include the following steps:

[0096] The first step is to expand the thickness of the shallow cross-stage local network modules in the backbone network of the YOLOv5 target detection model to extract the detailed features of the face wearing a mask at a shallow level.

[0097] In the second step, the CA attention mechanism is introduced. The CA attention mechanism is used to strengthen the extraction of facial features of people wearing masks during the backbone network feature extraction process of YOLOv5, and obtain feature maps with attention weights in the width and height directions.

[0098] For example, introducing the CA attention mechanism in the backbone network can include the following sub-steps:

[0099] In the first sub-step, during the Coordinate information embedding stage, the input image to be detected is divided into two directions: width w and height h, and global average pooling is performed to obtain feature maps in both width and height directions.

[0100] In the second substep, during the Coordinate feature mapping phase, the feature maps from the two directions obtained in the first substep of this example are concatenated and fed into a convolution transformation function F1 with a shared 1×1 kernel. This reduces their dimensionality to the original c / r, where c is the number of channels and r is the reduction factor. The batch-normalized feature map is fed into a Sigmoid activation function to obtain the feature map f.

[0101] In the third sub-step, in the Coordinate attention generation stage, the feature map f obtained in the second sub-step of this example is convolved with a 1×1 convolution kernel along the spatial dimension according to the original height h and width w to obtain the feature map f with the same number of channels as the original. h ,f w , and then use two 1×1 convolution transformation functions to transform f h ,f w Transform to obtain the attention feature weights g in the height and width directions h ,g w .

[0102] In the fourth sub-step, the original feature map is weighted again by multiplication to obtain a feature map with attention weights in the width and height directions.

[0103] In the third step, the obtained feature maps are weighted fused through the BiFPN feature fusion network to generate feature maps of sizes 16×16, 32×32, and 64×64 for multi-scale detection of the head part.

[0104] Among them, the BiFPN feature fusion network is a weighted bidirectional feature pyramid that can achieve efficient bidirectional cross-scale connection and perform weighted fusion of features extracted from different convolutional layers.

[0105] For example, the feature maps with attention weights in the width and height directions extracted in the third step of this example can be weighted fused through the BiFPN feature fusion network to generate feature maps of sizes 16×16, 32×32, and 64×64, and then the final detection result map of the face wearing a mask can be obtained.

[0106] As another example, training and testing the constructed YOLOv5-MD face detection model for wearing a mask based on YOLOv5 through the database (the database established in step S2) may include the following steps:

[0107] The first step is to set the training parameters of the YOLOv5-MD model for face detection with masks based on YOLOv5.

[0108] For example, the model training parameters are set, and the random optimization algorithm Adam is used for training. The training batch size is set to Batch = 64, the momentum is Momentum = 0.9, the initial learning rate is set to Ir = 0.001, and the number of training iterations is Epoch = 300.

[0109] In the second step, the training set of the database established in step S2 is sent to the mask-wearing face detection model YOLOv5-MD based on YOLOv5 constructed in step S3, and trained according to the parameters set in the first step of this example.

[0110] In the third step, the trained model is verified according to the validation set of the database to obtain the average error and accuracy change, and the corresponding parameters are adjusted to optimize the model to achieve the optimal state.

[0111] For example, the model trained in the second step of this example can be verified using the validation set of the database established in step S2 to obtain the average error and accuracy change, and the corresponding parameters can be adjusted to optimize the model to achieve the optimal state.

[0112] In the fourth step, the test set of the database is sent to the YOLOv5-MD model that has reached the optimal state for performance testing, and evaluated from the mAP accuracy mean and FPS detection speed indicators to obtain the final mask-wearing face detection model.

[0113] For example, the test set of the database established in step S2 can be sent to the YOLOv5-MD model that reaches the optimal state in the third step of this example for performance testing, and evaluated from the mAP accuracy mean and FPS detection speed indicators to obtain the final mask-wearing face detection model.

[0114] Step S4: The face image to be detected is passed into the trained mask-wearing face detection model YOLOv5-MD to determine whether the face to be detected is wearing a mask. If the face to be detected is wearing a mask, a mask-wearing detection result graph of the face to be detected is generated; if the face to be detected is not wearing a mask, a prompt message is generated indicating whether the face to be detected is wearing a mask or not.

[0115] In some embodiments, the face image to be detected can be passed to the mask-wearing face detection model YOLOv5-MD trained in step S3 to determine whether the face to be detected is wearing a mask. If the face to be detected is wearing a mask, a mask-wearing face detection result map is generated, and the subsequent step S5 is processed. If the face to be detected is not wearing a mask, a prompt message indicating whether the face to be detected is wearing a mask or not is generated, subsequent facial recognition is not performed, and the access control device does not allow the user to pass.

[0116] Step S5: extract the mask features of the face to be detected based on the obtained detection result image of the face to be detected wearing a mask, add the mask worn by the face to be detected to the face in the sample library, and generate an image of the face wearing a mask in the sample library.

[0117] In some embodiments, based on the mask-wearing detection result image of the face to be detected obtained in step S4, the mask features of the face to be detected can be extracted through a mask migration algorithm, the mask worn by the face to be detected can be added to the face in the sample library, and an image of the face wearing a mask in the sample library can be generated.

[0118] It should be noted that the mask migration algorithm included in step S5 is designed to avoid errors caused by mask color or type when identifying faces wearing masks, thereby improving the accuracy of face recognition. The difference between the mask migration algorithm included in step S2 is that the input of the mask migration algorithm included in step S2 is a random face image wearing a mask and a sample library of face images, while the input of the mask migration algorithm included in step S5 is an image of the face to be detected wearing a mask and a sample library of face images. The remaining steps of the mask migration algorithm remain unchanged.

[0119] Step S6: The generated detection result image of the face to be detected wearing a mask and the image of the face wearing a mask in the sample library are passed into the improved FaceNet face recognition network, and the facial features and mask features of the face to be detected wearing a mask and the faces wearing masks in the sample library are spatially mapped. The Euclidean distance is calculated through the triples to generate the identification information of the face wearing a mask.

[0120] In some embodiments, the mask-wearing face detection result image of the face to be detected generated in step S4 and the mask-wearing face image of the sample library generated in step S5 can be passed into the improved FaceNet face recognition network, and the facial features and mask features of the mask-wearing face to be detected and the mask-wearing faces in the sample library can be spatially mapped. The Euclidean distance is calculated through triples to generate recognition information of the mask-wearing face, thereby achieving accurate recognition of the mask-wearing face.

[0121] As an example, this step may include the following steps:

[0122] The first step is to replace the feature extraction network in FaceNet with the CSPNet feature extraction network of YOLOv5-MD.

[0123] For example, the feature extraction network in FaceNet can be replaced with the CSPNet feature extraction network of YOLOv5-MD, which can better solve the problem of gradient information duplication in network optimization and reduce the number of model parameters and FLOPS value.

[0124] In the second step, the masked face images obtained in YOLOv5-MD and the masked face images generated by the mask migration algorithm in the sample library are sent to the improved FaceNet feature extraction network for feature extraction and feature fusion.

[0125] For example, the mask-wearing detection result image of the face to be detected generated in step S4 and the mask-wearing image of the face in the sample library generated in step S5 can be sent to the improved FaceNet feature extraction network included in the first step of this example for feature extraction and feature fusion.

[0126] In the third step, by applying L2 normalization operation to the extracted features, the feature vectors of the face wearing a mask to be detected and the face wearing a mask in the sample library, as well as the feature vector of the mask, are obtained.

[0127] For example, by applying L2 normalization to the features extracted in the second step of this example, we can obtain feature vector representations of the face wearing a mask to be detected, the faces wearing masks in the sample library, and the feature vector representation of the mask. The L2 normalization operation maps the different features extracted in the second step of this example onto a sphere to form points, which is the representation of the extracted feature vector. The distance from all points to the center of the circle is 1. This transforms the problem of face recognition into a simple problem of finding coordinate distances.

[0128] In the fourth step, the Euclidean distance between the feature vectors of the mask-wearing face to be detected obtained in the third step of this example and the feature vectors of the mask-wearing face in the sample library and the feature vectors of the mask is calculated through triples.

[0129] A triplet consists of three examples: Anchor, Positive, and Negative. Anchor and Positive represent the same person's face. Anchor and Negative represent different people's face. The judgment condition is that the square of the Euclidean distance between feature vectors of the same face is less than the square of the Euclidean distance between feature vectors of different face images by α, where α represents the boundary between positive and negative samples.

[0130] For example, the above determination condition may be:

[0131]

[0132]

[0133]

[0134] in, represents the i-th Anchor image, represents the i-th Positive image, Represents the i-th Negative image, Represents the feature vector of the i-th Anchor image, Represents the feature vector of the i-th Positive image, represents the feature vector of the i-th Negative image, ||f(x)|| represents the distance length and α represents the boundary value between positive and negative samples, τ represents the set of possible triplets in the training set, and R d represents a d-dimensional Euclidean space.

[0135] The fifth step is to determine whether the calculated Euclidean distance meets the judgment conditions. If the judgment conditions are met, they are the same person. If the judgment conditions are not met, they are not the same person, and the recognition information of the face wearing a mask is generated.

[0136] For example, the Euclidean distance calculated in the fourth step of this example is used to determine whether the calculated Euclidean distance meets the judgment condition. If the judgment condition is met, identification information indicating that passage is allowed is generated. Otherwise, identification information indicating that passage is not allowed is generated, and the access control device does not allow passage.

[0137] The steps to implement the improved FaceNet face recognition network can be as follows: Figure 4 shown.

[0138] like Figure 5 As shown, the present invention provides an access control device for face recognition of people wearing masks, comprising:

[0139] The image acquisition module 501 includes a display module 507 for acquiring a face image to be detected and displaying the recognition result;

[0140] The masked face detection module 502 is used to determine whether a person is wearing a mask and feed the face image to be detected into the trained masked face detection model YOLOv5-MD based on YOLOv5. If the person is not wearing a mask, the display module 2 will display a failure. If the person is wearing a mask, the result image of the face to be detected will be fed into the feature extraction module 503.

[0141] The feature extraction module 503 is used for extracting the mask features of the face image to be detected obtained by the mask-wearing face detection module 502 using the mask migration algorithm, and adding the mask worn by the person to be detected to the face sample library;

[0142] The face recognition module 504 is used to identify the facial features and mask features of the face image obtained by the face recognition module 502 wearing a mask and the face image of the face wearing a mask obtained by the feature extraction module 503 through the face recognition network FaceNet, perform L2 normalization operation, and compare the face to be detected with the faces in the sample library by calculating the Euclidean distance to realize face recognition of the face to be detected;

[0143] The control module 505 includes a gate module 508, which is used to provide feedback on the results of the face recognition module 504. If the comparison is successful, the control module 505 controls the gate module 508 to open the gate, and the display module 507 displays "allowed to pass". If the comparison fails, the control module 505 controls the gate module 508 not to open the gate, and the display module 507 displays "no passage".

[0144] The system also includes a connection port module 506 for data transmission and power supply. The mask-wearing face detection module 502 and the face recognition module 504 are connected to the control module 505 via the connection port 506 to perform data transmission and determine whether the person to be detected can pass. The display module 507 is connected to the control module 505, and the results are displayed on the display module 507.

[0145] like Figure 6 As shown, the present invention provides an electronic device, including an image acquisition camera, a display, a memory, a processor, a power supply, a communication interface, an input / output port, an access control gate, and a communication bus. The processor includes one or more processing cores, and the processor is connected to the memory via a communication bus. The memory can be a read-only memory, a disk, or an optical disk, etc. The resources stored in the memory include an operating system, a computer program, and sample library data, and the processor is capable of executing the computer program in the memory and implementing any of the relevant steps of the above-mentioned face recognition method for wearing a mask based on YOLOv5 and FaceNet.

[0146] The flowchart of the face recognition method for a person wearing a mask provided by the present invention can be as follows: Figure 7 shown.

[0147] Based on the same inventive concept as the above-mentioned method embodiment, a readable medium is provided, wherein the computer program stored in the readable medium, when executed by a processor, implements the steps of a method for face recognition of a person wearing a mask.

[0148] Because the present invention constructs a new mask-wearing face detection model YOLOv5-MD based on a deep regression neural network by introducing the CA attention mechanism and improving the BiFPN feature fusion network, it effectively extracts shallow detail information and deep global features of the face wearing a mask, and improves the detection accuracy of the mask features. Because the present invention uses a mask migration algorithm to design a sample library face optimization mechanism, it reduces the spatial distance between the faces in the sample library and the faces to be detected, and improves the accuracy of face recognition in the improved FaceNet network. Because the present invention organically integrates the mask-wearing face detection model YOLOv5-MD and the feature extraction network of the face recognition network FaceNet, it avoids repeated feature extraction, reduces computational complexity and reduces model parameters. Because the mask-wearing face recognition method proposed by the present invention can be deployed on the proposed access control device, and can be widely used at entrances and exits with large traffic such as schools and communities, and can quickly and accurately recognize faces wearing masks.

[0149] Those skilled in the art will appreciate that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by programs instructing related hardware to accomplish the steps.

[0150] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A method for face recognition of a person wearing a mask, characterized in that: The following steps are involved: Establish a facial sample library of people not wearing masks for specific groups according to the required requirements; Expand the sample library of faces without masks and build a database, where the database contains samples of faces with and without masks; Build a YOLOv5-MD model for face detection of people wearing masks based on YOLOv5, and train and test the model using the database. The face image to be detected is passed into the trained mask-wearing face detection model YOLOv5-MD to determine whether the face to be detected is wearing a mask. If the face to be detected is wearing a mask, a mask detection result image of the face to be detected is generated. If the face to be detected is not wearing a mask, a prompt message indicating whether the face to be detected is wearing a mask or not is generated; According to the obtained detection result image of the face to be detected wearing a mask, the mask features of the face to be detected are extracted through the mask migration algorithm, the mask worn by the face to be detected is added to the sample library face, and an image of the face wearing a mask in the sample library is generated; The generated mask-wearing face detection result image and the sample library mask-wearing face image are fed into the improved FaceNet face recognition network. The facial features and mask features of the mask-wearing face to be detected and the sample library are spatially mapped. The Euclidean distance is calculated through the triples to generate the recognition information of the mask-wearing face. The construction of the mask-wearing face detection model YOLOv5-MD based on YOLOv5 includes: Expand the thickness of shallow cross-stage local network modules in the backbone network of the YOLOv5 object detection model to extract detailed features of faces wearing masks at shallow layers. The CA attention mechanism is introduced. The CA attention mechanism is used in the feature extraction process of the backbone network of YOLOv5 to enhance the extraction of facial features of people wearing masks, and obtain feature maps with attention weights in the width and height directions. The obtained feature maps are weighted fused through the BiFPN feature fusion network to generate feature maps of 16×16, 32×32, and 64×64 sizes for multi-scale detection of the head part; The generated mask-wearing face detection result image and the sample library mask-wearing face image are fed into the improved FaceNet face recognition network, and the facial features and mask features of the mask-wearing face to be detected and the sample library are spatially mapped. The Euclidean distance is calculated through the triples to generate the recognition information of the mask-wearing face, including: Replace the feature extraction network in FaceNet with the CSPNet feature extraction network of YOLOv5-MD; The masked face images obtained from YOLOv5-MD and the masked face images generated by the sample library through the mask transfer algorithm are fed into the improved FaceNet feature extraction network for feature extraction and feature fusion; By applying L2 normalization to the extracted features, the feature vectors of the face wearing a mask to be detected and the face wearing a mask in the sample library, as well as the feature vector of the mask, are obtained; The obtained feature vectors of the face wearing a mask to be detected and the face wearing a mask in the sample library and the feature vector of the mask are calculated through triples. The triples include three samples: Anchor, Positive and Negative. Anchor and Positive are face images of the same person, and Anchor and Negative are face images of different people. The judgment condition is that the square of the Euclidean distance between the feature vectors of the same face is smaller than the square of the Euclidean distance between the feature vectors of different face images. , Indicates the boundary value between positive and negative samples; Determine whether the calculated Euclidean distance meets the judgment conditions. If so, they are the same person. If not, they are not the same person, and generate recognition information of the face wearing a mask.

2. A face recognition method for people wearing masks according to claim 1, characterized in that: The mask migration algorithm is used to expand the sample library of faces without masks, including: Input the face image of the person wearing a mask and the face image in the sample library; Determine the key points of the nose, eyebrows, eyes and facial contours of the face images in the sample library; Determine the height and center line of the mask size through the key points of the nose, eyebrows, eyes and bottom of the face; Extract the mask features of the face wearing a mask, divide the mask into two equal parts on the left and right, and adjust the mask size according to the determined height and center line of the mask; Rotate the mask, the angle is the rotation angle of the center line relative to the y-axis, place the adjusted mask at the appropriate position on the face, complete the mask migration, and output the sample library face image wearing the mask.

3. A face recognition method for a person wearing a mask according to claim 1, characterized in that: The database is divided into training set, validation set and test set in a ratio of 7:2:

1.

4. A face recognition method for a person wearing a mask according to claim 3, characterized in that: The database is used to train and test the constructed YOLOv5-MD model for face detection of people wearing masks, including: Set the training parameters of the YOLOv5-MD model for face detection with masks based on YOLOv5; The training set of the database is fed into the constructed YOLOv5-MD model for face detection of people wearing masks based on YOLOv5, and trained according to the set parameters; The trained model is verified based on the validation set of the database to obtain the average error and accuracy change, and the corresponding parameters are adjusted to optimize the model to achieve the optimal state; The test set of the database is sent to the YOLOv5-MD model that has reached the optimal state for performance testing, and is evaluated based on the mAP accuracy mean and FPS detection speed indicators to obtain the final mask-wearing face detection model.

5. A face recognition method for a person wearing a mask according to claim 1, characterized in that: The determination conditions are: in, Indicates the Anchor images, Indicates the Positive images, Indicates the Negative images, Indicates the The feature vector of the anchor image, Indicates the The feature vector of the positive image, Indicates the The feature vector of the Negative image, Indicates the distance length and , Represents the boundary value between positive and negative samples, represents the set of possible triplets in the training set, express dimensional Euclidean space.

6. An access control device, characterized in that: The access control device is used to implement the face recognition method for a person wearing a mask as described in any one of claims 1 to 5. Specifically, the access control device includes: The image acquisition module includes a display module for acquiring the face image to be detected and displaying the recognition result; The masked face detection module is used to determine whether a person is wearing a mask. The face image to be detected is fed into the trained YOLOv5-MD model for masked face detection based on YOLOv5. If the person is not wearing a mask, the display module will show that the result has failed. If the person is wearing a mask, the result image of the face to be detected wearing a mask is fed into the feature extraction module. The feature extraction module is used to extract the mask features of the face image to be detected obtained by the face detection module wearing a mask through the mask migration algorithm, and add the mask worn by the person to be detected to the face sample library; The face recognition module is used to identify facial features and mask features of the face image with a mask obtained by the face detection module and the face image with a mask obtained by the feature extraction module through the face recognition network FaceNet, perform L2 normalization operation, and compare the face to be detected with the face in the sample library by calculating the Euclidean distance; The control module includes a gate module, which is used to provide feedback on the results of the face recognition module. If the comparison is successful, the control module controls the gate module to open the gate, and the display module displays that passage is allowed. If the comparison fails, the control module controls the gate module not to open the gate, and the display module displays that passage is prohibited.

7. An electronic device, characterized in that: The electronic device includes a memory, a processor, and a computer program that can be stored in the memory and can be run on the processor. The computer program running on the processor can implement the method for face recognition wearing a mask described in any one of claims 1-5.

8. A computer program stored in a readable medium, characterized in that: When the computer program is executed by a processor, the method for recognizing a face wearing a mask described in any one of claims 1 to 5 is implemented.

Images