Twin multi-task face recognition method and device

Abstract

The invention provides a twin multi-task face recognition method and device. The method comprises the following steps: acquiring a to-be-recognized face image; inputting the face image to be recognized into the face recognition model to obtain a face recognition result and a mask classification result output by the face recognition model; wherein the face recognition model is obtained by training based on twin face training data and face recognition tags and mask classification tags corresponding to the twin face training data; the face recognition model is used for performing feature extraction on an input to-be-recognized face image and performing face recognition and mask classification on extracted face features to obtain a face recognition result and a mask classification result. According to the invention, face recognition and mask classification are carried out through the face recognition model trained by using the twin face training data, so that the accuracy of face recognition is greatly improved and the generalization ability of the model is improved under the condition that the accuracy of non-shielding face recognition is not reduced.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a twin multi-task face recognition method and device. Background Art

[0002] In standard scenarios, face recognition technology has met the application needs of most industries and commerce. However, with the continuous emergence of various respiratory viruses, masks have become a common item in daily life, resulting in a significant decline in the performance of conventional face recognition algorithms. Therefore, face recognition algorithms with masks have emerged, providing the possibility to cope with more complex real-world scenarios, including epidemic prevention and control, identity authentication in special medical places, and public safety in smart city environments.

[0003] At present, the following two algorithms are mainly used to detect faces wearing masks. One is the Viola-Jones algorithm, which mainly detects the eye area in the cropped facial image by using the statistical region merging (SRM) method to generate an attribute map of the detected facial features; the other is to use FaceNet as the training framework and ConvNeXt-T as the network model, and add the ECA (efficient channel attention) mechanism to the network.

[0004] However, the above-mentioned Viola-Jones algorithm requires a two-stage algorithm, which greatly reduces the efficiency of face recognition and is difficult to meet the application needs of actual scenarios. The ConvNeXt-T network has a large number of parameters, and the triplet loss function used by FaceNet has obvious fitting defects, so it is not friendly to the deployment of some lightweight devices and has problems such as limitations in application scenarios. Summary of the invention

[0005] The present invention provides a twin multi-task face recognition method and device to solve the defects of low efficiency and limited application scenarios of face recognition in the prior art. Without reducing the accuracy of unobstructed face recognition, the accuracy of face recognition is greatly improved, and the recognition efficiency is improved by making the framework have very small complexity and parameter amount. The method is suitable for mobile devices and embedded systems with limited computing resources.

[0006] The present invention provides a twin multi-task face recognition method, comprising: obtaining a face image to be recognized; inputting the face image to be recognized into a face recognition model to obtain a face recognition result and a mask classification result output by the face recognition model; wherein the face recognition model is trained based on twin face training data and face recognition labels and mask classification labels corresponding to the twin face training data; the face recognition model is used to extract features of the input face image to be recognized, and perform face recognition and mask classification on the extracted face features respectively, to obtain a face recognition result and a mask classification result.

[0007] According to a twin multi-task face recognition method provided by the present invention, the face recognition model includes: a feature extraction layer, which extracts features of an input face image to be recognized to obtain face features, and sends the face features to a fully connected layer, and sends the corresponding face features to a face recognition layer according to the target dimension obtained after model training; a fully connected layer, which reduces the dimension of the face features to obtain reduced dimension features; a mask classification layer, which classifies the reduced dimension features to obtain mask classification results; a face recognition layer, which determines the Euclidean distance between the face features and each face feature in a previously constructed face feature library, and determines, in combination with a preset threshold, whether the received face features and the corresponding face features in the face feature library are the same person to obtain a face recognition result.

[0008] According to a twin multi-task face recognition method provided by the present invention, before inputting the face image to be recognized into the face recognition model, it also includes: obtaining twin face training data and face recognition labels and mask classification labels corresponding to the twin face training data; wherein the twin face training data includes unobstructed face training samples and mask-occluded face training samples corresponding to each unobstructed face training sample face; inputting the unobstructed face training samples into the first model to be trained to obtain a first recognition result and a first classification result output by the first model to be trained, and inputting the mask-occluded face training samples corresponding to the unobstructed face training sample faces into the second model to be trained to obtain a second recognition result and a first classification result output by the second model to be trained The second classification result; wherein, the parameters of the first model to be trained and the second model to be trained are shared; an unobstructed loss function is constructed based on the first recognition result, the first classification result, and the face recognition labels and mask classification labels corresponding to the unobstructed face training samples, and a mask occlusion loss function is constructed based on the second recognition result, the second classification result, and the face recognition labels and mask classification labels corresponding to the mask occluded face training samples; according to the unobstructed loss function and the mask occlusion loss function, combined with the weight distribution mechanism, a total loss function is obtained, and according to the total loss function, an optimizer is used to perform gradient back propagation on the first model to be trained and the second model to be trained to update the corresponding model parameters until the total loss function converges and the training is terminated.

[0009] According to a twin multi-task face recognition method provided by the present invention, the first model to be trained includes a first feature extraction layer, a first fully connected layer, a first mask classification layer, a first feature learning layer and a first face recognition layer; an unobstructed face training sample is input into the first model to be trained to obtain a first recognition result and a first classification result output by the first model to be trained, including: inputting the unobstructed face training sample into the first feature extraction layer for feature extraction to obtain a first face training feature output by the first feature extraction layer; inputting the first face training feature into the first fully connected layer for dimensionality reduction to obtain a first dimensionality reduction feature output by the first fully connected layer; inputting the first dimensionality reduction feature into the first mask classification layer to obtain a first classification result output by the first mask classification layer; inputting the first face training feature into the first feature learning layer according to a first preset dimension; In the method, the first face training feature of the first preset dimension is processed by using the adaptive face AdaFace, and the processing result is dimensionally upgraded according to the number of face recognition labels corresponding to the unobstructed face training sample, so as to obtain the second dimension-upgraded feature output by the first feature learning layer; wherein the first preset dimension is obtained by adjusting the first preset dimension used in the previous training according to the model accuracy determined after the previous training and the preset dimension adjustment rule, the model accuracy determined after the previous training is obtained by using the ablation experiment according to the first recognition result and the first classification result obtained after the previous training, and the preset dimension adjustment rule is used to limit the adjustment relationship between the model accuracy and the corresponding preset dimension; the second dimension-upgraded feature is input into the first face recognition layer for probability classification, so as to obtain the first recognition result output by the first face recognition layer; The second model to be trained includes a second feature extraction layer, a second fully connected layer, a second mask classification layer, a second feature learning layer and a second face recognition layer; the mask-occluded face training samples corresponding to the unobstructed face training sample faces are input into the second model to be trained, and the second recognition result and the second classification result output by the second model to be trained are obtained, including: the mask-occluded face training samples corresponding to the unobstructed face training sample faces are input into the second feature extraction layer for feature extraction, and the second face training features output by the second feature extraction layer are obtained; the second face training features are input into the second fully connected layer for dimensionality reduction, and the third dimensionality reduction features are output by the second fully connected layer; the third dimensionality reduction features are input into the second mask classification layer, and the second classification results output by the second mask classification layer are obtained; according to the second preset dimension, the second face training features are input into the second feature extraction layer; ... In the feature learning layer, the second face training features of the second preset dimension are processed by using the adaptive face AdaFace, and the processing results are upgraded according to the number of face recognition labels corresponding to the training samples of the face covered by the mask, so as to obtain the fourth upgraded feature output by the second feature learning layer; wherein, the second preset dimension is obtained by adjusting the second preset dimension used in the previous training according to the preset dimension adjustment rule based on the model accuracy determined after the previous training, and the model accuracy determined after the previous training is obtained by using the ablation experiment based on the second recognition result and the second classification result obtained after the previous training, and the preset dimension adjustment rule is used to limit the adjustment relationship between the model accuracy and the corresponding preset dimension; the fourth upgraded feature is input into the second face recognition layer for probability classification, and the second recognition result output by the second face recognition layer is obtained.

[0010] According to a twin multi-task face recognition method provided by the present invention, an unobstructed loss function is constructed based on a first recognition result, a first classification result, and a face recognition label and a mask classification label corresponding to an unobstructed face training sample, including: constructing a first classification loss function according to all first classification results obtained during the training process and the mask classification labels of the corresponding unobstructed face training samples, combined with a cross entropy loss function; constructing a first recognition loss function according to the first recognition result and the face recognition label corresponding to the unobstructed face training sample, combined with a cross entropy loss function; constructing an unobstructed loss function according to the first classification loss function, the first recognition loss function and a first preset weight; Based on the second recognition result, the second classification result, and the face recognition label and the mask classification label corresponding to the mask-occluded face training sample, a mask occlusion loss function is constructed, including: constructing a second classification loss function based on all the second classification results obtained during the training process and the mask classification labels of the corresponding mask-occluded face training samples in combination with the cross entropy loss function; constructing a second recognition loss function based on the second recognition result and the face recognition label corresponding to the mask-occluded face training sample in combination with the cross entropy loss function; constructing a mask occlusion loss function based on the second classification loss function, the second recognition loss function and the first preset weight.

[0011] According to a twin multi-task face recognition method provided by the present invention, twin face training data is obtained, including: obtaining unobstructed face training samples and face recognition labels and mask classification labels corresponding to the unobstructed face training samples; using the face masking tool MaskTheFace to mask the unobstructed face training samples to obtain corresponding mask-occluded face training samples; wherein the gradient boosting-based face detector in MaskTheFace is replaced with a maximum margin face detection algorithm based on a convolutional neural network (CNN); twin face training data is obtained based on the unobstructed face training samples and their corresponding mask-occluded face training samples.

[0012] According to a twin multi-task face recognition method provided by the present invention, a total loss function is obtained according to an unobstructed loss function and a mask occlusion loss function in combination with a weight allocation mechanism, including: according to the unobstructed loss function and the mask occlusion loss function in combination with a weight allocation mechanism, a first allocation weight is allocated to the unobstructed loss function, and a second allocation weight is allocated to the mask occlusion loss function; wherein the sum of the first allocation weight and the second allocation weight is 1, and the initially allocated first allocation weight is greater than the second allocation weight; the first allocation weight is reduced based on a preset value, and the second allocation weight is increased based on a preset value, and weights are iteratively allocated to the unobstructed loss function and the mask occlusion loss function until the second allocation weight approaches 1 and the first allocation weight approaches 0 and stops; according to the weights allocated to the unobstructed loss function and the mask occlusion loss function each time, the corresponding model accuracy is obtained by using an ablation experiment; the weights allocated to the unobstructed loss function and the mask occlusion loss function corresponding to the maximum model accuracy are selected, and the total loss function is obtained by combining the unobstructed loss function and the mask occlusion loss function.

[0013] The present invention also provides a twin multi-task face recognition device, including: an image acquisition module, which acquires a face image to be recognized; a face recognition module, which inputs the face image to be recognized into a face recognition model to obtain a face recognition result and a mask classification result output by the face recognition model; wherein the face recognition model is trained based on twin face training data and face recognition labels and mask classification labels corresponding to the twin face training data; the face recognition model is used to extract features of the input face image to be recognized, and perform face recognition and mask classification on the extracted face features respectively to obtain face recognition results and mask classification results.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the twin multi-task face recognition method as described above is implemented.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the twin multi-task face recognition methods described above.

[0016] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the twin multi-task face recognition methods described above.

[0017] The twin multi-task face recognition method and device provided by the present invention performs face recognition and mask classification on the input face image to be recognized through the face recognition model, so as to greatly improve the accuracy of face recognition without reducing the accuracy of unobstructed face recognition. In practical application scenarios, especially in special periods such as when masks need to be worn, it can not only meet the conventional face recognition needs, but also quickly determine whether the personnel are wearing masks, which provides convenience for the safety management of the site and the implementation of epidemic prevention measures. In addition, the model is trained using twin face training data to make full use of the characteristics of twin data and better capture the similarities and differences of faces, thereby improving the accuracy of the model in face recognition tasks. At the same time, combined with face recognition labels and mask classification labels for training, the model can learn the relationship between different face features and mask wearing status, improve the accuracy of mask classification, improve the generalization ability of the model, and can flexibly adapt to various complex and changeable practical application scenarios, and maintain good adaptability to the needs of different scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 It is a flowchart of the twin multi-task face recognition method provided by the present invention; Figure 2 It is a schematic diagram of the architecture of the face recognition model provided by the present invention; Figure 3 It is a schematic diagram of the architecture of the training face recognition model provided by the present invention; Figure 4 It is a structural schematic diagram of a twin multi-task face recognition device provided by the present invention; Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] Figure 1 : is a flow chart of the twin multi-task face recognition method provided by the present invention, the method comprising: S11, obtaining a face image to be recognized; S12, input the face image to be identified into the face recognition model, and obtain the face recognition result and mask classification result output by the face recognition model; wherein, the face recognition model is trained based on the twin face training data and the face recognition labels and mask classification labels corresponding to the twin face training data; the face recognition model is used to extract features of the input face image to be identified, and perform face recognition and mask classification on the extracted face features respectively, to obtain face recognition results and mask classification results.

[0022] It should be noted that the step numbers "S1N" in this manual do not represent the order of the twin multi-task face recognition method. Figure 2-Figure 3 The twin multi-task face recognition method of the present invention is described.

[0023] Step S11, obtaining a face image to be recognized.

[0024] It should be noted that the facial image to be identified can be an actually obtained facial image to be identified, and can be a facial image with or without a mask. It can be obtained according to the actual identification needs and is not further limited here.

[0025] Step S12, input the face image to be identified into the face recognition model, and obtain the face recognition result and mask classification result output by the face recognition model; wherein, the face recognition model is trained based on the twin face training data and the face recognition labels and mask classification labels corresponding to the twin face training data; the face recognition model is used to extract features of the input face image to be identified, and perform face recognition and mask classification on the extracted face features respectively, to obtain face recognition results and mask classification results.

[0026] In this embodiment, reference Figure 2 The face recognition model includes: a feature extraction layer, which extracts features from the input face image to be recognized to obtain face features, and sends the face features to the fully connected layer, and according to the target dimension obtained after model training, sends the corresponding face features to the face recognition layer; the fully connected layer reduces the dimension of the face features to obtain reduced dimension features; the mask classification layer classifies the reduced dimension features to obtain mask classification results; the face recognition layer determines the Euclidean distance between the received face features and each face feature in the previously constructed face feature library, and combines the preset threshold to determine whether the face features and the corresponding face features in the face feature library are the same person to obtain the face recognition result.

[0027] It should be noted that the target dimension is smaller than the dimension of the facial feature. For example, if the facial feature is 1*512, the target dimension can be 472. It is determined according to the dimension obtained after the model is finally trained. For example, the target dimension is 472, and the corresponding 40-dimensional blank is used. Thus, the facial features of 472 dimensions are identified to ensure the accuracy of face recognition while classifying the facial features of 512 dimensions (including 40-dimensional blanks). The blank space of 40 dimensions is fully utilized to isolate or distinguish the features of different tasks to a certain extent, so that the model can focus more accurately on the features related to the mask when performing mask detection, thereby improving the accuracy of mask detection. In addition, the determination of the target dimension can refer to the preset dimension in the training process below, which will not be repeated here.

[0028] It should be noted that the mask classification layer can be classified using the SoftMax function. The mask classification results include wearing a mask and not wearing a mask. The specific representation can be based on the actual design mark, such as 0 for not wearing a mask and 1 for wearing a mask. No further limitation is made here. In addition, the face features extracted and the face features in the face feature library are determined by a preset threshold to determine whether they are the same person, and the face recognition result is obtained. If the Euclidean distance is less than the preset threshold, it is considered to be the same person, otherwise it is considered not.

[0029] Correspondingly, the face image to be identified is input into the face recognition model to obtain the face recognition result and mask classification result output by the face recognition model, including: inputting the face image to be identified into the feature extraction layer for feature extraction to obtain the face features output by the feature extraction layer; inputting the face features into the fully connected layer for dimensionality reduction to obtain the reduced dimensionality features output by the fully connected layer; inputting the reduced dimensionality features into the mask classification layer for classification to obtain the mask classification results output by the mask classification layer; inputting the face features into the face recognition layer to determine the Euclidean distance with each face feature in the previously constructed face feature library, and combining with the preset threshold to determine whether the face features and the corresponding face features in the face feature library are the same person to obtain the face recognition result.

[0030] In an optional embodiment, before inputting the face image to be identified into the face recognition model, it also includes: obtaining twin face training data and face recognition labels and mask classification labels corresponding to the twin face training data; using the twin face training data as input data for training, and using the face recognition labels and mask classification labels as labels for training, training the model to be trained, and obtaining a face recognition model for face recognition and mask classification.

[0031] Specifically, the twin face training data includes unobstructed face training samples and mask-occluded face training samples corresponding to the faces of each unobstructed face training sample. Obtaining the twin face training data includes: obtaining the unobstructed face training samples and the face recognition labels and mask classification labels corresponding to the unobstructed face training samples; using the face masking tool MaskTheFace to mask the unobstructed face training samples to obtain the corresponding mask-occluded face training samples; the gradient boosting-based face detector in MaskTheFace is replaced with a maximum margin face detection algorithm based on a convolutional neural network CNN; based on the unobstructed face training samples and their corresponding mask-occluded face training samples, the twin face training data is obtained.

[0032] It should be noted that the unobstructed face training samples can be obtained by obtaining existing face recognition datasets and face test datasets in different environments. The face recognition labels corresponding to the unobstructed face training samples can be obtained by traversing the directory of the face test dataset, and the mask classification labels can be obtained by traversing the directory of the twin face training data. The face test dataset covers different postures, races, expressions, occlusions, lighting, and profiles, etc., so as to improve the performance, robustness, and generalization of the model in different scenarios and challenges by selecting diverse face test data. In addition, after obtaining the twin face training data, it also includes: preprocessing the twin face training data.

[0033] In an optional embodiment, after obtaining unobstructed face training samples and face recognition labels and mask classification labels corresponding to the unobstructed face training samples, the method includes: performing data cleaning on the obtained unobstructed face training samples and face recognition labels and mask classification labels corresponding to the unobstructed face training samples to remove noise data.

[0034] In an optional embodiment, a facial masking tool MaskTheFace is used to perform facial mask masking on unobstructed face training samples to obtain corresponding mask-occluded face training samples, including: performing face detection on each mask-occluded face training sample based on a maximum edge face detection algorithm of CNN to obtain face bounding box coordinates; performing key point detection on the unobstructed face training sample according to the face bounding box coordinates to obtain face key feature points; adjusting the size of a previously generated mask according to the face bounding box coordinates and the face key feature points; and synthesizing the adjusted mask with the unobstructed face training sample to obtain the corresponding mask-occluded face training sample.

[0035] Specifically, first, MaskTheFace uses a face detection algorithm to locate the face area in the image to determine the position and size of the face, so as to accurately add a mask in the subsequent steps; secondly, after detecting the face, MaskTheFace further uses a feature point detection algorithm to locate the key feature points of the face, such as the position of the eyes, nose and mouth, to obtain feature points that are crucial to accurately simulate the position and shape of the mask; finally, based on the detected feature points, MaskTheFace selects a mask template with the corresponding tilt angle to ensure the alignment and natural fusion of the mask and the face, and adjusts the selected mask template to an appropriate angle and size according to the estimated position of the mask feature points, thereby superimposing the adjusted mask image on the original face image.

[0036] Furthermore, after superimposing the mask, MaskTheFace is also used to: perform preset image processing operations, such as smoothing edges, adjusting colors and brightness, etc., to make the mask look more natural and real; save the processed image to generate a new image containing the face wearing the mask.

[0037] In addition, since MaskTheFace provides some different types of masks to choose from, mask templates can be generated based on different types of masks to increase the diversity of the generated masked face dataset. At the same time, various limitations such as facial angle, mask fit, lighting conditions, etc. are also considered to maximize the effectiveness and robustness of the face recognition algorithm.

[0038] It should be noted that after the adjusted mask is synthesized with the unobstructed face training samples, there are training samples that fail to be synthesized, so they need to be removed to match the mask-occluded face training samples of successfully wearing the mask with the corresponding unobstructed face training samples to obtain one-to-one corresponding twin face training data.

[0039] In addition, two models to be trained are selected for training, including a first model to be trained and a second model to be trained, and the first model to be trained and the second model to be trained share parameters. In other words, the first training model and the second model to be trained are the same model. After the training is completed, it is the above-mentioned face recognition model. In addition, the model to be trained can be an existing network built into the training device, which usually includes a network structure, or it can be other networks specified by the user, such as a lightweight face recognition MobileFaceNet network on a mobile terminal, etc., which is not further limited here.

[0040] Further, refer to Figure 3 , training the model to be trained, including: inputting the unobstructed face training sample into the first model to be trained, obtaining the first recognition result and the first classification result output by the first model to be trained, and inputting the mask-occluded face training sample corresponding to the unobstructed face training sample into the second model to be trained, obtaining the second recognition result and the second classification result output by the second model to be trained; wherein the first model to be trained and the second model to be trained share parameters; constructing an unobstructed loss function based on the first recognition result, the first classification result, and the face recognition labels and the mask classification labels corresponding to the unobstructed face training sample, and constructing a mask-occluded loss function based on the second recognition result, the second classification result, and the face recognition labels and the mask classification labels corresponding to the mask-occluded face training sample; obtaining a total loss function based on the unobstructed loss function and the mask-occluded loss function in combination with a weight distribution mechanism, and performing gradient backpropagation on the first model to be trained and the second model to be trained based on the total loss function using an optimizer to update the corresponding model parameters until the total loss function converges and the training ends.

[0041] Furthermore, the first model to be trained includes a first feature extraction layer, a first fully connected layer, a first mask classification layer, a first feature learning layer and a first face recognition layer; the unobstructed face training sample is input into the first model to be trained for feature extraction, and a first recognition result and a first classification result output by the first model to be trained are obtained, including: the unobstructed face training sample is input into the first feature extraction layer, and the first face training feature output by the first feature extraction layer is obtained; the first face training feature is input into the first fully connected layer for dimensionality reduction, and the first dimensionality reduction feature output by the first fully connected layer is obtained; the first dimensionality reduction feature is input into the first mask classification layer, and the first classification result output by the first mask classification layer is obtained; according to the first preset dimension, the first face training feature is input into the first feature learning layer to utilize the adaptive AdaFace processes the first face training feature of the first preset dimension, and upgrades the processing result according to the number of face recognition labels corresponding to the unobstructed face training samples to obtain the second upgraded feature output by the first feature learning layer; wherein, the first preset dimension is obtained by adjusting the first preset dimension used in the previous training according to the preset dimension adjustment rule based on the model accuracy determined after the previous training, the model accuracy determined after the previous training is obtained by using the ablation experiment based on the first recognition result and the first classification result obtained after the previous training, and the preset dimension adjustment rule is used to limit the adjustment relationship between the model accuracy and the corresponding preset dimension; the second upgraded feature is input into the first face recognition layer for probability classification to obtain the first recognition result output by the first face recognition layer.

[0042] In addition, the second model to be trained includes a second feature extraction layer, a second fully connected layer, a second mask classification layer, a second feature learning layer and a second face recognition layer; the mask-occluded face training samples corresponding to the unobstructed face training sample faces are input into the second model to be trained, and the second recognition result and the second classification result output by the second model to be trained are obtained, including: the mask-occluded face training samples corresponding to the unobstructed face training sample faces are input into the second feature extraction layer for feature extraction, and the second face training features output by the second feature extraction layer are obtained; the second face training features are input into the second fully connected layer for dimensionality reduction, and the third dimensionality reduction features are output by the second fully connected layer; the third dimensionality reduction features are input into the second mask classification layer, and the second classification results output by the second mask classification layer are obtained; according to the second preset dimension, the second face training features are input into the second In the feature learning layer, the second face training features of the second preset dimension are processed by using the adaptive face AdaFace, and the processing results are upgraded according to the number of face recognition labels corresponding to the training samples of the faces covered by masks, so as to obtain the fourth upgraded feature output by the second feature learning layer; wherein, the second preset dimension is obtained by adjusting the second preset dimension used in the previous training according to the preset dimension adjustment rule based on the model accuracy determined after the previous training, and the model accuracy determined after the previous training is obtained by using the ablation experiment based on the second recognition result and the second classification result obtained after the previous training, and the preset dimension adjustment rule is used to limit the adjustment relationship between the model accuracy and the corresponding preset dimension; the fourth upgraded feature is input into the second face recognition layer for probability classification to obtain the second recognition result output by the second face recognition layer.

[0043] It should be noted that when the corresponding facial features are input into the corresponding fully connected layer for dimensionality reduction, the dimensionality reduction can be performed based on the number of classification targets. For example, if there are 10 classification targets, the facial features are reduced to 10 dimensions. This is only an example and is not further limited. Similarly, when the corresponding facial training features are input into the corresponding feature learning layer for dimensionality increase, the dimensionality is increased according to the number of identities corresponding to the facial features in the facial feature library. For example, if the number of identities corresponding to the facial features in the facial feature library is 10,000, the facial features are increased to 10,000 dimensions. This is only an example and is not further limited. In addition, probabilistic classification can use softmax, etc., which is not further limited here.

[0044] In an optional embodiment, an unobstructed face loss function is constructed based on the first recognition result, the first classification result, and the face recognition labels and mask classification labels corresponding to the unobstructed face training samples, including: constructing a first classification loss function based on all the first classification results obtained during the training process and the mask classification labels of the corresponding unobstructed face training samples, combined with the cross entropy loss function; constructing a first recognition loss function based on the first recognition result and the face recognition labels corresponding to the unobstructed face training samples, combined with the cross entropy loss function; constructing an unobstructed face loss function based on the first classification loss function, the first recognition loss function and the first preset weight.

[0045] In addition, based on the second recognition result, the second classification result, and the face recognition labels and mask classification labels corresponding to the mask-occluded face training samples, a mask occlusion loss function is constructed, including: constructing a second classification loss function based on all the second classification results obtained during the training process and the mask classification labels of the corresponding mask-occluded face training samples, combined with the cross-entropy loss function; constructing a second recognition loss function based on the second recognition result and the face recognition labels corresponding to the mask-occluded face training samples, combined with the cross-entropy loss function; constructing a mask occlusion loss function based on the second classification loss function, the second recognition loss function and the first preset weight.

[0046] It should be added that both the unoccluded loss function and the mask occluded loss function can be expressed as: in, Indicates the corresponding unoccluded loss function or mask occluded loss function; represents a first preset weight; represents the first recognition loss function corresponding to the unoccluded loss function or the second recognition loss function corresponding to the mask occluded loss function; Represents the first classification loss function corresponding to the unoccluded loss function or the second classification loss function corresponding to the mask occluded loss function.

[0047] In an optional embodiment, a total loss function is obtained according to the unoccluded loss function and the mask occlusion loss function in combination with a weight allocation mechanism, including: according to the unoccluded loss function and the mask occlusion loss function in combination with a weight allocation mechanism, a first allocation weight is allocated to the unoccluded loss function, and a second allocation weight is allocated to the mask occlusion loss function; wherein the sum of the first allocation weight and the second allocation weight is 1, and the initially allocated first allocation weight is greater than the second allocation weight; the first allocation weight is reduced based on a preset value, and the second allocation weight is increased based on a preset value, and weights are iteratively allocated to the unoccluded loss function and the mask occlusion loss function until the second allocation weight approaches 1 and the first allocation weight approaches 0; according to the weights allocated to the unoccluded loss function and the mask occlusion loss function each time, the corresponding model accuracy is obtained using an ablation experiment; the weights allocated to the unoccluded loss function and the mask occlusion loss function corresponding to the maximum model accuracy are selected, and the total loss function is obtained by combining the unoccluded loss function and the mask occlusion loss function.

[0048] It should be added that the total loss function is expressed as: in, represents the total loss function; represents the first allocation weight assigned to the no-occlusion loss function; Represents the second allocation weight assigned to the mask occlusion loss function; represents the no-occlusion loss function, Represents the mask occlusion loss function, which can be found above. , which will not be repeated here.

[0049] In summary, the embodiment of the present invention uses a face recognition model to perform face recognition and mask classification on the input face image to be recognized, so as to greatly improve the accuracy of face recognition without reducing the accuracy of unobstructed face recognition. In actual application scenarios, especially in special periods such as when masks need to be worn, it can not only meet conventional face recognition needs, but also quickly determine whether people are wearing masks, which provides convenience for the safety management of the venue and the implementation of epidemic prevention measures. In addition, the model is trained using twin face training data to make full use of the characteristics of twin data and better capture the similarities and differences of faces, thereby improving the accuracy of the model in face recognition tasks. At the same time, combined with face recognition labels and mask classification labels for training, the model can learn the relationship between different face features and mask wearing status, improve the accuracy of mask classification, and improve the generalization ability of the model. It can flexibly adapt to various complex and changeable actual application scenarios and maintain good adaptability to the needs of different scenarios.

[0050] The twin multi-task face recognition device provided by the present invention is described below. The twin multi-task face recognition device described below and the twin multi-task face recognition method described above can be referenced to each other.

[0051] Figure 4 A schematic diagram of the structure of a twin multi-task face recognition device is shown, the device comprising: An image acquisition module 41 is used to acquire a face image to be recognized; The face recognition module 42 inputs the face image to be recognized into the face recognition model, and obtains the face recognition result and mask classification result output by the face recognition model; wherein, the face recognition model is trained based on the twin face training data and the face recognition labels and mask classification labels corresponding to the twin face training data; the face recognition model is used to extract features of the input face image to be recognized, and perform face recognition and mask classification on the extracted face features respectively, to obtain face recognition results and mask classification results.

[0052] It should be noted that the facial image to be identified can be an actually obtained facial image to be identified, and can be a facial image with or without a mask. It can be obtained according to the actual identification needs and is not further limited here.

[0053] In this embodiment, the face recognition model includes: a feature extraction layer, which extracts features from an input face image to be recognized to obtain face features, and sends the face features to a fully connected layer, and sends the corresponding face features to a face recognition layer according to the target dimension obtained after model training; a fully connected layer, which reduces the dimension of the face features to obtain reduced dimension features; a mask classification layer, which classifies the reduced dimension features to obtain mask classification results; a face recognition layer, which determines the Euclidean distance between the received face features and each face feature in a previously constructed face feature library, and determines, in combination with a preset threshold, whether the face features and the corresponding face features in the face feature library are of the same person to obtain a face recognition result.

[0054] Correspondingly, the face recognition module 42 includes: a feature extraction unit, which inputs the face image to be recognized into the feature extraction layer for feature extraction, and obtains the face features output by the feature extraction layer; a fully connected unit, which inputs the face features into the fully connected layer for dimensionality reduction, and obtains the reduced dimensionality features output by the fully connected layer; a classification unit, which inputs the reduced dimensionality features into the mask classification layer for classification, and obtains the mask classification results output by the mask classification layer; an identification unit, which inputs the face features into the face recognition layer to determine the Euclidean distance with each face feature in the previously constructed face feature library, and combines the preset threshold to determine whether the face features and the corresponding face features in the face feature library are the same person, and obtain the face recognition result.

[0055] In an optional embodiment, the device also includes: a training data acquisition module, which obtains twin face training data and face recognition labels and mask classification labels corresponding to the twin face training data before inputting the face image to be recognized into the face recognition model; a training module, which uses the twin face training data as input data for training, and uses the face recognition labels and mask classification labels as labels for training, trains the model to be trained, and obtains a face recognition model for face recognition and mask classification.

[0056] Specifically, the twin face training data includes unobstructed face training samples and mask-occluded face training samples corresponding to the faces of each unobstructed face training sample. The training data acquisition module includes: a data acquisition unit, which obtains the unobstructed face training samples and the face recognition labels and mask classification labels corresponding to the unobstructed face training samples; a mask masking unit, which uses the facial masking tool MaskTheFace to mask the unobstructed face training samples to obtain the corresponding mask-occluded face training samples; the gradient boosting-based face detector in MaskTheFace is replaced with a maximum margin face detection algorithm based on a convolutional neural network CNN; a twin data construction unit, which obtains the twin face training data based on the unobstructed face training samples and their corresponding mask-occluded face training samples.

[0057] In addition, the training data acquisition module also includes: a data preprocessing unit, which preprocesses the twin face training data after obtaining the twin face training data.

[0058] In an optional embodiment, the device also includes: a data cleaning module, which, after obtaining the unobstructed face training samples and the face recognition labels and mask classification labels corresponding to the unobstructed face training samples, performs data cleaning on the obtained unobstructed face training samples and the face recognition labels and mask classification labels corresponding to the unobstructed face training samples to remove noise data.

[0059] In an optional embodiment, the mask masking unit includes: a face detection subunit, which performs face detection on each mask-occluded face training sample based on the maximum edge face detection algorithm of CNN to obtain the face bounding box coordinates; a key point detection subunit, which performs key point detection on the unobstructed face training sample according to the face bounding box coordinates to obtain the face key feature points; a mask adjustment subunit, which adjusts the size of the previously generated mask according to the face bounding box coordinates and the face key feature points; and a synthesis subunit, which synthesizes the adjusted mask with the unobstructed face training sample to obtain the corresponding mask-occluded face training sample.

[0060] Specifically, first, MaskTheFace uses a face detection algorithm to locate the face area in the image to determine the position and size of the face, so as to accurately add a mask in the subsequent steps; secondly, after detecting the face, MaskTheFace further uses a feature point detection algorithm to locate the key feature points of the face, such as the position of the eyes, nose and mouth, to obtain feature points that are crucial to accurately simulate the position and shape of the mask; finally, based on the detected feature points, MaskTheFace selects a mask template with the corresponding tilt angle to ensure the alignment and natural fusion of the mask and the face, and adjusts the selected mask template to an appropriate angle and size according to the estimated position of the mask feature points, thereby superimposing the adjusted mask image on the original face image.

[0061] Furthermore, after superimposing the mask, MaskTheFace is also used to: perform preset image processing operations, such as smoothing edges, adjusting colors and brightness, etc., to make the mask look more natural and real; save the processed image to generate a new image containing the face wearing the mask.

[0062] In addition, since MaskTheFace provides some different types of masks to choose from, mask templates can be generated based on different types of masks to increase the diversity of the generated masked face dataset. At the same time, various limitations such as facial angle, mask fit, lighting conditions, etc. are also considered to maximize the effectiveness and robustness of the face recognition algorithm.

[0063] It should be noted that after the adjusted mask is synthesized with the unobstructed face training samples, there are training samples that fail to be synthesized, so they need to be removed to match the mask-occluded face training samples of successfully wearing the mask with the corresponding unobstructed face training samples to obtain one-to-one corresponding twin face training data.

[0064] In addition, two models to be trained are selected for training, including a first model to be trained and a second model to be trained, and the first model to be trained and the second model to be trained share parameters. In other words, the first training model and the second model to be trained are the same model. After the training is completed, it is the above-mentioned face recognition model. In addition, the model to be trained can be an existing network built into the training device, which usually includes a network structure, or it can be other networks specified by the user, such as a lightweight face recognition MobileFaceNet network on a mobile terminal, etc., which is not further limited here.

[0065] Furthermore, the training module includes: a model prediction unit, which inputs the unobstructed face training sample into the first model to be trained to obtain the first recognition result and the first classification result output by the first model to be trained, and inputs the mask-occluded face training sample corresponding to the unobstructed face training sample into the second model to be trained to obtain the second recognition result and the second classification result output by the second model to be trained; wherein the parameters of the first model to be trained and the second model to be trained are shared; a function construction unit, which constructs an unobstructed loss function based on the first recognition result, the first classification result, and the face recognition labels and mask classification labels corresponding to the unobstructed face training sample, and constructs a mask occlusion loss function based on the second recognition result, the second classification result, and the face recognition labels and mask classification labels corresponding to the mask-occluded face training sample; a training unit, which obtains the total loss function according to the unobstructed loss function and the mask occlusion loss function in combination with the weight distribution mechanism, and according to the total loss function, uses the optimizer to perform gradient back propagation on the first model to be trained and the second model to be trained to update the corresponding model parameters until the total loss function converges and the training ends.

[0066] Furthermore, the first model to be trained includes a first feature extraction layer, a first fully connected layer, a first mask classification layer, a first feature learning layer and a first face recognition layer, and the model prediction unit includes: a first feature extraction subunit, inputting the unobstructed face training sample into the first feature extraction layer for feature extraction, and obtaining the first face training feature output by the first feature extraction layer; a first fully connected subunit, inputting the first face training feature into the first fully connected layer for dimensionality reduction, and obtaining the first reduced dimensionality feature output by the first fully connected layer; a first classification subunit, inputting the first reduced dimensionality feature into the first mask classification layer, and obtaining the first classification result output by the first mask classification layer; a first feature learning subunit, inputting the first face training feature into the first feature learning layer according to a first preset dimension, so as to utilize the adaptive face AdaF ace processes the first face training feature of the first preset dimension, and upgrades the processing result according to the number of face recognition labels corresponding to the unobstructed face training samples, to obtain the second upgraded feature output by the first feature learning layer; wherein, the first preset dimension is obtained by adjusting the first preset dimension used in the previous training according to the model accuracy determined after the previous training, and the model accuracy determined after the previous training is obtained by using the ablation experiment based on the first recognition result and the first classification result obtained after the previous training, and the preset dimension adjustment rule is used to limit the adjustment relationship between the model accuracy and the corresponding preset dimension; the first recognition subunit inputs the second upgraded feature into the first face recognition layer for probability classification, to obtain the first recognition result output by the first face recognition layer.

[0067] In addition, the second model to be trained includes a second feature extraction layer, a second fully connected layer, a second mask classification layer, a second feature learning layer and a second face recognition layer; the model prediction unit also includes: a second feature extraction subunit, inputting the mask-occluded face training sample corresponding to the unobstructed face training sample face into the second feature extraction layer for feature extraction, and obtaining the second face training feature output by the second feature extraction layer; a second fully connected subunit, inputting the second face training feature into the second fully connected layer for dimensionality reduction, and obtaining the third dimensionality reduction feature output by the second fully connected layer; a second classification subunit, inputting the third dimensionality reduction feature into the second mask classification layer, and obtaining the second classification result output by the second mask classification layer; a second feature learning subunit, inputting the second face training feature into the second feature learning layer according to the second preset dimension, so as to utilize the adaptive AdaFace processes the second face training features of the second preset dimension, and upgrades the processing results according to the number of face recognition labels corresponding to the training samples of the faces covered by masks, to obtain the fourth upgraded features output by the second feature learning layer; wherein, the second preset dimension is obtained by adjusting the second preset dimension used in the previous training according to the model accuracy determined after the previous training, and the preset dimension adjustment rules, the model accuracy determined after the previous training is obtained by using the ablation experiment based on the second recognition result and the second classification result obtained after the previous training, and the preset dimension adjustment rules are used to limit the adjustment relationship between the model accuracy and the corresponding preset dimension; the second recognition subunit inputs the fourth upgraded feature into the second face recognition layer for probability classification, and obtains the second recognition result output by the second face recognition layer.

[0068] In an optional embodiment, the function construction unit includes: a first classification loss function construction subunit, which constructs a first classification loss function based on all first classification results obtained during the training process and the mask classification labels of the corresponding unobstructed face training samples, combined with the cross entropy loss function; a first recognition loss function construction subunit, which constructs a first recognition loss function based on the first recognition results and the face recognition labels corresponding to the unobstructed face training samples, combined with the cross entropy loss function; an unobstructed loss function construction subunit, which constructs an unobstructed loss function based on the first classification loss function, the first recognition loss function and the first preset weight.

[0069] In addition, the function construction unit also includes: a second classification loss function construction subunit, which constructs a second classification loss function based on all second classification results obtained during the training process and the mask classification labels of the corresponding mask-occluded face training samples, combined with the cross entropy loss function; a second recognition loss function construction subunit, which constructs a second recognition loss function based on the second recognition results and the face recognition labels corresponding to the mask-occluded face training samples, combined with the cross entropy loss function; a mask occlusion loss function construction subunit, which constructs a mask occlusion loss function based on the second classification loss function, the second recognition loss function and the first preset weight.

[0070] The training unit includes: a weight allocation subunit, which allocates a first allocation weight to the unoccluded loss function and a second allocation weight to the mask occlusion loss function according to the unoccluded loss function and the mask occlusion loss function in combination with a weight allocation mechanism; wherein the sum of the first allocation weight and the second allocation weight is 1, and the initially allocated first allocation weight is greater than the second allocation weight; the first allocation weight is reduced based on a preset value, and the second allocation weight is increased based on a preset value, and weights are allocated to the unoccluded loss function and the mask occlusion loss function iteratively until the second allocation weight approaches 1 and the first allocation weight approaches 0; a model accuracy determination subunit, which obtains the corresponding model accuracy according to the weights allocated to the unoccluded loss function and the mask occlusion loss function each time by using an ablation experiment; a total loss function determination subunit, which selects the weights allocated to the unoccluded loss function and the mask occlusion loss function corresponding to the maximum model accuracy, and obtains the total loss function by combining the unoccluded loss function and the mask occlusion loss function.

[0071] In summary, the embodiment of the present invention uses the face recognition model through the face recognition module to perform face recognition and mask classification on the face image to be recognized acquired by the image acquisition module, so as to greatly improve the accuracy of face recognition without reducing the accuracy of unobstructed face recognition. In actual application scenarios, especially in special periods such as when masks need to be worn, it can not only meet the conventional face recognition needs, but also quickly determine whether people are wearing masks, which provides convenience for the safety management of the venue and the implementation of epidemic prevention measures. In addition, the model is trained using twin face training data to make full use of the characteristics of twin data and better capture the similarities and differences of faces, thereby improving the accuracy of the model in face recognition tasks. At the same time, combined with face recognition labels and mask classification labels for training, the model can learn the relationship between different face features and mask wearing status, improve the accuracy of mask classification, improve the generalization ability of the model, and can flexibly adapt to various complex and changeable actual application scenarios, and maintain good adaptability to the needs of different scenarios.

[0072] Figure 5 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 5As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530 and a communication bus 540, wherein the processor 510, the communication interface 520 and the memory 530 communicate with each other through the communication bus 540. The processor 510 may call the logic instructions in the memory 530 to execute the twin multi-task face recognition method, which includes: obtaining a face image to be recognized; inputting the face image to be recognized into the face recognition model to obtain a face recognition result and a mask classification result output by the face recognition model; wherein the face recognition model is trained based on the twin face training data and the face recognition labels and mask classification labels corresponding to the twin face training data; the face recognition model is used to extract features of the input face image to be recognized, and perform face recognition and mask classification on the extracted face features respectively, to obtain a face recognition result and a mask classification result.

[0073] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0074] On the other hand, the present invention also provides a computer program product, which includes a computer program, and the computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the twin multi-task face recognition method provided by the above methods, and the method includes: obtaining a face image to be identified; inputting the face image to be identified into a face recognition model to obtain a face recognition result and a mask classification result output by the face recognition model; wherein the face recognition model is trained based on twin face training data and face recognition labels and mask classification labels corresponding to the twin face training data; the face recognition model is used to extract features of the input face image to be identified, and perform face recognition and mask classification on the extracted face features respectively to obtain face recognition results and mask classification results.

[0075] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the twin multi-task face recognition method provided by the above-mentioned methods, the method comprising: obtaining a face image to be identified; inputting the face image to be identified into a face recognition model to obtain a face recognition result and a mask classification result output by the face recognition model; wherein the face recognition model is trained based on twin face training data and face recognition labels and mask classification labels corresponding to the twin face training data; the face recognition model is used to extract features of the input face image to be identified, and perform face recognition and mask classification on the extracted face features respectively to obtain face recognition results and mask classification results.

[0076] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0077] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0078] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A twin multi-task face recognition method, characterized in that: include: Obtaining a face image to be recognized; Input the face image to be recognized into the face recognition model to obtain the face recognition result and the mask classification result output by the face recognition model; wherein the face recognition model is trained based on the twin face training data and the face recognition labels and mask classification labels corresponding to the twin face training data; The face recognition model is used to extract features of an input face image to be recognized, and perform face recognition and mask classification on the extracted face features to obtain face recognition results and mask classification results.

2. The twin multi-task face recognition method according to claim 1, characterized in that: The face recognition model comprises: The feature extraction layer extracts features from the input face image to be recognized, obtains face features, and sends the face features to the fully connected layer, and according to the target dimensions obtained after model training, sends the corresponding face features to the face recognition layer; A fully connected layer performs dimensionality reduction on the facial features to obtain dimensionality reduction features; The mask classification layer classifies the dimension reduction features to obtain mask classification results; The face recognition layer determines the Euclidean distance between the received face features and each face feature in the previously constructed face feature library, and combines a preset threshold to determine whether the face features and the corresponding face features in the face feature library are the same person, thereby obtaining a face recognition result.

3. The twin multi-task face recognition method according to claim 1, characterized in that: Before inputting the face image to be recognized into the face recognition model, the method further includes: Obtaining twin face training data and face recognition labels and mask classification labels corresponding to the twin face training data; wherein the twin face training data includes unobstructed face training samples and mask-occluded face training samples corresponding to each of the unobstructed face training sample faces; Input the unobstructed face training sample into a first model to be trained, and obtain a first recognition result and a first classification result output by the first model to be trained; and input the mask-occluded face training sample corresponding to the unobstructed face training sample into a second model to be trained, and obtain a second recognition result and a second classification result output by the second model to be trained; wherein the first model to be trained and the second model to be trained share parameters; Based on the first recognition result, the first classification result, and the face recognition labels and mask classification labels corresponding to the unobstructed face training samples, construct an unobstructed loss function; and based on the second recognition result, the second classification result, and the face recognition labels and mask classification labels corresponding to the mask-obstructed face training samples, construct a mask-obstructed loss function; According to the unoccluded loss function and the mask occluded loss function, combined with the weight distribution mechanism, a total loss function is obtained, and according to the total loss function, an optimizer is used to perform gradient backpropagation on the first model to be trained and the second model to be trained to update the corresponding model parameters until the total loss function converges and the training is terminated.

4. The twin multi-task face recognition method according to claim 3, characterized in that: The first model to be trained includes a first feature extraction layer, a first fully connected layer, a first mask classification layer, a first feature learning layer and a first face recognition layer; Inputting the unobstructed face training sample into the first model to be trained, and obtaining a first recognition result and a first classification result output by the first model to be trained, including: Inputting the unobstructed face training sample into the first feature extraction layer for feature extraction, and obtaining the first face training feature output by the first feature extraction layer; Inputting the first face training feature into the first fully connected layer for dimensionality reduction, thereby obtaining a first dimensionality reduction feature output by the first fully connected layer; Inputting the first dimension reduction feature into the first mask classification layer to obtain a first classification result output by the first mask classification layer; According to a first preset dimension, the first face training feature is input into the first feature learning layer, so as to process the first face training feature of the first preset dimension by using an adaptive face AdaFace, and the processing result is dimensionally upgraded according to the number of face recognition labels corresponding to the unobstructed face training sample, so as to obtain a second dimensionally upgraded feature output by the first feature learning layer; wherein, the first preset dimension is obtained by adjusting the first preset dimension corresponding to the previous training according to a preset dimension adjustment rule based on the model accuracy determined after the previous training, the model accuracy determined after the previous training is obtained by using an ablation experiment based on the first recognition result and the first classification result obtained after the previous training, and the preset dimension adjustment rule is used to limit the adjustment relationship between the model accuracy and the corresponding preset dimension; Inputting the second dimension-upgraded feature into the first face recognition layer for probabilistic classification to obtain a first recognition result output by the first face recognition layer; The second model to be trained includes a second feature extraction layer, a second fully connected layer, a second mask classification layer, a second feature learning layer and a second face recognition layer; the mask-occluded face training samples corresponding to the unobstructed face training sample faces are input into the second model to be trained, and a second recognition result and a second classification result output by the second model to be trained are obtained, including: Inputting the mask-occluded face training sample corresponding to the unobstructed face training sample face into the second feature extraction layer for feature extraction, and obtaining the second face training feature output by the second feature extraction layer; Inputting the second face training feature into the second fully connected layer for dimensionality reduction, to obtain a third dimensionality reduction feature output by the second fully connected layer; Inputting the third dimension reduction feature into the second mask classification layer to obtain a second classification result output by the second mask classification layer; According to the second preset dimension, the second face training feature is input into the second feature learning layer, so as to process the second face training feature of the second preset dimension by using the adaptive face AdaFace, and the processing result is dimensionally upgraded according to the number of face recognition labels corresponding to the mask-occluded face training sample, so as to obtain the fourth dimension-upgraded feature output by the second feature learning layer; wherein, the second preset dimension is obtained by adjusting the second preset dimension corresponding to the previous training according to the preset dimension adjustment rule based on the model accuracy determined after the previous training, the model accuracy determined after the previous training is obtained by using the ablation experiment based on the second recognition result and the second classification result obtained after the previous training, and the preset dimension adjustment rule is used to limit the adjustment relationship between the model accuracy and the corresponding preset dimension; The fourth up-dimensional feature is input into the second face recognition layer for probabilistic classification to obtain a second recognition result output by the second face recognition layer.

5. The twin multi-task face recognition method according to claim 3, characterized in that: Based on the first recognition result, the first classification result, and the face recognition label and the mask classification label corresponding to the unobstructed face training sample, an unobstructed loss function is constructed, including: Based on all the first classification results obtained during the training process and the mask classification labels of the corresponding unobstructed face training samples, combined with the cross entropy loss function, a first classification loss function is constructed; Constructing a first recognition loss function according to the first recognition result and the face recognition label corresponding to the unobstructed face training sample in combination with a cross entropy loss function; Constructing a non-occlusion loss function according to the first classification loss function, the first recognition loss function and the first preset weight; Based on the second recognition result, the second classification result, and the face recognition label and the mask classification label corresponding to the mask-occluded face training sample, a mask occlusion loss function is constructed, including: According to all the second classification results obtained during the training process and their corresponding mask classification labels of the mask-occluded face training samples, combined with the cross entropy loss function, a second classification loss function is constructed; Constructing a second recognition loss function according to the second recognition result and the face recognition label corresponding to the mask-occluded face training sample and the cross entropy loss function; A mask occlusion loss function is constructed according to the second classification loss function, the second recognition loss function and the first preset weight.

6. The twin multi-task face recognition method according to claim 3, characterized in that: Obtaining the twin face training data includes: Obtain unobstructed face training samples and face recognition labels and mask classification labels corresponding to the unobstructed face training samples; Using the facial mask tool MaskTheFace to mask the unobstructed face training sample, and obtain the corresponding mask-obstructed face training sample; wherein the gradient boosting-based face detector in MaskTheFace is replaced with a maximum margin face detection algorithm based on a convolutional neural network (CNN); The twin face training data is obtained according to the unobstructed face training samples and their corresponding mask-obstructed face training samples.

7. The twin multi-task face recognition method according to claim 3, characterized in that: According to the unoccluded loss function and the mask occluded loss function, combined with the weight distribution mechanism, the total loss function is obtained, including: According to the unoccluded loss function and the mask occluded loss function, in combination with a weight allocation mechanism, a first allocation weight is allocated to the unoccluded loss function, and a second allocation weight is allocated to the mask occluded loss function; wherein the sum of the first allocation weight and the second allocation weight is 1, and the initially allocated first allocation weight is greater than the second allocation weight; The first allocation weight is reduced based on a preset value, and the second allocation weight is increased based on the preset value, and weights are iteratively allocated to the unoccluded loss function and the mask occluded loss function until the second allocation weight approaches 1 and the first allocation weight approaches 0; According to the weights assigned to the unoccluded loss function and the mask occluded loss function each time, using an ablation experiment, the corresponding model accuracy is obtained; The weights assigned to the unoccluded loss function and the mask occlusion loss function corresponding to the maximum model accuracy are selected, and the total loss function is obtained by combining the unoccluded loss function and the mask occlusion loss function.

8. A twin multi-task face recognition device, characterized in that: include: An image acquisition module is used to acquire a face image to be recognized; A face recognition module inputs the face image to be recognized into a face recognition model to obtain a face recognition result and a mask classification result output by the face recognition model; wherein the face recognition model is trained based on twin face training data and face recognition labels and mask classification labels corresponding to the twin face training data; The face recognition model is used to extract features of an input face image to be recognized, and perform face recognition and mask classification on the extracted face features to obtain face recognition results and mask classification results.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the twin multi-task face recognition method as described in any one of claims 1 to 7 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, it implements the twin multi-task face recognition method as described in any one of claims 1 to 7.

Images