Training method of face recognition model for people wearing masks and recognition method of faces wearing masks

Abstract

The embodiments of the present application provide a training method for a face recognition model for a person wearing a mask and a method for recognizing a face wearing a mask. The training method constructs a face recognition model for a person wearing a mask based on a pre-trained full-face recognition model, inputs a first masked image of a person in a triplet into the face recognition model for a person wearing a mask, obtains features of the face wearing a mask of the first masked image, inputs a first image of the person not wearing a mask and a second image of the person not wearing a mask into the full-face recognition model, obtains features of the face not wearing a mask of the first image not wearing a mask and the second image not wearing a mask, and performs model training for the face recognition model for a person wearing a mask based on the obtained features of the face wearing a mask and the features of the face not wearing a mask, so that the face recognition model for a person wearing a mask can recognize the face of a person wearing a mask.

Description

Technical Field

[0001] The present application relates to the field of face recognition technology, and in particular to a training method for a face recognition model for people wearing masks and a method for recognizing faces wearing masks. Background Art

[0002] In scenarios where facial recognition is required, if users are required to take off their masks for facial recognition, it will cause inconvenience to users on the one hand, but on the other hand it will not be conducive to maintaining public health and safety, thus giving rise to the demand for facial recognition of people wearing masks. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a training method for a face recognition model for people wearing masks and a method for recognizing faces wearing masks, so as to realize face recognition for people wearing masks. The specific technical solutions are as follows:

[0004] In the first aspect, the present application provides a training method for a face recognition model wearing a mask, including:

[0005] Obtaining a pre-trained full-face recognition model; the full-face recognition model includes a first feature extraction network and a classification network;

[0006] Constructing a mask-wearing face recognition model based on the full face recognition model, wherein the mask-wearing face recognition model includes a second feature extraction network generated based on the first feature extraction network and the classification network;

[0007] Acquire a current triplet to be trained from the triplet image sample set, wherein the triplet includes a first mask-wearing image and a first mask-free image of the same person, and a second mask-free image of another person;

[0008] Inputting the first mask-wearing image into the mask-wearing face recognition model to obtain mask-wearing face features of the first mask-wearing face image;

[0009] Inputting the first image without a mask and the second image without a mask into the full face recognition model respectively, and obtaining the facial features of the first image without a mask and the second image without a mask;

[0010] Determine a first loss of the masked face recognition model based on the masked face features and the unmasked face features;

[0011] Based on the first loss, parameters of the second feature extraction network are adjusted to obtain a trained face recognition model for people wearing masks.

[0012] The training method of the masked face recognition model provided in the embodiment of the present application builds a masked face recognition model based on a pre-trained full-face recognition model, inputs a first masked image of a person in a triple into the masked face recognition model, obtains the masked face features of the first masked image, inputs a first maskless image of the person and a second maskless image of another person into the full-face recognition model, obtains the maskless face features of the first maskless image and the second maskless image, and performs model training for the masked face recognition model based on the obtained masked face features and maskless face features. By integrating the pre-trained full-face recognition model, the establishment and training process of the masked face recognition model can be simplified, and the model training of the masked face recognition model is performed on the basis of the triple, which can ensure that the trained masked face recognition model can still accurately identify the identity of the person to be identified without taking off the mask.

[0013] In one embodiment of the present application, the mask-wearing face recognition model is constructed based on the full face recognition model, including:

[0014] A multi-layer convolutional network is added between the first feature extraction network and the classification network to obtain the mask-wearing face recognition model; the second feature extraction network includes the first feature extraction network and the multi-layer convolutional network, the multi-layer convolutional network takes the output of the first feature extraction network as input, and the classification network takes the output of the multi-layer convolutional network as input.

[0015] By adding a multi-layer convolutional network between the first feature extraction network and the classification network to construct a face recognition model for people wearing masks, the construction process of the face recognition model for people wearing masks and its subsequent training process can be simplified.

[0016] In one embodiment of the present application, the parameter adjustment of the second feature extraction network based on the first loss to obtain a trained face recognition model for people wearing masks includes:

[0017] Based on the first loss, parameters of the multi-layer convolutional network in the second feature extraction network are adjusted to obtain a trained face recognition model for people wearing masks.

[0018] During the model training process, the original first feature extraction network is frozen. On the one hand, there is no need to completely retrain the model, which simplifies the training process. On the other hand, it helps to retain the full-face recognition capability of the original full-face recognition model, so that the masked face recognition model does not reduce its ability to recognize complete faces in full-face recognition scenarios.

[0019] In one embodiment of the present application, the classification network includes a global pooling network and a category determination network;

[0020] The step of inputting the first mask-wearing image into the mask-wearing face recognition model to obtain mask-wearing face features of the first mask-wearing face image includes:

[0021] Input the first mask-wearing image into the mask-wearing face recognition model to obtain the feature map output by the second feature extraction network and the first face feature output by the global pooling network;

[0022] The step of inputting the first image without a mask and the second image without a mask into the full face recognition model respectively to obtain the facial features of the first image without a mask and the second image without a mask, comprising:

[0023] Inputting the first mask-free image into the full face recognition model to obtain the second facial features output by the global pooling network;

[0024] Inputting the second mask-free image into the full face recognition model to obtain the third facial features output by the global pooling network;

[0025] The determining the first loss of the masked face recognition model based on the masked face features and the unmasked face features includes:

[0026] The feature map is discriminated by a discriminant model to obtain a discriminant result; the discriminant model is used to discriminate whether the input feature map is a feature map of an image with a mask or a feature map of an image without a mask;

[0027] Based on the discrimination result, determining the generation loss of the mask-wearing face recognition model;

[0028] Determine the ternary loss of the mask-wearing face recognition model based on the first face feature, the second face feature and the third face feature;

[0029] The first loss is determined based on the generation loss and the ternary loss.

[0030] In an embodiment of the present application, the two model training techniques of generative adversarial network and ternary loss are integrated to train the masked face recognition model, which can enable the second feature extraction network to extract effective features that are helpful in distinguishing people from the first masked image as much as possible, so as to achieve the effect of removing the mask at the feature level, and ensure that the masked face recognition model can accurately distinguish different people for the masked face image, thereby enhancing the recognition effect of the masked face recognition model for masked faces.

[0031] In one embodiment of the present application, the first loss is determined based on the following formula:

[0032] L1 =w 1 ×L G +w 2 ×L T

[0033] Among them, L 1 Characterizing the first loss, L G Characterize the generation loss, w 1 The weight coefficient that characterizes the generation loss, L T Characterize the ternary loss, w 2 A weight coefficient characterizing the ternary loss;

[0034] L G =log[1-D(G(z))], where log represents the logarithmic function, G(z) represents the feature map output by the second feature extraction network, and D(G(z)) represents the probability that the discriminant model discriminates the feature map output by the second feature extraction network as the feature map of the image without a mask;

[0035] L T =max(d(a,p)-d(a,n)+margin, 0), where a represents the first facial feature, p represents the second facial feature, n represents the third facial feature, margin is a preset constant, d(a,p) represents the distance between the first facial feature and the second facial feature, d(a,n) represents the distance between the first facial feature and the third facial feature, and max(d(a,p)-d(a,n)+margin, 0) represents the maximum value between d(a,p)-d(a,n)+margin and 0.

[0036] In one embodiment of the present application, the multi-layer convolutional network is a three-layer convolutional network or a five-layer convolutional network.

[0037] In the second aspect, the present application provides a method for recognizing a face wearing a mask, comprising:

[0038] Obtain the face image of the person wearing a mask to be identified;

[0039] Inputting the mask-wearing face image to be identified into a pre-trained mask-wearing face recognition model to obtain a face recognition result of the mask-wearing face image;

[0040] Wherein, the face recognition model for people wearing masks is obtained by training based on the method described in any one of the first aspects.

[0041] Based on the method for recognizing faces of people wearing masks provided in this application, it is possible to accurately identify the identity of a person without removing the mask.

[0042] In a third aspect, the present application provides an electronic device comprising a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the electronic device is triggered to execute the method described in any one of the first aspect or the second aspect.

[0043] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the method described in any one of the first aspect or the second aspect.

[0044] In a fifth aspect, the present application provides a computer program product, which includes executable instructions. When the executable instructions are executed on a computer, the computer executes the method described in any one of the first aspect or the second aspect.

[0045] It can be understood that the electronic device provided in the third aspect, the computer storage medium provided in the fourth aspect, and the computer program product provided in the fifth aspect are all used to execute the training method of the face recognition model with a mask or the recognition method of the face with a mask provided in this application. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0047] Figure 1 A schematic diagram of an application scenario of an embodiment of the present application;

[0048] Figure 2 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application;

[0049] Figure 3 A schematic diagram of the software module architecture of a terminal device provided in an embodiment of the present application;

[0050] Figure 4 A flowchart of a method for training a face recognition model for a person wearing a mask provided in an embodiment of the present application;

[0051] Figure 5 A schematic diagram of the network architecture of a full face recognition model provided in an embodiment of the present application;

[0052] Figure 6A schematic diagram of the network architecture of a face recognition model for people wearing masks provided in an embodiment of the present application;

[0053] Figure 7 A schematic diagram of a process for determining the first loss of a face recognition model for a person wearing a mask provided in an embodiment of the present application;

[0054] Figure 8 A schematic diagram of the training process of the face recognition model for wearing a mask provided in an embodiment of the present application;

[0055] Fig. 9 A flowchart of a method for recognizing a face wearing a mask provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] In order to better understand the technical solution of the present application, the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0057] In order to facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, the words "first" and "second" are used to distinguish the same items or similar items with substantially the same functions and effects. For example, the first instruction and the second instruction are to distinguish different user instructions, and their order is not limited. Those skilled in the art can understand that the words "first" and "second" do not limit the quantity and execution order, and the words "first" and "second" do not necessarily limit them to be different.

[0058] It should be noted that, in this application, words such as "exemplarily" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplarily" or "for example" in this application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplarily" or "for example" is intended to present related concepts in a specific way.

[0059] For ease of understanding, let's first combine Figure 1 The application scenarios of the present application are illustrated.

[0060] Figure 1 The application scenario shown includes a terminal device 100 and a server 200 .

[0061] The terminal device 100 may be a mobile phone, tablet computer, multimedia player, wearable device, desktop computer, biometric integrated device or other electronic device, and may be installed with relevant applications for face recognition requirements. For example, when the terminal device 100 is a mobile phone, the relevant application may be a transaction application, and when the terminal device 100 is a face recognition device installed in places such as railway stations and airports, the relevant application may be a security application for verifying the identity of a person.

[0062] The server 200 may be a single server, or a server cluster consisting of several servers, or a cloud computing server center, etc. In this application scenario, the training method for the face recognition model with a mask provided in this application may be used to train the model, and the trained face recognition model with a mask may be deployed on the server 200. Exemplarily, the model training process may be performed on the server 200, or may be performed first on other servers, and then the trained face recognition model with a mask may be copied to the server 200.

[0063] A communication connection is established between the terminal device 100 and the server 200, which can be a wired connection or a wireless connection.

[0064] After the trained masked face recognition model is deployed, when it is necessary to perform face recognition on a person wearing a mask, the terminal device 100 can collect images of the person wearing a mask, and send the collected images to the server 200, requesting the server 200 to identify the person therein. After receiving the masked image, the server 200 performs face recognition on the masked image through the masked face recognition model, obtains a face recognition result, and feeds the face recognition result back to the terminal device 100. After receiving the face recognition result, the terminal device 100 can perform processing corresponding to the face recognition result, for example, determining whether the real-name authentication is passed, whether to instruct the gate to release, etc.

[0065] As another example, after completing the model training of the face recognition model for people wearing masks in the server 200, the trained face recognition model for people wearing masks can be copied to the terminal device 100. Thus, after the terminal device 100 captures the image of a person wearing a mask, the face recognition model for people wearing masks can be directly used to perform face recognition on the image of people wearing masks.

[0066] Figure 2 A schematic diagram of a hardware architecture of the terminal device 100 is provided. Figure 2 The terminal device 100 may include a processor 110, an external memory access interface 120, an internal memory 121, an antenna 1, an antenna 2, a mobile communication module 130, a wireless communication module 140, a camera 151, a display screen 152, a sensor module 160, and the like.

[0067] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0068] The controller can be the nerve center and command center of the terminal device. The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0069] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0070] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuits sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0071] It is understandable that the interface connection relationship between the modules illustrated in this embodiment is only a schematic illustration and does not constitute a structural limitation on the terminal device. In other embodiments, the terminal device may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0072] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in terminal device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network.

[0073] The mobile communication module 130 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to terminal devices. The mobile communication module 130 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 130 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 130 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1.

[0074] The modulation and demodulation processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is transmitted to the application processor. The application processor can display images or videos through the display screen 152.

[0075] The wireless communication module 140 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to terminal devices. The wireless communication module 140 can be one or more devices integrating at least one communication processing module. The wireless communication module 140 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 140 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0076] In some embodiments, the antenna 1 of the terminal device is coupled to the mobile communication module 130, and the antenna 2 is coupled to the wireless communication module 140, so that the terminal device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0077] The number of cameras 151 may be 1 to N. For example, the terminal device 100 may include 2 front cameras and 4 rear cameras.

[0078] The external memory access interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal device 100. The external memory card communicates with the processor 110 through the external memory access interface 120 to implement a data storage function. For example, files such as music and videos are stored in an external memory card. The internal memory 121 can be used to store computer executable program codes, and the executable program codes include instructions. The processor 110 can execute various functional applications and data processing of the terminal device 100 by running the instructions stored in the internal memory 121. For example, in an embodiment of the present application, the processor 110 can execute instructions stored in the internal memory 121, and the internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area can store data created during the use of the terminal device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0079] exist Figure 1 In the illustrated scenario, the terminal device 100 can collect the masked facial images of people wearing masks in the environment through the camera 151, and send the masked facial images to the server 200 with the help of the communication function of the mobile communication module 130 or the wireless communication module 140. The server 200 performs facial recognition on the masked facial images and feeds back the facial recognition results to the terminal device 100.

[0080] The software system of the terminal device 100 may adopt a layered architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the terminal device 100.

[0081] Figure 3 It is a software structure block diagram of the terminal device of an embodiment of the present application.

[0082] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the system library of the Android runtime (Android runtime), and the kernel layer.

[0083] The application layer can include a series of application packages.

[0084] like Figure 3As shown, the application package may include applications such as camera, face recognition, etc.

[0085] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0086] like Figure 3 As shown, the application framework layer may include a window manager, a content provider, a resource manager, and the like.

[0087] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, touch the screen, drag the screen, capture the screen, etc.

[0088] Content providers are used to store and retrieve data and make it accessible to applications. The data can include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0089] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0090] Android runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0091] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0092] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0093] The system library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0094] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0095] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MP3 (Moving Picture Experts Group Audio Layer3), AAC (Advanced Audio Coding), JPEG (Joint Photographic Experts Group), PNG (Portable Network Graphics), etc.

[0096] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0097] A 2D graphics engine is a drawing engine for 2D drawings.

[0098] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, sensor driver, etc.

[0099] The training method of the mask face recognition model provided in this application is described in detail below in conjunction with specific embodiments.

[0100] See also Figure 4 The face recognition model for people wearing masks provided in the embodiment of the present application includes the following steps:

[0101] Step S101: Obtain a pre-trained full face recognition model; the full face recognition model includes a first feature extraction network and a classification network.

[0102] The pre-trained full-face recognition model is specifically a face recognition model for identifying faces without masks. It should be noted that this application is specifically based on the pre-trained full-face recognition model to construct an initial face recognition model for wearing masks, and then train the face recognition model for wearing masks to obtain a face recognition model for wearing masks that can recognize faces wearing masks.

[0103] In actual applications, you can obtain a trained full-face recognition model through a public database, or you can train a full-face recognition model yourself based on an existing algorithm.

[0104] As an example, a full face recognition model might use Figure 5The model structure shown in the figure includes a first feature extraction network and a classification network. The first feature extraction network takes the actual face image as input and is used to extract a feature map based on the face image. The classification network is used to further process the feature map extracted by the first feature network and output a classification label for the face image recognition. The classification label is used to identify the specific person recognized from the face image.

[0105] The first feature extraction network may specifically be a CNN (Convolutional Neural Networks) network. The number of convolutional layers, the number and size of convolutional kernels in the CNN network may be selected in accordance with actual needs, and the embodiments of the present application are not limited thereto.

[0106] The classification network may include a global pooling network and a category determination network. The global pooling network is used to perform dimensionality reduction processing on the feature map obtained by the first feature extraction network and output a feature vector of the facial features. The category determination network is used to perform category determination based on the facial features and output the corresponding classification label. Exemplarily, the global pooling network may be a global average pooling layer, and the category determination network may be an activation function layer, such as a softmax (an activation function) layer.

[0107] As an example, if you want to train a full face recognition model yourself, you can first build a Figure 5 An initial model of the architecture shown is obtained, and a full-face dataset of people without masks is obtained. Based on the full-face dataset, the initial model is trained by the Arcface Loss (a model training algorithm) algorithm to obtain a pre-trained full-face recognition model.

[0108] Step S102: construct a mask-wearing face recognition model based on the full face recognition model, and the mask-wearing face recognition model includes a second feature extraction network and a classification network generated based on the first feature extraction network.

[0109] Exemplarily, the first feature extraction network can be directly used as the second feature extraction network, or the network architecture of the first feature extraction network can be improved or expanded to obtain the second feature extraction network.

[0110] Step S103: obtaining a triplet to be trained from the ternary image sample set, wherein the triplet includes a first image of the same person wearing a mask and a first image of the same person not wearing a mask, and a second image of the same person not wearing a mask.

[0111] The triplet image sample set contains multiple triplets, each of which contains images of the same person wearing a mask and without a mask, as well as an image of another person without a mask.

[0112] As an example, we can first obtain a full face dataset of complete face images from a public database, and then add masks to the faces in these complete face images based on image processing technology to obtain a mask-wearing image dataset. Then, we construct a triple based on the original full face dataset and the mask-wearing image dataset to obtain a triple image sample set.

[0113] Step S104: input the first mask-wearing face image into the mask-wearing face recognition model to obtain the mask-wearing face features of the first mask-wearing face image.

[0114] Those skilled in the art will understand that the classification model needs to first identify feature data from the input image, and then output the corresponding classification label based on the feature data. In the embodiment of the present application, the first mask-wearing image in the triplet is specifically input into the mask-wearing face recognition model established based on the aforementioned step S102 to obtain the feature data (mask-wearing face features) identified by the mask-wearing face recognition model from the first mask-wearing image, and then the loss of the mask-wearing face recognition model is determined based on the feature data.

[0115] Step S105: Input the first image without a mask and the second image without a mask into the full face recognition model respectively to obtain the facial features of the first image without a mask and the second image without a mask.

[0116] Specifically, the first image without a mask in the triplet is input into the full face recognition model to obtain the feature data (features of the face without a mask) recognized by the full face recognition model from the first image without a mask, and the second image without a mask in the triplet is input into the full face recognition model to obtain the feature data recognized by the full face recognition model from the second image without a mask.

[0117] Step S106: Determine the first loss of the masked face recognition model based on the features of the face wearing a mask and the face not wearing a mask.

[0118] Step S107: Adjust parameters of the second feature extraction network based on the first loss to obtain a trained face recognition model for people wearing masks.

[0119] Specifically, the first loss of the masked face recognition model is determined based on the masked face features identified by the masked face recognition model from the first masked face image, the unmasked face features identified by the full face recognition model from the first unmasked face image, and the unmasked face features identified by the full face recognition model from the second unmasked face image. Then, based on the first loss, the parameters of the second feature extraction network in the masked face recognition model are adjusted until the preset termination training condition is met, and a trained masked face recognition model can be obtained.

[0120] The purpose of this training process is to make the feature data recognized by the masked face recognition model from an image of a person wearing a mask as close as possible to the feature data recognized by the full face recognition model from an image of the same person not wearing a mask, and as far as possible from the feature data recognized by the full face recognition model from an image of another person not wearing a mask.

[0121] After achieving the training goal, it can be ensured that when the masked face image to be identified is input into the trained masked face recognition model, the masked face recognition model outputs the most accurate classification label possible based on the feature data identified from the masked face image, thereby realizing the recognition of the masked face.

[0122] Exemplarily, after obtaining the feature data of the first masked image, the first non-mask image, and the second non-mask image in the triplet based on the above steps S104-S105, the first loss can be calculated based on the first similarity between the feature data of the first masked image and the first non-mask image, and the second similarity between the feature data of the first masked image and the second non-mask image. The masked face recognition model is trained based on the first loss to increase the first similarity and reduce the second similarity to improve the recognition ability of the masked face recognition model for masked face images.

[0123] The training method of the masked face recognition model provided in the embodiment of the present application builds a masked face recognition model based on a pre-trained full-face recognition model, inputs a first masked image of a person in a triple into the masked face recognition model, obtains the masked face features of the first masked image, inputs a first maskless image of the person and a second maskless image of another person into the full-face recognition model, obtains the maskless face features of the first maskless image and the second maskless image, and performs model training for the masked face recognition model based on the obtained masked face features and maskless face features. By integrating the pre-trained full-face recognition model, the establishment and training process of the masked face recognition model can be simplified, and the model training of the masked face recognition model is performed on the basis of the triple, which can ensure that the trained masked face recognition model can still accurately identify the identity of the person to be identified without taking off the mask.

[0124] In one embodiment of the present application, in order to retain the full face recognition capability of the original full face recognition model in the face recognition model for people wearing masks, a multi-layer convolutional network can be added between the first feature extraction network and the classification network to realize the construction of the face recognition model for people wearing masks.

[0125] exist Figure 5 The model structure of the mask-wearing face recognition model built on the basis of the schematic full face recognition model is as follows Figure 6 As shown, it includes a second feature extraction network and a classification network. The second feature extraction network includes the original first feature extraction network in the full face recognition model and a newly added multi-layer convolutional network.

[0126] In the mask-wearing face recognition model, the newly added multi-layer convolutional network uses the output of the first feature extraction network as input, and is used to further convolve the first feature map extracted by the first feature extraction network for the input image, and output the second feature map. The classification network uses the output of the multi-layer convolutional network as input, and is used to further process the second feature map, and output the classification label obtained for the input image.

[0127] The number of layers of the newly added multi-layer convolutional network, as well as the number and size of the convolutional kernels can be selected in combination with actual needs, and the embodiments of the present application do not limit this. For example, the multi-layer convolutional network can be a three-layer convolutional network or a five-layer convolutional network.

[0128] One point that those skilled in the art should note when setting up a multi-layer convolutional network is that in order to ensure the rationality of the internal operation logic of the constructed mask-wearing face recognition model, the dimensions (width, height, number of channels) of the second feature map and the first feature map should be consistent. As an example, if Figure 6 In the illustrated network structure, the first feature extraction network is used to process the input image of 112×112×3 into a first feature map of 7×7×512. After the multi-layer convolutional network performs convolution processing on the first feature map to obtain the second feature map, the dimension of the second feature map should still be 7×7×512.

[0129] After establishing a masked face recognition model by adding a multi-layer convolutional network to the full face recognition model, part of the first feature extraction network in the masked face recognition model can be frozen. In the subsequent model training process, only the parameters of the multi-layer convolutional network in the masked face recognition model are adjusted based on the first loss to obtain the trained masked face recognition model.

[0130] In an embodiment of the present application, a masked face recognition model is established by adding a multi-layer convolutional network between the first feature extraction network and the classification network, and the original first feature extraction network is frozen during the model training process. On the one hand, there is no need to completely retrain the model, which simplifies the training process. On the other hand, it also helps to retain the full-face recognition capability of the original full-face recognition model, so that the masked face recognition model does not reduce its ability to recognize complete faces in full-face recognition scenarios.

[0131] Therefore, in actual application scenarios, after obtaining the masked face recognition model based on the training method of the masked face recognition model provided in the embodiment of the present application, the trained masked face recognition model can be used to be compatible with face recognition of faces wearing masks and face recognition of complete faces without masks.

[0132] In one embodiment of the present application, after obtaining the current triplet to be trained from the triplet image sample set, specifically based on Figure 7 The following steps are shown to determine the first loss of the mask face recognition model:

[0133] Step S201: Input the first masked face image into the masked face recognition model to obtain the feature map output by the second feature extraction network and the first face feature output by the global pooling network.

[0134] Step S202: Input the first mask-free image into the full face recognition model to obtain the second facial features output by the global pooling network.

[0135] Step S203: Input the second image without a mask into the full face recognition model to obtain the third facial features output by the global pooling network.

[0136] Step S204: discriminate the feature map through the discriminant model to obtain a discrimination result; the discriminant model is used to discriminate whether the input feature map is a feature map of an image with a mask or a feature map of an image without a mask.

[0137] Step S205: Based on the discrimination result, determine the generation loss of the mask-wearing face recognition model.

[0138] Step S206: Determine the ternary loss of the mask-wearing face recognition model based on the first facial feature, the second facial feature and the third facial feature.

[0139] Step S207: Determine the first loss based on the generated loss and the ternary loss.

[0140] The above steps are explained in detail below. Specifically, the embodiment of the present application combines two model training technologies, Generative Adversarial Network (GAN) and Triplet Loss, to train the face recognition model for people wearing masks. For ease of understanding, the two model training technologies are explained separately:

[0141] The Generative Adversarial Network consists of two models, one is the Generative Model (Generator) and the other is the Discriminator Model (Discriminator). When applying the Generative Adversarial Network, it is necessary to first give a real sample set (referred to as the real sample set). The Generative Network is used to generate another sample set similar to the real sample set (referred to as the fake sample set); the Discriminator model is used to discriminate between the positive sample set and the fake sample set, specifically to discriminate whether the input sample is a real sample or a fake sample. In the Generative Adversarial Network, the training goal of the Generative Model is to make the Discriminator model discriminate the fake samples generated by itself as real samples as much as possible, and the training goal of the Discriminator model is to discriminate the fake samples as fake samples as much as possible, and the real samples as real samples.

[0142] In practical applications, the generative model and the discriminative model are generally trained separately in alternating iterations. For example, you can first obtain a true sample set, and generate a false sample set through the initial generative model, discriminate between the true sample set and the false sample set through the discriminative model, and update the model parameters of the discriminative model based on the discrimination results; then fix the discriminative model, discriminate between the true sample set and the false sample set based on the updated discriminative model, and update the model parameters of the generative model based on the discrimination results; then fix the generative model, generate a new false sample set based on the updated generative model, discriminate between the true sample set and the new false sample set through the discriminative model, and update the model parameters of the discriminative model based on the discrimination results, and so on. When a certain number of iterations are reached, the training results will tend to be stable.

[0143] Ternary loss is often used in the field of face recognition to improve the model's ability to distinguish different people. Ternary loss involves three data points, namely Anchor, Positive, and Negative. The anchor is the central data point to be embedded, the positive sample point is a data point similar to the anchor (for example, the positive sample point and the anchor are images of the same person), and the negative sample point is a data point completely different from the anchor (for example, the negative sample point and the anchor are images of different people). The training goal of the model training based on ternary loss is to make the feature distance between the anchor and the positive sample point as small as possible, and the distance between the anchor and the negative sample point as large as possible.

[0144] The following is a detailed description of the process of training the mask-wearing face recognition model based on the above two model training techniques:

[0145] In order to ensure that the second feature extraction network can extract as many effective features as possible from the first masked image that are helpful in distinguishing people, so as to achieve the effect of removing masks at the feature level, in an embodiment of the present application, the second feature extraction network is used as a generative model and combined with the discriminative model to train a generative adversarial network.

[0146] Specifically, the first image without a mask in the triplet can be input into the full face recognition model to obtain the feature map output by the first feature extraction network in the full face recognition model for the first image without a mask, and the feature map is used as a true sample, denoted as x; the first image with a mask in the triplet is input into the face recognition model with a mask to obtain the feature map output by the second feature extraction network in the face recognition model with a mask for the first image with a mask, and the feature map is used as a false sample, denoted as G(z). The discriminant model is used to discriminate the input feature map to determine whether the input feature map is a feature map of a masked image or a feature map of a non-masked image. The discriminant model's discriminant result for the true sample x is denoted as D(x), which represents the probability that the discriminant model discriminates the true sample x as a feature map of a non-masked image. The discriminant model's discriminant result for the false sample G(z) is denoted as D(G(z)), which represents the probability that the discriminant model discriminates the false sample G(z) as a feature map of a non-masked image.

[0147] In the actual training process, we can first obtain x based on the full face recognition model, and input the first masked image into the initial second feature extraction network. The feature map output by the second feature extraction network is used as G(z). The discriminant model is used to discriminate x and G(z). The discriminant loss of the discriminant model is determined based on the discriminant results D(x) and D(G(z)), and the parameters of the discriminant model are updated based on the discriminant loss.

[0148] Exemplarily, the discriminant loss of the discriminant model can be determined based on the following formula:

[0149] L D =-{log[D(x)]+log[1-D(G(z))]};

[0150] Among them, L D represents the discriminant loss, and log represents the logarithmic function. The training goal of the discriminant model is to minimize the discriminant loss.

[0151] After completing the preliminary update of the discriminant model parameters, the discriminant model can be fixed, and the feature map G(z) output by the second feature extraction network is discriminated by the updated discriminant model, and the generation loss of the generation model is determined based on the discrimination result D(G(z)).

[0152] Exemplarily, the generation loss of the generation model can be determined based on the following formula:

[0153] L G =log[1-D(G(z))];

[0154] Among them, L G Characterize the generation loss.

[0155] In the embodiment of the present application, after the generation loss is obtained, it is necessary to combine the ternary loss to determine the first loss of the mask-wearing face recognition model. The process of determining the ternary loss is described below:

[0156] As mentioned above, the classification network usually includes a global pooling network and a category determination network. The global pooling network is used to reduce the dimension of the feature map output by the second feature extraction network and output the reduced-dimensional face features so that the category determination network can output the category label based on the face features. Figure 6 In the illustrated model structure, the dimension of the feature map output by the second feature extraction network is 7×7×512, and the global pooling network can process the feature map into 1×1×512 facial features.

[0157] Since the facial features output by the global pooling network are the final basis for the masked face recognition model to derive classification labels, in order to ensure the discrimination of different people by the masked face recognition model, the embodiment of the present application specifically calculates the ternary loss based on the facial features output by the global pooling network.

[0158] In the embodiment of the present application, the first masked image in the triplet is input into the masked face recognition model, the first face feature output by the global pooling network in the masked face recognition model for the first masked image is obtained, and the first face feature is used as the Anchor; the first maskless image in the triplet is input into the full face recognition model, the second face feature output by the global pooling network in the full face recognition model for the first maskless image is obtained, and the second face feature is used as the Positive; the second maskless image in the triplet is input into the full face recognition model, the third face feature output by the global pooling network in the full face recognition model for the second maskless image is obtained, and the third face feature is used as the Negative. Then the triplet loss is determined based on the first face feature, the second face feature, and the third face feature.

[0159] Exemplarily, the ternary loss may be determined based on the following formula:

[0160] L T =max(d(a,p)-d(a,n)+margin, 0);

[0161] Among them, L Trepresents the ternary loss, a represents the first facial feature, p represents the second facial feature, n represents the third facial feature, margin is a preset constant, d(a,p) represents the distance between the first facial feature and the second facial feature, d(a,n) represents the distance between the first facial feature and the third facial feature, and max(d(a,p)-d(a,n)+margin, 0) represents the maximum value between d(a,p)-d(a,n)+margin and 0.

[0162] For example, when calculating d(a,p) and d(a,n), the distance between features may be calculated using a calculation method such as Euclidean distance, Manhattan distance, or Chebyshev distance.

[0163] After the generation loss and the ternary loss are obtained, the first loss of the mask recognition model can be determined by combining the two losses. In one embodiment of the present application, the first loss can be determined based on the following formula:

[0164] L 1 =w 1 ×L G +w 2 ×L T ;

[0165] Among them, L 1 Characterize the first loss, w 1 The weight coefficient representing the generation loss, w 2 The weight coefficient that characterizes the ternary loss.

[0166] As an example, we can adopt an equal weight strategy and take w 1 and w 2 = 1, L G +L T As the first loss for model training. Alternatively, you can also set w based on actual needs 1 and w 2 , then based on the set w 1 and w 2 Determine the first loss for model training.

[0167] As another example, w may be adjusted based on a manual adjustment method, a grid search method, or an automatic adjustment algorithm for weight coefficients. 1 and w 2 Dynamic adjustments are made to determine the first loss.

[0168] After the first loss is obtained, the parameters of the second feature extraction network in the masked face recognition model can be adjusted based on the first loss. In the embodiment of the present application, the training goal of the masked face recognition model is specifically to minimize the first loss.

[0169] In addition, since the embodiment of the present application integrates the model training technology of the generative adversarial network, after calculating the first loss based on the above description and adjusting the parameters of the second feature extraction network based on the first loss, the second feature extraction network can be fixed, and the first mask image is input into the second feature extraction network after parameter adjustment, and the new feature map output by the second feature extraction network is used as the new G(z), and the discriminant loss of the discriminant model is calculated based on the discriminant model for the new G(z) and the discriminant result of x, and the parameters of the discriminant model are updated based on the discriminant loss. After the parameters of the discriminant model are updated, the discriminant model can be fixed again, and the generation loss and ternary loss of the masked face recognition network are determined based on the above description to calculate the first loss, and a new round of parameter updates are performed on the masked face recognition model. Therefore, the training of the masked face recognition model and the discriminant network can be performed alternately and iteratively until the preset termination condition is met. For example, the preset termination condition can be set to reach a preset number of training times, the loss function tends to be stable, or the loss function meets the preset condition, etc.

[0170] In an embodiment of the present application, the two model training techniques of generative adversarial network and ternary loss are integrated to train the masked face recognition model, which can enable the second feature extraction network to extract effective features that are helpful in distinguishing people from the first masked image as much as possible, so as to achieve the effect of removing the mask at the feature level, and ensure that the masked face recognition model can accurately distinguish different people for the masked face image, thereby enhancing the recognition effect of the masked face recognition model for masked faces.

[0171] For ease of understanding, Figure 8 FIG. 1 illustrates a training method for a face recognition model with a mask provided in an embodiment of the present application, and an example of a training process for training a face recognition model with a mask, wherein the training process includes:

[0172] Freeze the first feature extraction network in the full face recognition model, add a multi-layer convolutional network between the first feature extraction network and the classification network, and obtain a mask-wearing face recognition model.

[0173] The first masked image of 112×112×3 is input into the masked face recognition model, and the first feature extraction network outputs a feature map of 7×7×512. The feature map is processed by the multi-layer convolutional network to output another feature map of 7×7×512. The feature map output by the multi-layer convolutional network for the first masked image and the feature map output by the first feature extraction network for the first non-mask image are discriminated by the discriminant network, and the generation loss of the masked face recognition model is determined based on the discrimination result.

[0174] The first facial feature of 1×1×512 output by the global pooling network in the masked face recognition model for the first masked image is used as the anchor point, the second facial feature of 1×1×512 output by the global pooling network in the full face recognition model for the first unmasked image is used as the positive sample point, and the third facial feature of 1×1×512 output by the global pooling network in the full face recognition model for the second unmasked image is used as the negative sample point to determine the ternary loss of the masked face recognition model.

[0175] The first loss is determined based on the generation loss and the ternary loss, and the parameters of the convolutional network in the masked face recognition model are adjusted according to the first loss. The purpose of the training process is to make the feature map extracted from the masked image by the masked face recognition model close to the feature map of the non-mask image, so as to achieve the effect of removing the mask at the feature level, and make the feature distance of the facial features recognized by the masked face recognition model for a person's masked image close to the feature distance of the facial features of the same person's non-mask image, and greater than the feature distance of the facial features of non-mask images of different people.

[0176] Specifically, the current masked face recognition solutions mainly adopt two ideas: one is to deploy a masked face recognition model and a non-mask face recognition model in the algorithm respectively. These two models are independent of each other and can ensure their own accuracy. The disadvantage is that deploying two models will result in greater storage space overhead; the second is to use the masked face dataset and the full-face dataset as training sets together, and use the same model to be compatible with both masked and non-mask scenarios. The disadvantage is that it will cause the model's full-face recognition rate to drop.

[0177] Compared with the above two solutions, the masked face recognition model obtained by the above training method provided in the embodiment of the present application is plug-and-play. Since the first feature extraction network is frozen during the training process, all original data sets are not required, only a small amount of masked image data sets are required, and there is no need to completely retrain the model. The training difficulty of the model is low and the time consumption is short. In addition, since the original first feature extraction network in the full face recognition model is frozen, it is possible to optimize the recognition rate of the model for the masked face while ensuring that the recognition rate of the model for the full face does not decrease. It can be applied to the recognition of masked faces and complete faces at the same time. Compared with the masked face recognition model deployed solely for the mask recognition scenario, it can save the amount of parameters.

[0178] Based on the same inventive concept, the present application also provides a method for recognizing a face wearing a mask, see Fig. 9 , the method comprising:

[0179] Step S301: Obtain a face image of a person wearing a mask to be identified.

[0180] by Figure 1 Taking the illustrated scenario as an example, the terminal device 100 can specifically collect images of people wearing masks in the scene to obtain images of people wearing masks to be identified.

[0181] Step S302: input the masked face image to be identified into a pre-trained masked face recognition model to obtain a face recognition result of the masked face image.

[0182] Among them, the pre-trained masked face recognition model is trained based on the training method of the masked face recognition model provided in any of the aforementioned embodiments of the present application. The training process can be found in the description in the previous text.

[0183] by Figure 1 Taking the illustrated scenario as an example, after the terminal device 100 collects the face image of the person wearing a mask to be identified, it can send the image to the server 200, and the server 200 performs face recognition on the image through its internal face recognition model for wearing a mask, and returns the face recognition result to the terminal device 100. Alternatively, the face recognition model for wearing a mask in the server 200 can be copied to the terminal device 100 in advance, so that the terminal device 100 can directly recognize the collected face image of the person wearing a mask based on the face recognition model for wearing a mask.

[0184] In a specific implementation, the present application also provides an electronic device, which includes a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the electronic device is triggered to execute the above-mentioned training method embodiment of the face recognition model with a mask, or part or all of the steps in the embodiment of the method for recognizing a face with a mask.

[0185] In a specific implementation, the present application also provides a computer storage medium, wherein the computer storage medium may store a program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the above-mentioned embodiment of the training method for the face recognition model wearing a mask, or some or all steps in the embodiment of the recognition method for the face wearing a mask. The above-mentioned storage medium may be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0186] In a specific implementation, an embodiment of the present application also provides a chip system, which is applied to an electronic device. The chip system includes one or more processors, and the processors are used to call computer instructions so that the electronic device executes the above-mentioned training method embodiment of the face recognition model with a mask, or part or all of the steps in the embodiment of the method for recognizing a face with a mask.

[0187] The various embodiments of the mechanism disclosed in the present application can be implemented in hardware, software, firmware or a combination of these implementation methods. The embodiments of the present application can be implemented as a computer program or program code executed on a programmable system, which includes at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device and at least one output device.

[0188] Program code can be applied to input instructions to perform each function described in the present application and generate output information. Output information can be applied to one or more output devices in a known manner. For the purpose of the present application, the processing system includes any system with a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC) or a microprocessor.

[0189] Program code can be implemented with high-level programming language or object-oriented programming language to communicate with the processing system. When necessary, program code can also be implemented with assembly language or machine language. In fact, the mechanism described in this application is not limited to the scope of any specific programming language. In either case, the language can be a compiled language or an interpreted language.

[0190] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, the instructions may be distributed over a network or through other computer-readable media. Therefore, a machine-readable medium may include any mechanism for storing or transmitting information in a machine (e.g., computer) readable form, including but not limited to floppy disks, optical disks, optical disks, compact disc read-only memories (CD-ROMs), magneto-optical disks, read-only memories (ROMs), random access memories (RAMs), erasable programmable read-only memories (EPROMs), electrically erasable programmable read-only memories (EEPROMs), magnetic or optical cards, flash memory, or tangible machine-readable memories for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in electrical, optical, acoustic, or other forms of propagation signals. Accordingly, machine-readable media include any type of machine-readable media suitable for storing or transmitting electronic instructions or information in a form readable by a machine (eg, a computer).

[0191] In the accompanying drawings, some structural or method features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be required. Instead, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the accompanying drawings of the specification. In addition, the inclusion of structural or method features in a particular figure does not mean that such features are required in all embodiments, and in some embodiments, these features may not be included or may be combined with other features.

[0192] It should be noted that the units / modules mentioned in the various device embodiments of the present application are all logical units / modules. Physically, a logical unit / module can be a physical unit / module, or a part of a physical unit / module, or can be implemented as a combination of multiple physical units / modules. The physical implementation method of these logical units / modules themselves is not the most important. The combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed by the present application. In addition, in order to highlight the innovative part of the present application, the above-mentioned device embodiments of the present application do not introduce units / modules that are not closely related to solving the technical problems proposed by the present application, which does not mean that there are no other units / modules in the above-mentioned device embodiments.

[0193] It should be noted that in the examples and description of this patent, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including one" do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0194] Although the present application has been illustrated and described with reference to certain preferred embodiments thereof, it will be apparent to those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present application.

Claims

1. A training method for a face recognition model for people wearing masks, characterized in that: include: Obtaining a pre-trained full-face recognition model; the full-face recognition model includes a first feature extraction network and a classification network; A mask-wearing face recognition model is constructed based on the full face recognition model, wherein the mask-wearing face recognition model includes a second feature extraction network and the classification network generated based on the first feature extraction network; the second feature extraction network includes the first feature extraction network and a multi-layer convolutional network, the multi-layer convolutional network uses the output of the first feature extraction network as input, and the classification network uses the output of the multi-layer convolutional network as input; Acquire a current triplet to be trained from the triplet image sample set, wherein the triplet includes a first mask-wearing image and a first mask-free image of the same person, and a second mask-free image of another person; Inputting the first mask-wearing image into the mask-wearing face recognition model to obtain mask-wearing face features of the first mask-wearing face image; Inputting the first image without a mask and the second image without a mask into the full face recognition model respectively, and obtaining the facial features of the first image without a mask and the second image without a mask; Determine a first loss of the masked face recognition model based on the masked face features and the unmasked face features; Freeze the first feature extraction network within the second feature extraction network, and adjust parameters of the multi-layer convolutional network within the second feature extraction network based on the first loss to obtain a trained face recognition model for people wearing masks.

2. The method according to claim 1, characterized in that The classification network includes a global pooling network and a category determination network; The step of inputting the first mask-wearing image into the mask-wearing face recognition model to obtain mask-wearing face features of the first mask-wearing face image includes: Input the first mask-wearing image into the mask-wearing face recognition model to obtain the feature map output by the second feature extraction network and the first face feature output by the global pooling network; The step of inputting the first image without a mask and the second image without a mask into the full face recognition model respectively to obtain the facial features of the first image without a mask and the second image without a mask, comprising: Inputting the first mask-free image into the full face recognition model to obtain the second facial features output by the global pooling network; Inputting the second mask-free image into the full face recognition model to obtain the third facial features output by the global pooling network; The determining the first loss of the masked face recognition model based on the masked face features and the unmasked face features includes: The feature map is discriminated by a discriminant model to obtain a discriminant result; the discriminant model is used to discriminate whether the input feature map is a feature map of an image with a mask or a feature map of an image without a mask; Based on the discrimination result, determining the generation loss of the mask-wearing face recognition model; Determine the ternary loss of the mask-wearing face recognition model based on the first face feature, the second face feature and the third face feature; The first loss is determined based on the generation loss and the ternary loss.

3. The method according to claim 2, characterized in that The first loss is determined based on the following formula: L1=w1×L G +w2×L T , Among them, L1 represents the first loss, L G represents the generation loss, w1 represents the weight coefficient of the generation loss, L T represents the ternary loss, w2 represents the weight coefficient of the ternary loss; L G =log[1-D(G(z))], where log represents the logarithmic function, G(z) represents the feature map output by the second feature extraction network, and D(G(z)) represents the probability that the discriminant model discriminates the feature map output by the second feature extraction network as the feature map of the image without a mask; L T =max(d(a,p)-d(a,n)+margin, 0), where a represents the first facial feature, p represents the second facial feature, n represents the third facial feature, margin is a preset constant, d(a,p) represents the distance between the first facial feature and the second facial feature, d(a,n) represents the distance between the first facial feature and the third facial feature, and max(d(a,p)-d(a,n)+margin, 0) represents the maximum value between d(a,p)-d(a,n)+margin and 0.

4. The method according to claim 1, characterized in that: The multi-layer convolutional network is a three-layer convolutional network or a five-layer convolutional network.

5. A method for recognizing a face wearing a mask, characterized in that: include: Obtain the face image of the person wearing a mask to be identified; Inputting the mask-wearing face image to be identified into a pre-trained mask-wearing face recognition model to obtain a face recognition result of the mask-wearing face image; The mask-wearing face recognition model is trained based on the method described in any one of claims 1 to 4.

6. An electronic device, characterized in that: The device comprises a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the electronic device is triggered to execute the method according to any one of claims 1 to 4 or 5.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 4 or 5.

8. A computer program product, characterized in that The computer program product comprises executable instructions, and when the executable instructions are executed on a computer, the computer is caused to perform the method according to any one of claims 1 to 4 or 5.

Images