Face recognition model training method and device based on contrast learning, and face recognition method and device
By employing a comparative learning-based face recognition model training method, and utilizing feature decoupling algorithms and deconvolutional layers to filter out the inherent synthetic modalities of edited faces, this approach solves the problem of recognizing edited faces in existing technologies, thereby improving the robustness and accuracy of face recognition.
Patent Information
- Application Number
- CN202310175022.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-02-28
- Publication Date
- 2026-02-10
- Estimated Expiration
- 2043-02-28
AI Technical Summary
Existing face recognition systems struggle to effectively identify edited faces, especially synthetic faces generated by generative adversarial networks, leading to decreased recognition accuracy and an inability to effectively filter out the inherent synthetic modalities of edited faces.
A face recognition model training method based on contrastive learning is adopted. The face image and the generated image are converted into feature maps through the backbone extraction network. The entangled features and feature vectors are extracted using a fully connected layer. Combined with the feature decoupling algorithm and the deconvolution layer, the cosine similarity is calculated and non-face identification information is filtered out to generate face identification information.
It effectively filters out the inherent synthetic modality of edited faces, improves the robustness and accuracy of face recognition, and can identify synthetic faces generated by adversarial networks.
Smart Images
Figure CN116311435B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence, and in particular to a face recognition model training method, recognition method and device based on contrastive learning. Background Technology
[0002] With the rapid development of artificial intelligence, various deep learning-based applications have emerged and permeated all aspects of our lives, with facial recognition being one of the most common. Facial recognition has been applied in digital cameras, access control systems, identity recognition (including authentication), internet applications (including online payments and transaction authorization), and entertainment applications, among others. The unique nature of facial data has also raised privacy concerns. More and more people are unwilling to disclose their private facial data, which has created significant obstacles to the collection of facial recognition datasets. The accuracy of deep learning relies heavily on the amount of data; a small amount of sample data can significantly impact the model's accuracy. Increasingly, facial recognition models are being trained using data augmentation methods, including edited or augmented faces. However, experiments have shown that edited faces, due to their inherent characteristics, can negatively affect the performance of facial recognition. The widespread application of facial recognition has also raised security concerns. It's not just adversarial attacks that degrade the performance of facial recognition systems; increasingly, applications based on Generative Adversarial Networks (GANs) for face editing, such as FaceApp and FaceLab, generate various synthetic faces that can disable facial recognition systems. For example, when uploading a face image for facial recognition, if someone wants to use a beautified photo or an edited face to conceal their identity, they can use facial editing software to manipulate their image, making them appear older, younger, or even changing their gender. Similarly, in the physical world, disguises can be used to make the collected facial information mismatched with one's true identity.
[0003] Existing facial recognition systems often cannot effectively recognize faces that have been edited or that differ significantly from those in the system, and they cannot filter out the inherent synthetic modalities of edited faces. Summary of the Invention
[0004] In view of this, embodiments of this application provide a face recognition model training method, recognition method, and apparatus based on contrastive learning, so as to eliminate or improve one or more defects existing in the prior art.
[0005] The first aspect of this application provides a method for training a face recognition model based on contrastive learning, the method comprising:
[0006] Edit a facial image to generate a facial image;
[0007] The face image and the generated face image are converted into a face feature map and a generated feature map, respectively, based on the backbone extraction network;
[0008] The first fully connected layer is used to extract the face entanglement features of the face feature map and the generation entanglement features of the generated feature map;
[0009] The face feature vector of the face entanglement feature and the generated feature vector of the generated entanglement feature are obtained according to the projector;
[0010] The cosine similarity is calculated based on the face feature vector and the generated feature vector. Then, a feature decoupling algorithm is used to extract features from the face feature map and the generated feature map to obtain the face identification information of the face feature map and the face identification information of the generated feature map, thereby generating a face recognition model for extracting the face image identification information of a person.
[0011] In some embodiments of this application, the step of using a feature decoupling algorithm to extract features from the face feature map and the generated feature map to obtain the face identification information of the face feature map and the face identification information of the generated feature map respectively includes:
[0012] The non-face identifier information of the face feature map and the non-generated face identifier information of the generated feature map are extracted using a second fully connected layer;
[0013] The face feature map is obtained by subtracting the face entanglement features from the face feature map and the non-face identification information of the face feature map. The face identification information of the generated feature map is obtained by subtracting the generated entanglement features from the generated feature map and the non-generated face identification information of the generated feature map.
[0014] In some embodiments of this application, after extracting the face entanglement features of the face feature map and the generation entanglement features of the generated feature map using the first fully connected layer, the method further includes:
[0015] A deconvolutional layer is used to recover the entangled facial features of the face feature map to obtain a feature recovery map. A recovery loss is calculated based on the face feature map and the feature recovery map. Additionally, the deconvolutional layer is used to recover the entangled features of the generated feature map to obtain a generated feature recovery map. A generation recovery loss is calculated based on the generated feature map and the generated feature recovery map.
[0016] In some embodiments of this application, after subtracting the face entanglement features of the face feature map from the non-face identification information of the face feature map to obtain the face identification information of the face feature map, and after subtracting the generation entanglement features of the generated feature map from the non-generation face identification information of the generated feature map to obtain the face identification information of the generated feature map, the method further includes:
[0017] Based on the face recognition model, loss calculations are performed on the face identifier information of the face feature map and the face identifier information of the generated feature map to obtain classification loss and generation classification loss respectively.
[0018] In some embodiments of this application, it also includes:
[0019] The contrast loss is obtained by performing loss calculations on the face entanglement features, the generated entanglement features, the face feature vector, and the generated feature vector.
[0020] In some embodiments of this application, it also includes:
[0021] The overall loss is calculated by the classification loss, the recovery loss, and the contrast loss according to preset parameters, and the overall generation loss is calculated by the generation classification loss, the generation recovery loss, and the contrast loss according to preset parameters.
[0022] A second aspect of this application provides a face recognition method based on contrastive learning, the method comprising:
[0023] A face image is input into a preset face recognition model, which outputs the face identification information of the face image and the face identification information of the generated face image. The face recognition model is trained in advance using contrastive learning and feature decoupling, and the generated face image is generated by the preset face recognition model by editing the face image.
[0024] A third aspect of this application provides a face recognition model training device based on contrastive learning, the device comprising:
[0025] The image editing module is used to edit human face images into human face-generated images;
[0026] The feature extraction module is used to convert the face image and the generated face image into a face feature map and a generated feature map, respectively, based on the backbone extraction network;
[0027] The entanglement feature extraction module is used to extract the face entanglement features of the face feature map and the generation entanglement features of the generated feature map using a first fully connected layer;
[0028] The feature projection module is used to obtain the face feature vector of the face entanglement feature and the generated feature vector of the generated entanglement feature based on the projector;
[0029] The similarity verification module is used to calculate the cosine similarity based on the face feature vector and the generated feature vector, and to use a feature decoupling algorithm to extract features from the face feature map and the generated feature map respectively to obtain the face identification information of the face feature map and the face identification information of the generated feature map, thereby generating a face recognition model for extracting the face image identification information of a person.
[0030] A fourth aspect of this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the face recognition model training method based on contrastive learning described in the first aspect, or executes the face recognition method based on contrastive learning described in the second aspect.
[0031] The fifth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the face recognition model training method based on contrastive learning as described in the first aspect above, or performs the face recognition method based on contrastive learning as described in the second aspect above.
[0032] This application provides a face recognition model training method, recognition method, and apparatus based on contrastive learning. The method includes: editing a face image into a generated face image; converting the face image and the generated face image into a face feature map and a generated feature map, respectively, using a backbone extraction network; extracting face entanglement features from the face feature map and the generated entanglement features from the generated feature map using a first fully connected layer; obtaining face feature vectors and generated feature vectors from the face entanglement features and the generated entanglement features using a projector; calculating cosine similarity based on the face feature vectors and the generated feature vectors; and using a feature decoupling algorithm to extract features from the face feature map and the generated feature map, respectively, to obtain face identification information from the face feature map and the generated feature map, thereby generating a face recognition model for extracting face image identification information. This application can effectively filter out the inherent synthetic modality of the edited face.
[0033] Additional advantages, objectives, and features of this application will be set forth in part in the description which follows, and will in part become apparent to those skilled in the art upon review of the following description, or may be learned by practice of the application. The objectives and other advantages of this application can be realized and obtained by means of the structures specifically pointed out in the specification and drawings.
[0034] Those skilled in the art will understand that the purposes and advantages that can be achieved with this application are not limited to those specifically described above, and that the above and other purposes that this application can achieve will be more clearly understood from the following detailed description. Attached Figure Description
[0035] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, do not constitute a limitation thereof. The components in the drawings are not drawn to scale but are merely for illustrating the principles of this application. For ease of illustration and description of certain parts of this application, corresponding portions in the drawings may be enlarged, i.e., may appear larger relative to other components in an exemplary device actually manufactured according to this application. In the drawings:
[0036] Figure 1 This is a flowchart illustrating a face recognition model training method based on contrastive learning in one embodiment of this application.
[0037] Figure 2 This is a schematic diagram of the structure of a face recognition model training device based on contrastive learning in another embodiment of this application.
[0038] Figure 3 This is a schematic diagram of the overall architecture for training the model in this application. Detailed Implementation
[0039] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and their descriptions are used to explain this application, but are not intended to limit it.
[0040] It should also be noted that, in order to avoid obscuring this application with unnecessary details, only the structures and / or processing steps closely related to the scheme according to this application are shown in the accompanying drawings, while other details that are not closely related to this application are omitted.
[0041] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.
[0042] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.
[0043] In the following description, embodiments of the present application will be illustrated with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.
[0044] The existing ArcFace face recognition normalizes the feature vectors extracted by Babbone and proposes an additive angular distance loss, which directly maximizes the classification boundary in the angular space, improving inter-class classability, reducing intra-class distance, and increasing inter-class distance. However, the ArcFace series of studies only improves inter-class classability by agreeing on a loss function, without improving the robustness of the face recognition model through the model's own feature processing capabilities.
[0045] Feature deconstruction-based face recognition is an improvement on ArcFace face recognition, with the goal of distinguishing face ID information from other non-ID information (including pose, age, etc.) and using only the ID information for face recognition. Feature deconstruction-based face recognition employs an encoder and decoder architecture. The backbone model acts as the encoder, and the deconvolutional layer as the decoder. The encoder-decoder structure does not aim to recover the original image, but rather a feature map of appropriate depth that contains sufficient information. The specific loss is the sum of the classification loss and the MSE loss in a suitable ratio. Existing face recognition models ignore the possibility of edited face images during data collection and recognition. These edited images often possess inherent modalities, such as pupil shape. Features with these inherent modalities tend to cluster together, which can be detrimental to the training of the face recognition model.
[0046] The following examples will provide a detailed description.
[0047] This application provides a method for training a face recognition model based on contrastive learning, which can be executed by a face recognition model training device based on contrastive learning. See [link to relevant documentation]. Figure 1 The face recognition model training method based on contrastive learning specifically includes the following:
[0048] Step 110: Edit the face image to generate a face image.
[0049] Step 120: Convert the face image and the generated face image into face feature maps and generated feature maps respectively according to the backbone extraction network.
[0050] Step 130: Use the first fully connected layer to extract the face entanglement features of the face feature map and the generation entanglement features of the generated feature map.
[0051] Step 140: Obtain the face feature vector of the face entanglement feature and the generated feature vector of the generated entanglement feature based on the projector.
[0052] Step 150: Calculate the cosine similarity based on the face feature vector and the generated feature vector, and use a feature decoupling algorithm to extract features from the face feature map and the generated feature map respectively to obtain the face identification information of the face feature map and the face identification information of the generated feature map, thereby generating a face recognition model for extracting the face image identification information of a person.
[0053] Specifically, the server can use generative adversarial networks such as StarGAN to edit face images in the training set into generated face images. Then, based on the backbone extraction network, the face images and generated face images are converted into face feature maps and generated feature maps, respectively. The first fully connected layer is used to extract the face entanglement features of the face feature map and the generated entanglement features of the generated feature map. The projector obtains the face feature vector of the face entanglement features and the generated feature vector of the generated entanglement features. Finally, the cosine similarity is calculated based on the face feature vector and the generated feature vector, and the feature decoupling algorithm is used to extract features from the face feature map and the generated feature map, respectively, to obtain the face identification information of the face feature map and the face identification information of the generated feature map. This generates a face recognition model for extracting the face image identification information of a person, which can effectively filter the inherent synthetic modal information of the edited face, such as the shape of the pupil.
[0054] To further filter out non-face identifier information in the face image and the generated face image, step 150 employs a feature decoupling algorithm to extract features from the face feature map and the generated feature map respectively, obtaining the face identifier information of the face feature map and the face identifier information of the generated feature map, including:
[0055] Step 010: Use the second fully connected layer to extract the non-face identifier information of the face feature map and the non-generated face identifier information of the generated feature map.
[0056] Step 020: Subtract the face entanglement features of the face feature map from the non-face identification information of the face feature map to obtain the face identification information of the face feature map; and subtract the generation entanglement features of the generated feature map from the non-generation face identification information of the generated feature map to obtain the face identification information of the generated feature map.
[0057] Specifically, the server uses a second fully connected layer to extract non-face identification information from the face feature map and non-generated face identification information from the generated feature map; it subtracts the face entanglement features of the face feature map from the non-face identification information of the face feature map to obtain the face identification information of the face feature map; and it subtracts the generation entanglement features of the generated feature map from the non-generated face identification information of the generated feature map to obtain the face identification information of the generated feature map, thereby effectively filtering out non-face identification information from the face image and the face generated image.
[0058] In step 130, the first fully connected layer and the second fully connected layer refer to the fact that each node in the convolutional layer is connected to all nodes in the next layer. This is constructed by imitating the neural structure of the human brain. The non-face identification information includes hair, the pose and expression of the photo, etc., and is specifically represented as a feature vector.
[0059] To further improve the effectiveness of filtering out the inherent synthetic modality information of the edited face, after step 130, the following is also included:
[0060] Step 131: Use a deconvolution layer to recover the entangled features of the face feature map to obtain a feature recovery map, calculate the recovery loss based on the face feature map and the feature recovery map, and use the deconvolution layer to recover the entangled features of the generated feature map to obtain a generated feature recovery map, calculate the generation recovery loss based on the generated feature map and the generated feature recovery map.
[0061] Specifically, the server uses deconvolution layers to recover the entangled features of the face feature map to obtain a feature recovery map. The recovery loss is calculated based on the face feature map and the feature recovery map. Additionally, deconvolution layers are used to recover the entangled features of the generated feature map to obtain a generated feature recovery map. The generation recovery loss is calculated based on the generated feature map and the generated feature recovery map. This further improves the accuracy of filtering out the inherent synthetic modality information of the edited face and enhances the robustness of the face recognition model.
[0062] Among them, recovery loss L 恢复 As shown in equation (1), m1 is the feature map of the face image after being processed by the backbone extraction network, and m2 is the vector f containing entanglement information. entangle (Entangled features) The feature map is recovered by deconvolution, and L is the mean square error loss function (i.e., the average squared error, which is the square of the difference between the predicted value and the true value. The recovery loss is the same as in formula (1).
[0063] L 恢复 =L(m1,m2) Equation (1)
[0064] To further improve the effect of filtering out the inherent synthetic modality information of the edited face, after step 020, the following is also included:
[0065] Step 021: Based on the face recognition model, calculate the loss for the face identifier information in the face feature map and the face identifier information in the generated feature map to obtain the classification loss and the generation classification loss.
[0066] Specifically, the server calculates the classification loss and generation classification loss by performing loss calculations on the face identification information of the face feature map and the face identification information of the generated feature map respectively based on the face recognition model, thereby further improving the accuracy of filtering out the inherent synthetic modality information of the edited face.
[0067] Among them, the classification loss L 分类 The ID loss is as follows: s is set to 32, and m is the margin penalty term with m set to 0.5. Let x be the angle between the input feature and the normalized weight w, and N be the number of face categories. Generate the classification loss using the same formula (2).
[0068]
[0069] To further improve the effectiveness of filtering out the inherent synthetic modality information of the edited face, after step 150, the following is also included:
[0070] Step 160: Perform loss calculation on the face entanglement features, the generated entanglement features, the face feature vector, and the generated feature vector to obtain the contrast loss.
[0071] Specifically, the server calculates the contrast loss by performing loss calculations on the face entanglement features, generated entanglement features, face feature vectors, and generated feature vectors, thereby further improving the accuracy of filtering out the inherent synthetic modality information of the edited face.
[0072] Among them, the contrast loss L 对比As shown in equation (3), z1 and z2 are the feature vectors obtained by the backbone extraction network and the first fully connected layer, i.e., the encoder, from the face image. p1 and p2 are the feature vectors obtained by z1 and z2 through the projector (both the encoder and the projector are fully connected layers with the same input and output vector dimensions, which is a concept in contrastive learning. Two different input vectors can be extracted through two fully connected layers. The encoder is the first step of the extraction process, and the projector is the second step. The projector is set to be discarded. There are too many extraction steps in the process of obtaining the final features, but it is just right when calculating the similarity between the two. This has also been proven in a series of papers on contrastive learning. Finally, the second step of feature extraction is discarded. The first step of feature extraction has already filtered out some inherent information of the synthetic face, so there is no need for the second step of feature extraction). D is the cosine similarity. The larger the cosine similarity, the greater the loss.
[0073]
[0074] To improve the overall effectiveness of filtering out the inherent synthetic modality information of the edited face through the loss function, the following is added after any one of steps 131, 021, and 160:
[0075] Step 170: Calculate the overall loss by performing the classification loss, the recovery loss, and the contrast loss according to preset parameters; and calculate the overall generation loss by performing the generation classification loss, the generation recovery loss, and the contrast loss according to preset parameters.
[0076] Specifically, the server calculates the overall loss by using preset parameters to calculate the classification loss, recovery loss, and contrast loss, and calculates the overall generation loss by using preset parameters to calculate the generation classification loss, generation recovery loss, and contrast loss. This improves the accuracy of filtering out the inherent synthetic modality information of the edited face from the perspective of overall loss.
[0077] The overall loss L is the sum of the three losses, as shown in equation (4). α1 and α2 are proportionality coefficients that control the contribution of different losses to the total loss. In this application, α1 is set to 1 and α2 is set to -1. Because the larger the contrast loss, the higher the cosine similarity, we hope to increase the cosine similarity, which means reducing the negative value of the contrast loss.
[0078] L = L 分类 +α1L 恢复 +α2L 对比 Equation (4)
[0079] This application also provides a face recognition method based on contrastive learning, executed by a client device on the application side, the method comprising:
[0080] A face image is input into a preset face recognition model, which outputs the face identification information of the face image and the face identification information of the generated face image. The face recognition model is trained in advance using contrastive learning and feature decoupling, and the generated face image is generated by the preset face recognition model by editing the face image.
[0081] Specifically, the client device inputs a face image into a preset face recognition model, so that the face recognition model outputs the face identification information of the face image and the face identification information of the generated face image, thereby effectively filtering the inherent synthetic modality information of the edited face.
[0082] The face recognition model is trained in advance using contrastive learning and feature decoupling, while the generated face image is produced by editing the face image using the preset face recognition model.
[0083] From a software perspective, this application also provides a face recognition model training apparatus for performing all or part of the contrastive learning-based face recognition model training method described above. See [link to relevant documentation]. Figure 2 The face recognition model training device based on contrastive learning specifically includes the following components:
[0084] Image editing module 10 is used to edit a face image into a face generation image.
[0085] The feature extraction module 20 is used to convert the face image and the generated face image into a face feature map and a generated feature map, respectively, according to the backbone extraction network.
[0086] Entangled feature extraction module 30 is used to extract the face entanglement features of the face feature map and the generation entanglement features of the generated feature map using a first fully connected layer.
[0087] The feature projection module 40 is used to obtain the face feature vector of the face entanglement feature and the generated feature vector of the generated entanglement feature according to the projector.
[0088] The similarity verification module 50 is used to calculate the cosine similarity based on the face feature vector and the generated feature vector, and to use a feature decoupling algorithm to extract features from the face feature map and the generated feature map respectively to obtain the face identification information of the face feature map and the face identification information of the generated feature map, thereby generating a face recognition model for extracting the face image identification information of a person.
[0089] The embodiments of the face recognition model training device based on contrastive learning provided in this application can be used to execute the processing flow of the face recognition model training method based on contrastive learning in the above embodiments. Its functions will not be repeated here, but can be referred to the detailed description of the face recognition model training method based on contrastive learning in the above embodiments.
[0090] For the overall architecture of the model training process in this application, please refer to [link / reference]. Figure 3 The entire face recognition model is divided into two branches: the original image branch and the edited face branch, which are based on the contrastive learning algorithm (SimSiam). The original image branch takes the original face image as input, while the edited face branch takes the image after StarGAN face editing as input. In the StarGAN face editing method, the generator consists of a series of convolutional layers. It takes the original image information and target domain information as input and outputs the edited image, i.e., the face generated image. Both are vectors (the image is essentially a vector with length * width * number of channels).
[0091] Both branches pass through a shared-weight backbone extraction network to generate a 512*7*7 dimensional feature map, i.e., a face feature map and a generated feature map (calculated as y1 = w1 * x1, where x1 is the input image (112*112*3), y1 is the feature map * (512*7*7), and w1 is the learnable parameter vector of the fully connected layer). The resulting feature map is then connected to two fully connected layers. The feature map passes through the first fully connected layer (a fully connected layer is one where each node connects to all nodes in the next layer; it mimics the neural structure of the human brain, and its core implementation uses matrix multiplication: y = wx, where x is the input vector, y is the output vector, and w...). The feature matrix of the fully connected layer (i.e., the parameters that need to be learned) yields 512-dimensional entangled information, namely face entangled features and generated entangled features (entangled information is essentially a vector containing a person's ID information and non-ID information, but the two are coupled together without clear distinction. This application aims to distinguish these two types of information by using a large amount of data to learn the parameters of the fully connected layer through the connection of the fully connected layer). Then, a 512-dimensional feature vector is generated through the projector, namely the face feature vector and the generated feature vector (calculated as z1 = w2 * y1, where y1 is the feature map (512 * 7 * 7), z1 is the feature vector (512), and w2... (These are the learnable parameter vectors of the fully connected layer). The model aims to maximize the cosine similarity between feature vectors (cosine similarity is calculated by multiplying the two vectors and dividing by their modulus). A high cosine similarity value indicates that the two feature vectors are more similar. Subsequently, the projector is discarded, and the encoder then filters the intrinsic modalities of the synthesized face. Simultaneously, entanglement information can be recovered through the deconvolution layer to obtain 512*7*7 entanglement information. Here, only the feature map is recovered (i.e., the feature recovery map and the generated feature recovery map), not the original image, because recovering the original image is costly and has poor recovery results. This is detrimental to model training. The second fully connected layer obtains non-ID information (essentially a vector containing information unrelated to a person's identity features, such as hair, pose, and expression, but represented as a vector). Subtracting the entangled information from the non-ID information (i.e., non-face identifier information) and the non-generated face identifier information yields the ID information, i.e., the face identifier information (also a vector, strongly correlated with the person's identity, representing the most essential feature of the face; this information allows us to identify the person). This effectively filters out some identity-irrelevant information, resulting in a further refined and processed face representation.
[0092] Finally, this invention uses the ID information to obtain the final face category output through the Arcface face margin model. The Arcface face margin improves the original Softmax loss by adding an additive penalty term to the original loss function, which increases the training difficulty of faces with the same identity. This can reduce intra-class distance and increase inter-class distance, further enhancing the robustness of the face model.
[0093] This application provides a face recognition model training method, recognition method, and apparatus based on contrastive learning. The method includes: editing a face image into a generated face image; converting the face image and the generated face image into a face feature map and a generated feature map, respectively, using a backbone extraction network; extracting face entanglement features from the face feature map and the generated entanglement features from the generated feature map using a first fully connected layer; obtaining face feature vectors and generated feature vectors from the face entanglement features and the generated entanglement features using a projector; calculating cosine similarity based on the face feature vectors and the generated feature vectors; and using a feature decoupling algorithm to extract features from the face feature map and the generated feature map, respectively, to obtain face identification information from the face feature map and the generated feature map, thereby generating a face recognition model for extracting face image identification information. This application can effectively filter out the inherent synthetic modality of the edited face.
[0094] This application also provides an electronic device, such as a central server, which may include a processor, a memory, a receiver, and a transmitter. The processor is used to execute the face recognition model training method based on contrastive learning or the face recognition method based on contrastive learning mentioned in the above embodiments. The processor and memory can be connected via a bus or other means, taking a bus connection as an example. The receiver can be connected to the processor and memory via wired or wireless means.
[0095] The processor can be a central processing unit (CPU). The processor can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations of the above types of chips.
[0096] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the contrastive learning-based face recognition model training method or the contrastive learning-based face recognition method in the embodiments of this application. The processor executes various functional applications and data processing by running the non-transitory software programs, instructions, and modules stored in the memory, thereby implementing the contrastive learning-based face recognition model training method or the contrastive learning-based face recognition method in the above method embodiments.
[0097] The memory may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the processor, etc. Furthermore, the memory may include high-speed random access memory and non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, which can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
[0098] The one or more modules are stored in the memory, and when executed by the processor, they execute the face recognition model training method based on contrastive learning or the face recognition method based on contrastive learning in the embodiment.
[0099] In some embodiments of this application, the user equipment may include a processor, a memory, and a transceiver unit. The transceiver unit may include a receiver and a transmitter. The processor, memory, receiver, and transmitter may be connected via a bus system. The memory is used to store computer instructions, and the processor is used to execute the computer instructions stored in the memory to control the transceiver unit to send and receive signals.
[0100] As one implementation method, the functions of the receiver and transmitter in this application can be implemented by transceiver circuits or dedicated transceiver chips, and the processor can be implemented by dedicated processing chips, processing circuits or general-purpose chips.
[0101] As another implementation approach, the server provided in this application embodiment can be implemented using a general-purpose computer. That is, the program code implementing the processor, receiver, and transmitter functions is stored in memory, and the general-purpose processor implements the processor, receiver, and transmitter functions by executing the code in memory.
[0102] This application also provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the steps of the aforementioned face recognition model training method based on contrastive learning or face recognition method based on contrastive learning. The computer-readable storage medium can be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, floppy disk, hard disk, removable storage disk, CD-ROM, or any other form of storage medium known in the art.
[0103] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave.
[0104] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.
[0105] In this application, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.
[0106] The above description is merely a preferred embodiment of this application and is not intended to limit this application. For those skilled in the art, various modifications and variations can be made to the embodiments of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.
Claims
1. A face recognition model training method based on contrastive learning, characterized in that, include: Edit a facial image to generate a facial image; The face image and the generated face image are converted into a face feature map and a generated feature map, respectively, based on the backbone extraction network; The first fully connected layer is used to extract the face entanglement features of the face feature map and the generation entanglement features of the generated feature map; The face feature vector of the face entanglement feature and the generated feature vector of the generated entanglement feature are obtained according to the projector; The cosine similarity is calculated based on the face feature vector and the generated feature vector. Then, a feature decoupling algorithm is used to extract features from the face feature map and the generated feature map to obtain the face identification information of the face feature map and the face identification information of the generated feature map, thereby generating a face recognition model for extracting face image identification information of a person. The step involves using a feature decoupling algorithm to extract features from the face feature map and the generated feature map, respectively, to obtain the face identification information of the face feature map and the face identification information of the generated feature map, including: The non-face identifier information of the face feature map and the non-generated face identifier information of the generated feature map are extracted using a second fully connected layer; The face entanglement features of the face feature map are subtracted from the non-face identification information of the face feature map to obtain the face identification information of the face feature map; and the generation entanglement features of the generated feature map are subtracted from the non-generation face identification information of the generated feature map to obtain the face identification information of the generated feature map. After extracting the face entanglement features of the face feature map and the generation entanglement features of the generated feature map using the first fully connected layer, the method further includes: A deconvolutional layer is used to recover the entangled facial features of the face feature map to obtain a feature recovery map. A recovery loss is calculated based on the face feature map and the feature recovery map. Additionally, the deconvolutional layer is used to recover the entangled features of the generated feature map to obtain a generated feature recovery map. A generation recovery loss is calculated based on the generated feature map and the generated feature recovery map.
2. The face recognition model training method based on contrastive learning according to claim 1, characterized in that, After subtracting the face entanglement features of the face feature map from the non-face identification information of the face feature map to obtain the face identification information of the face feature map, and subtracting the generation entanglement features of the generated feature map from the non-generation face identification information of the generated feature map to obtain the face identification information of the generated feature map, the method further includes: Based on the face recognition model, loss calculations are performed on the face identifier information of the face feature map and the face identifier information of the generated feature map to obtain classification loss and generation classification loss respectively.
3. The face recognition model training method based on contrastive learning according to claim 2, characterized in that, Also includes: The contrast loss is obtained by performing loss calculations on the face entanglement features, the generated entanglement features, the face feature vector, and the generated feature vector.
4. The face recognition model training method based on contrastive learning according to claim 3, characterized in that, Also includes: The overall loss is calculated by the classification loss, the recovery loss, and the contrast loss according to preset parameters, and the overall generation loss is calculated by the generation classification loss, the generation recovery loss, and the contrast loss according to preset parameters.
5. A face recognition method based on contrastive learning, characterized in that, include: A face image is input into a preset face recognition model, so that the face recognition model outputs the face identification information of the face image and the face identification information of the generated face image. The face recognition model is trained in advance using the face recognition model training method based on contrastive learning as described in any one of claims 1 to 4. The generated face image is generated by the preset face recognition model by editing the face image.
6. A face recognition model training device based on contrastive learning, characterized in that, include: The image editing module is used to edit human face images into human face-generated images; The feature extraction module is used to convert the face image and the generated face image into a face feature map and a generated feature map, respectively, based on the backbone extraction network; The entanglement feature extraction module is used to extract the face entanglement features of the face feature map and the generation entanglement features of the generated feature map using a first fully connected layer; The feature projection module is used to obtain the face feature vector of the face entanglement feature and the generated feature vector of the generated entanglement feature based on the projector; The similarity verification module is used to calculate the cosine similarity based on the face feature vector and the generated feature vector, and to use a feature decoupling algorithm to extract features from the face feature map and the generated feature map respectively to obtain the face identification information of the face feature map and the face identification information of the generated feature map, thereby generating a face recognition model for extracting the face image identification information of a person; The step involves using a feature decoupling algorithm to extract features from the face feature map and the generated feature map, respectively, to obtain the face identification information of the face feature map and the face identification information of the generated feature map, including: The non-face identifier information of the face feature map and the non-generated face identifier information of the generated feature map are extracted using a second fully connected layer; The face entanglement features of the face feature map are subtracted from the non-face identification information of the face feature map to obtain the face identification information of the face feature map; and the generation entanglement features of the generated feature map are subtracted from the non-generation face identification information of the generated feature map to obtain the face identification information of the generated feature map. After extracting the face entanglement features of the face feature map and the generation entanglement features of the generated feature map using the first fully connected layer, the face recognition model training device based on contrastive learning also performs the following: A deconvolutional layer is used to recover the entangled facial features of the face feature map to obtain a feature recovery map. A recovery loss is calculated based on the face feature map and the feature recovery map. Additionally, the deconvolutional layer is used to recover the entangled features of the generated feature map to obtain a generated feature recovery map. A generation recovery loss is calculated based on the generated feature map and the generated feature recovery map.
7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the face recognition model training method based on contrastive learning as described in any one of claims 1 to 4, or it implements the face recognition method based on contrastive learning as described in claim 5.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the face recognition model training method based on contrastive learning as described in any one of claims 1 to 4, or performs the face recognition method based on contrastive learning as described in claim 5.