Training method, device and computer program product for face recognition model
Through unsupervised feature clustering and machine learning, the face recognition model is updated, and identity and non-identity information is decoupled, and the problem of face recognition accuracy bottleneck in the existing technology is solved, and the accuracy of complex light and large-angle face recognition is improved. .
Patent Information
- Application Number
- CN202110932551.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-08-13
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2041-08-13
AI Technical Summary
In complex situations such as complex light, large-angle faces, face occlusion, and blurred picture, it is difficult for existing deep learning models to effectively decouple the identity information and non-identity information of face objects, resulting in a reduction in facial recognition accuracy.
Unsupervised feature clustering method is adopted to extract non-identity tags based on shallow features, update the initial face recognition model through machine learning, decouple identity and non-identity information, and improve recognition accuracy.
It improves the recognition accuracy of the face recognition model, saves resource consumption for obtaining tags, and enhances the determination efficiency of non-identity tags.
Smart Images

Figure CN113657248B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of artificial intelligence, specifically to computer vision and deep learning technology, and in particular to training methods, devices, electronic devices, storage media and computer program products for face recognition models, which can be used in face recognition scenarios. Background Art
[0002] In recent years, with the development of deep learning technology, the accuracy of face recognition based on deep learning technology has been greatly improved. Face recognition has important applications in many scenarios, such as ID card matching in transfer scenarios such as airports and train stations, access control face scanning in private domain management scenarios, and real-person verification in financial scenarios. In these rich landing scenarios, there are complex situations such as complex light, large-angle faces, face occlusion, and blurred images. The face features extracted by the deep learning model couple identity information and non-identity information, which is a bottleneck for further improving the accuracy of face recognition. Summary of the invention
[0003] The present disclosure provides a training method, device, electronic device, storage medium and computer program product for a face recognition model.
[0004] According to a first aspect, a method for training a face recognition model is provided, comprising: obtaining a training sample set, wherein the training samples in the training sample set include sample face images and identity labels corresponding to identity information of face objects in the sample face images and non-identity labels corresponding to non-identity information, and the non-identity labels of the training samples are obtained based on shallow feature clustering of the sample face images in the training sample set; using a machine learning method, taking the sample face images as input, extracting identity features and non-identity features of the input sample face images through an initial face recognition model, and using the identity labels and non-identity labels corresponding to the input sample face images as expected outputs of a first output result obtained according to the extracted identity features and a second output result obtained according to the extracted non-identity features, respectively, to update the initial image recognition model, and train the face recognition model.
[0005] According to the second aspect, a face recognition method is provided, comprising: obtaining an image to be recognized; recognizing the image to be recognized by a pre-trained face recognition model to obtain a face recognition result, wherein the face recognition model is trained by any implementation method of the first aspect.
[0006] According to a third aspect, a training device for a face recognition model is provided, comprising: a first acquisition unit, configured to acquire a training sample set, wherein the training samples in the training sample set include a sample face image and an identity label corresponding to the identity information of the face object in the sample face image and a non-identity label corresponding to the non-identity information, and the non-identity label of the training sample is obtained based on shallow feature clustering of the sample face images in the training sample set; a training unit, configured to use a machine learning method, with the sample face image as input, to extract the identity features and non-identity features of the input sample face image through an initial face recognition model, and to use the identity label and non-identity label corresponding to the input sample face image as the expected output of the first output result obtained according to the extracted identity features and the second output result obtained according to the extracted non-identity features, respectively, to update the initial image recognition model, and to train the face recognition model.
[0007] According to a fourth aspect, a face recognition device is provided, comprising: a second acquisition unit, configured to acquire an image to be recognized; and a recognition unit, configured to recognize the image to be recognized through a pre-trained face recognition model to obtain a face recognition result, wherein the face recognition model is trained through any implementation method of the first aspect.
[0008] According to the fifth aspect, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in any implementation of the first aspect or the second aspect.
[0009] According to a sixth aspect, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause a computer to execute the method described in any implementation of the first aspect or the second aspect.
[0010] According to a seventh aspect, a computer program product is provided, comprising: a computer program, which implements the method described in any implementation of the first aspect or the second aspect when executed by a processor.
[0011] According to the technology disclosed in the present invention, an unsupervised method is adopted to obtain non-identity labels of training samples in a training sample set based on feature clustering, thereby improving the efficiency of determining non-identity labels and saving users' resource consumption in obtaining labels; in the training process of the face recognition model, the identity information and non-identity information of the face objects in the sample face images are decoupled, thereby improving the recognition accuracy of the trained face recognition model.
[0012] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.
[0014] Figure 1 is an exemplary system architecture diagram in which an embodiment of the present disclosure may be applied;
[0015] Figure 2 is a flowchart of an embodiment of a method for training a face recognition model according to the present disclosure;
[0016] Figure 3 is a schematic diagram of an application scenario of the training method for a face recognition model according to this embodiment;
[0017] Figure 4 is a flowchart of another embodiment of a method for training a face recognition model according to the present disclosure;
[0018] Figure 5 is a structural schematic diagram of an embodiment of a face recognition method according to the present disclosure;
[0019] Figure 6 is a structural diagram of an embodiment of a training device for a face recognition model according to the present disclosure;
[0020] Figure 7 is a structural diagram of an embodiment of a face recognition device according to the present disclosure;
[0021] Figure 8 It is a schematic diagram of the structure of a computer system suitable for implementing the embodiments of the present disclosure. DETAILED DESCRIPTION
[0022] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.
[0023] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0024] Figure 1An exemplary architecture 100 is shown to which the face recognition model training method and apparatus, and the face recognition method and apparatus of the present disclosure can be applied.
[0025] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The communication connection between the terminal devices 101, 102, 103 constitutes a topological network, and the network 104 is used to provide a medium for the communication link between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.
[0026] Terminal devices 101, 102, 103 may be hardware devices or software that support network connection for data interaction and data processing. When terminal devices 101, 102, 103 are hardware, they may be various electronic devices that support network connection, information acquisition, interaction, display, processing and other functions, including but not limited to monitoring equipment, smart phones, tablet computers, e-book readers, laptop portable computers and desktop computers, etc. When terminal devices 101, 102, 103 are software, they may be installed in the electronic devices listed above. They may be implemented as multiple software or software modules for providing distributed services, for example, or they may be implemented as a single software or software module. No specific limitation is made here.
[0027] Server 105 can be a server that provides various services, such as a background server that receives training requests sent by users through terminal devices 101, 102, and 103, and trains a face recognition model using a machine learning method. Before training, an unsupervised method is used to obtain non-identity labels of training samples in a training sample set based on feature clustering; during the training process, the identity information and non-identity information of the face objects in the sample face images are decoupled. After obtaining the pre-trained face recognition model, the server can also receive images to be recognized sent by users through terminal devices, perform face recognition, and obtain face recognition results. As an example, server 105 can be a cloud server.
[0028] It should be noted that the server can be hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or it can be implemented as a single server. When the server is software, it can be implemented as multiple software or software modules (for example, software or software modules used to provide distributed services), or it can be implemented as a single software or software module. No specific limitation is made here.
[0029] It should also be noted that the face recognition model training method and face recognition method provided in the embodiments of the present disclosure can be executed by a server, or by a terminal device, or by a server and a terminal device in cooperation with each other. Accordingly, the face recognition model training device and the face recognition device, including various parts (such as various units), can all be set in the server, or all be set in the terminal device, or can be set in the server and the terminal device respectively.
[0030] It should be understood that Figure 1 The number of terminal devices, networks, and servers in the system is only illustrative. Depending on the implementation requirements, there may be any number of terminal devices, networks, and servers. When the training method of the face recognition model and the electronic device on which the face recognition method runs do not need to transmit data with other electronic devices, the system architecture may only include the training method of the face recognition model and the electronic device on which the face recognition method runs (e.g., a server or terminal device).
[0031] Please refer to Figure 2 , Figure 2 This is a flowchart of a method for training a face recognition model provided by an embodiment of the present disclosure. Process 200 includes the following steps:
[0032] Step 201: Obtain a training sample set.
[0033] In this embodiment, the execution subject of the training method of the face recognition model (for example, Figure 1 The terminal device or server in the network can obtain the training sample set remotely or locally through a wired network connection or a wireless network connection.
[0034] The training samples in the training sample set include sample face images and identity labels corresponding to identity information of face objects in the sample face images and non-identity labels corresponding to non-identity information. The non-identity labels of the training samples are obtained based on shallow feature clustering of the sample face images in the training sample set.
[0035] Identity information can be attribute information that represents a person's identity, such as age, gender, facial information, and body information, and non-identity information can be any information other than identity information. The identity label corresponding to the identity information of each sample face image is pre-set, and the non-identity label corresponding to the non-identity information is obtained by feature clustering.
[0036] As an example, the execution subject can perform feature extraction on each sample face image in the training sample set to obtain shallow features of each sample face image; then, based on the obtained multiple shallow features, determine the similarity between each of them, and then perform clustering based on the similarity to obtain multiple categories. For each sample face image in the training sample set, the category corresponding to the sample face image is determined as a non-identity label. The shallow features of the sample face image mainly include the non-identity information of the face object.
[0037] In the field of deep learning, it is often necessary to use the deep features corresponding to face images. Deep features can be obtained by performing multiple or multiple stages of feature extraction on face images. In the process of obtaining deep features, the initially obtained features can be used as shallow features of face images. Taking the residual network as an example, it generally performs feature extraction and other processing operations on the image in five stages, and the features obtained in the first and second stages can be used as shallow features.
[0038] When clustering, you can use methods such as partition-based clustering methods, hierarchy-based clustering methods, and density-based clustering methods.
[0039] It should be noted that the multiple categories obtained by clustering do not necessarily have information with definite indicative significance, and can be different clusters obtained by simply clustering shallow features. Different categories can be distinguished by text, numbers, pictures, etc., so as to set non-identity labels for each sample face image in the training sample set.
[0040] In some optional implementations of this embodiment, the execution subject may determine the non-identity label of the training sample in the following manner:
[0041] First, for the sample face images in the training sample set, shallow features are extracted through a pre-trained feature extraction network.
[0042] It can be understood that the pre-trained feature extraction network can be a network that can extract deep features of the sample face image, or a model that can only extract shallow features. When the feature extraction network can extract deep features of the sample face image, shallow features can be selected from the extraction process of deep features.
[0043] Second, the shallow features of the sample face images in the training sample set are clustered to obtain a preset number of categories.
[0044] Among them, the preset number can be specifically set according to the actual situation (such as the parameter scale of the face recognition model) and is not limited here.
[0045] Third, for each sample face image in the training sample set, the category corresponding to the sample face image is determined as the non-identity label corresponding to the sample face image.
[0046] In this implementation, a process for obtaining non-identity labels of a preset number of categories based on a clustering method is provided, which improves the flexibility of obtaining non-identity labels.
[0047] The identity labels in the training samples in the training sample set can be obtained based on data collection. As an example, in transfer scenarios such as airports and railway stations, it is generally necessary to compare the passenger images collected on site with the passenger ID images to verify whether the passenger and the person represented by the ID image are the same person. In this scenario, the above-mentioned execution entity can use the passenger images collected on site as sample face images and the identity information represented by the ID image representing the same person as the passenger as the identity label.
[0048] Step 202, using a machine learning method, taking a sample facial image as input, extracting identity features and non-identity features of the input sample facial image through an initial facial recognition model, and using the identity label and non-identity label corresponding to the input sample facial image as the expected output of the first output result obtained according to the extracted identity feature and the second output result obtained according to the extracted non-identity feature, respectively, to update the initial image recognition model and train a facial recognition model.
[0049] In this embodiment, the execution subject can use a machine learning method to take a sample face image as input, extract the identity features and non-identity features of the input sample face image through the initial face recognition model, and use the identity label and non-identity label corresponding to the input sample face image as the first output result obtained according to the extracted identity feature and the expected output of the second output result obtained according to the extracted non-identity feature, update the initial image recognition model, and train the face recognition model.
[0050] The initial face recognition model can be any deep learning model with face recognition function, including but not limited to network models such as recurrent neural network, convolutional neural network, residual network, etc. The above-mentioned execution subject can adopt batch training, random training or single training to train the initial face recognition model. Specifically, first, select untrained training samples from the training sample set, input the sample face images in the selected training samples into the initial face recognition model, and extract features of the input sample face images through the feature extraction network in the initial face recognition model to obtain identity features and non-identity features of the sample face images. Then, for the identity feature, the corresponding identity label of the input sample face image is used as the expected output to obtain the first output result about the identity information; for the non-identity feature, the corresponding non-identity label of the input sample face image is used as the expected output to obtain the second output result about the non-identity information. Then, the classification loss between the first output result about the identity information and the identity label, and the classification loss between the second output result about the non-identity information and the non-identity label are calculated. Finally, the gradient is calculated according to the classification loss, and the parameters of the initial face recognition model are updated based on the gradient descent method and the stochastic gradient descent method.
[0051] The execution subject executes the training process in a loop, and when it is determined that the preset end condition is reached, the trained initial face recognition model is determined as the face recognition model. The preset end condition may be, for example, that the number of training times exceeds a preset number threshold, the training time exceeds a preset time threshold, and the classification loss tends to converge.
[0052] In some optional implementations of this embodiment, the execution subject may perform step 202 in the following manner:
[0053] Using machine learning methods, with sample face images as input, the identity features and non-identity features of the input sample face images are extracted through the initial face recognition model, and the identity labels and non-identity labels corresponding to the input sample face images are used as the expected outputs of the first output result and the second output result respectively. With the goal of minimizing the correlation between the extracted identity features and non-identity features, the initial image recognition model is updated to train a face recognition model.
[0054] The identity information and non-identity information of the face object in the sample face image are two relatively independent types of information, and the correlation between the two should be low. Furthermore, the correlation between the identity features that represent the identity information and the non-identity features that represent the non-identity information should also be low. Based on the identity features and non-identity features extracted from the input sample face image, the identity labels and non-identity labels corresponding to the input sample face image are taken as the expected outputs in turn, and the correlation between the identity features and the non-identity features is minimized. This can completely decouple the identity information and the non-identity information, and further improve the recognition accuracy of the trained face recognition model.
[0055] As an example, a first objective function for minimizing the classification loss between the actual output regarding identity information and the identity label, a second objective function for minimizing the classification loss between the actual output regarding non-identity information and the non-identity label, and a third objective function for minimizing the correlation between identity features and non-identity features can be set to train the initial face recognition model through the first objective function, the second objective function, and the third objective function.
[0056] In some optional implementations of this embodiment, the execution subject may be divided into two training stages, and the initial face recognition model may be trained in the following manner:
[0057] First, the backbone network used to extract features in the initial face recognition model is fixed, and the first regularized network and the second regularized network in the face recognition model are trained using machine learning methods with sample face images as input and the goal of maximizing the correlation between regularized identity features and regularized non-identity features.
[0058] The first regularization network and the second regularization network are used to obtain regularized identity features and regularized non-identity features according to the identity features and non-identity features of the input sample face image, respectively. Specifically, the first regularization network is used to regularize the identity features, and the second regularization network is used to regularize the non-identity features.
[0059] In this stage, with the goal of maximizing the correlation between the regularized identity features and the regularized non-identity features, only the first regularized network and the second regularized network are updated, and the backbone network is not updated.
[0060] Second, fix the first regularized network and the second regularized network, use the machine learning algorithm, take the sample face image as input, extract the identity features and non-identity features of the input sample face image through the initial face recognition model, and use the identity label and non-identity label corresponding to the input sample face image as the expected output of the first output result and the second output result respectively, and update the backbone network with the goal of minimizing the correlation between the extracted identity features and non-identity features.
[0061] In this stage, for the identity features and non-identity features of the input sample face images, the identity labels and non-identity labels corresponding to the input sample face images are taken as the expected outputs in turn, and the correlation between the regularized identity features corresponding to the identity features and the regularized non-identity features corresponding to the non-identity features is minimized. Only the backbone network is updated, and the first regularized network and the second regularized network are not updated. The update process of the above two stages can refer to the training process of calculating gradients and updating model parameters according to the gradients.
[0062] During the training process, the above execution entity can calculate the correlation between the regularized identity features and the regularized non-identity features by the following formula:
[0063]
[0064] Among them, ρ represents the correlation, v a 、v b They represent regularized identity features and regularized non-identity features respectively, Cov() represents covariance, and Var() represents variance.
[0065] In this implementation, in the first stage, during the training process of the first regularized network and the second regularized network, the goal is to maximize the correlation between the regularized identity features and the regularized non-identity features; in the second stage, during the training process of the backbone network, the goal is to minimize the correlation between the regularized identity features and the regularized non-identity features, thereby further improving the recognition accuracy of the trained face recognition model.
[0066] During the training process, the execution subject may extract identity features through the feature extraction network corresponding to identity information in the initial face recognition model, and extract non-identity features through the feature extraction network corresponding to non-identity information in the initial face recognition model. In some optional implementations of this embodiment, the execution subject may obtain the identity features and non-identity features of the sample face image in the following manner:
[0067] First, the backbone network in the initial face recognition model is used to extract features from the input sample face image to obtain facial features.
[0068] Among them, facial features include identity features and non-identity features.
[0069] Second, the facial features are segmented to obtain identity features and non-identity features.
[0070] Taking the feature vector with facial features as the preset dimension as an example, the above-mentioned execution entity can determine the vector represented by the first preset dimension as the identity feature, and determine the vector represented by the second preset dimension as the non-identity feature.
[0071] In this implementation, a method is provided for obtaining identity features and non-identity features by segmenting facial features, thereby simplifying the network structure of the face recognition model.
[0072] Continue to see Figure 3 , Figure 3 FIG. 3 is a schematic diagram 300 of an application scenario of the training method for a face recognition model according to this embodiment. Figure 3 In the application scenario, the server first obtains a training sample set 301. The training samples in the training sample set include sample face images and identity labels corresponding to the identity information of the face objects in the sample face images and non-identity labels corresponding to the non-identity information. The non-identity labels of the training samples are obtained based on shallow feature clustering of the sample face images in the training sample set. After obtaining the training sample set 301, the server uses a machine learning method, takes the sample face images 302 as input, and for the identity features 303 and non-identity features 304 extracted from the input sample face images 302, uses the identity labels 305 and non-identity labels 306 corresponding to the input sample face images as the expected outputs of the first output result obtained according to the extracted identity features and the second output result obtained according to the extracted non-identity features, respectively, to train a face recognition model.
[0073] In this embodiment, an unsupervised method is adopted to obtain non-identity labels of training samples in the training sample set based on feature clustering, which improves the efficiency of determining non-identity labels and saves users' resource consumption in obtaining labels; in the training process of the face recognition model, the identity information and non-identity information of the face objects in the sample face images are decoupled, thereby improving the recognition accuracy of the face recognition model.
[0074] Continue to refer Figure 4 , shows a schematic process 400 of an embodiment of a training method for a face recognition model according to the method of the present disclosure. The process 400 includes the following steps:
[0075] Step 401: Obtain a training sample set.
[0076] The training samples in the training sample set include sample face images and identity labels corresponding to identity information of face objects in the sample face images.
[0077] Step 402: For the sample face images in the training sample set, shallow features are extracted through a pre-trained feature extraction network.
[0078] Step 403: cluster the shallow features of the sample face images in the training sample set to obtain a preset number of categories.
[0079] Step 404: for each sample face image in the training sample set, determine the category corresponding to the sample face image as the non-identity label corresponding to the sample face image.
[0080] Step 405, fix the backbone network for extracting features in the initial face recognition model, use a machine learning method, take the sample face image as input, and take maximizing the correlation between the regularized identity features and the regularized non-identity features as the goal to train the first regularized network and the second regularized network in the face recognition model.
[0081] Among them, the first regularization network and the second regularization network are used to obtain regularized identity features and regularized non-identity features according to the identity features and non-identity features of the input sample face image respectively.
[0082] Step 406, fix the first regularized network and the second regularized network, use a machine learning algorithm, take the sample face image as input, extract the identity features and non-identity features of the input sample face image through the initial face recognition model, use the identity label and non-identity label corresponding to the input sample face image as the expected output of the first output result and the second output result respectively, and update the backbone network with the goal of minimizing the correlation between the extracted identity features and non-identity features.
[0083] It can be seen from this embodiment that Figure 2 Compared with the corresponding embodiments, process 400 of the training method of the face recognition model in this embodiment specifically illustrates the process of clustering to obtain non-identity labels. The training process of the face recognition model improves the efficiency of determining non-identity labels, saves users' resource consumption in obtaining labels, and improves the recognition accuracy of the face recognition model.
[0084] Please refer to Figure 5 , Figure 5 A flow chart of a face recognition method provided in an embodiment of the present disclosure, wherein process 500 includes the following steps:
[0085] Step 501: Obtain an image to be recognized.
[0086] In this embodiment, the execution subject of the face recognition method (for example Figure 1 The terminal device or server in the system can obtain the image to be identified remotely or locally through a wired network connection or a wireless network connection.
[0087] The image to be identified can be any image. As an example, the image to be identified is each frame of the video captured by the monitoring device.
[0088] Step 502: recognize the image to be recognized by using the pre-trained face recognition model to obtain a face recognition result.
[0089] In this embodiment, the execution subject can recognize the image to be recognized by using the pre-trained face recognition model to obtain the face recognition result. The face recognition result represents the identity information of the face object in the image to be recognized.
[0090] Among them, the face recognition model is obtained based on the training method shown in the above embodiments 200 and 400.
[0091] In this implementation example, the face recognition result of the image to be recognized is obtained through the face recognition model, thereby improving the recognition accuracy of the face recognition result.
[0092] Continue to refer Figure 6 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a training device for a face recognition model. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.
[0093] like Figure 6 As shown, a training device for a face recognition model includes: a first acquisition unit 601, configured to acquire a training sample set, wherein the training samples in the training sample set include a sample face image and an identity label corresponding to the identity information of the face object in the sample face image and a non-identity label corresponding to the non-identity information, and the non-identity label of the training sample is obtained based on shallow feature clustering of the sample face images in the training sample set; a training unit 602, configured to use a machine learning method, with the sample face image as input, to extract the identity feature and non-identity feature of the input sample face image through an initial face recognition model, and to use the identity label and non-identity label corresponding to the input sample face image as the expected output of the first output result obtained according to the extracted identity feature and the second output result obtained according to the extracted non-identity feature, respectively, to update the initial image recognition model, and to train the face recognition model.
[0094] In some optional implementations of the present embodiment, the training unit 602 is further configured to: utilize a machine learning method, take a sample facial image as input, extract identity features and non-identity features of the input sample facial image through an initial facial recognition model, use the identity label and non-identity label corresponding to the input sample facial image as the expected output of the first output result and the second output result respectively, and update the initial image recognition model with the goal of minimizing the correlation between the extracted identity features and non-identity features, so as to train and obtain a facial recognition model.
[0095] In some optional implementations of this embodiment, the training unit 602 is further configured to: fix the backbone network used for extracting features in the initial face recognition model, use a machine learning method, take a sample face image as input, and train the first regularized network and the second regularized network in the face recognition model with the goal of maximizing the correlation between the regularized identity features and the regularized non-identity features, wherein the first regularized network and the second regularized network are used to obtain the regularized identity features and the regularized non-identity features according to the identity features and the non-identity features of the input sample face image, respectively; fix the first regularized network and the second regularized network, use a machine learning algorithm, take the sample face image as input, extract the identity features and the non-identity features of the input sample face image through the initial face recognition model, use the identity label and the non-identity label corresponding to the input sample face image as the expected output of the first output result and the second output result, respectively, and update the backbone network with the goal of minimizing the correlation between the extracted identity features and the non-identity features.
[0096] In some optional implementations of the present embodiment, the above-mentioned device also includes: a determination unit (not shown in the figure), which is configured to extract shallow features from the sample face images in the training sample set through a pre-trained feature extraction network; cluster the shallow features of the sample face images in the training sample set to obtain a preset number of categories; for each sample face image in the training sample set, determine the category corresponding to the sample face image as the non-identity label corresponding to the sample face image.
[0097] In some optional implementations of this embodiment, the training unit 602 is further configured to: extract features of the input sample face image through the backbone network in the initial face recognition model to obtain face features; and segment the face features to obtain identity features and non-identity features.
[0098] In this embodiment, an unsupervised method is adopted to obtain non-identity labels of training samples in the training sample set based on feature clustering, which improves the efficiency of determining non-identity labels and saves users' resource consumption in obtaining labels; in the training process of the face recognition model, the identity information and non-identity information of the face objects in the sample face images are decoupled, thereby improving the recognition accuracy of the face recognition model.
[0099] Continue to refer Figure 7 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a face recognition device. Figure 5 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.
[0100] like Figure 7 As shown, the training device of the face recognition model includes: a second acquisition unit 701, configured to acquire an image to be recognized; and a recognition unit 702, configured to recognize the image to be recognized through a pre-trained face recognition model to obtain a face recognition result. The face recognition model is obtained by training in embodiments 200 and 400.
[0101] In this implementation example, the face recognition result of the image to be recognized is obtained through the face recognition model, thereby improving the recognition accuracy of the face recognition result.
[0102] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that when the at least one processor executes, the training method of the face recognition model and the face recognition method described in any of the above embodiments can be implemented.
[0103] According to an embodiment of the present disclosure, the present disclosure also provides a readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to implement the training method of the face recognition model and the face recognition method described in any of the above embodiments when executed.
[0104] The embodiments of the present disclosure provide a computer program product, which, when executed by a processor, can implement the training method of the face recognition model and the face recognition method described in any of the above embodiments.
[0105] Figure 8A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.
[0106] like Figure 8 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.
[0107] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0108] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as the training method of the face recognition model and the face recognition method. For example, in some embodiments, the training method of the face recognition model and the face recognition method may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the training method of the face recognition model and the face recognition method described above may be executed. Alternatively, in other embodiments, the computing unit 801 may be configured to execute a training method for a face recognition model or a face recognition method in any other appropriate manner (for example, by means of firmware).
[0109] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0110] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0111] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0112] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0113] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.
[0114] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and virtual private servers (VPS) services; it may also be a server for a distributed system, or a server combined with a blockchain.
[0115] According to the technical solution of the embodiment of the present disclosure, an unsupervised method is adopted to obtain non-identity labels of training samples in a training sample set based on feature clustering, thereby improving the efficiency of determining non-identity labels and saving the resource consumption of users in obtaining labels; in the training process of the face recognition model, the identity information and non-identity information of the face objects in the sample face images are decoupled, thereby improving the recognition accuracy of the face recognition model.
[0116] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution provided by this disclosure can be achieved, and this document does not limit this.
[0117] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.
Claims
1. A training method for a face recognition model, include: Acquire a training sample set, wherein the training samples in the training sample set include sample face images and identity labels corresponding to identity information of face objects in the sample face images and non-identity labels corresponding to non-identity information, and the non-identity labels of the training samples are obtained based on shallow feature clustering of the sample face images in the training sample set; Using a machine learning method, with a sample face image as input, the identity features and non-identity features of the input sample face image are extracted through an initial face recognition model; a first output result is obtained based on the extracted identity features; a second output result is obtained based on the extracted non-identity features; the classification loss between the first output result and the identity label corresponding to the input sample face image, and the classification loss between the second output result and the non-identity label corresponding to the input sample face image are calculated; based on the classification loss, the initial face recognition model is updated to obtain the face recognition model through training.
2. The method according to claim 1, in, The machine learning method is used to take the sample face image as input, and the identity features and non-identity features of the input sample face image are extracted through the initial face recognition model; Obtaining a first output result based on the extracted identity feature; obtaining a second output result based on the extracted non-identity feature; Calculating the classification loss between the first output result and the identity label corresponding to the input sample face image, and the classification loss between the second output result and the non-identity label corresponding to the input sample face image; The initial face recognition model is updated according to the classification loss, and the face recognition model is obtained by training, including: Using a machine learning method, with a sample face image as input, the identity features and non-identity features of the input sample face image are extracted through the initial face recognition model, and the identity label and non-identity label corresponding to the input sample face image are used as the expected output of the first output result and the second output result respectively, and with the goal of minimizing the correlation between the extracted identity features and non-identity features, the initial face recognition model is updated to obtain the face recognition model through training.
3. The method according to claim 2, in, The method of using a machine learning method, taking a sample face image as input, extracting identity features and non-identity features of the input sample face image through the initial face recognition model, taking the identity label and non-identity label corresponding to the input sample face image as the expected output of the first output result and the second output result in turn, and updating the initial face recognition model with the goal of minimizing the correlation between the extracted identity features and non-identity features, and training to obtain the face recognition model, includes: The backbone network for extracting features in the initial face recognition model is fixed, and a first regularized network and a second regularized network in the face recognition model are trained by using a machine learning method, taking a sample face image as input and maximizing the correlation between the regularized identity features and the regularized non-identity features as the goal, wherein the first regularized network and the second regularized network are used to obtain regularized identity features and regularized non-identity features according to the identity features and non-identity features of the input sample face image, respectively; The first regularized network and the second regularized network are fixed, and a machine learning algorithm is used to take a sample face image as input. The identity features and non-identity features of the input sample face image are extracted through the initial face recognition model, and the identity label and non-identity label corresponding to the input sample face image are used as the expected outputs of the first output result and the second output result respectively, and the backbone network is updated with the goal of minimizing the correlation between the extracted identity features and non-identity features.
4. The method according to any one of claims 1 to 3, in, Also includes: For the sample face images in the training sample set, shallow features are extracted through a pre-trained feature extraction network; Clustering the shallow features of the sample face images in the training sample set to obtain a preset number of categories; For each sample face image in the training sample set, the category corresponding to the sample face image is determined as the non-identity label corresponding to the sample face image.
5. The method according to any one of claims 1 to 3, in, The identity features and non-identity features of the sample face image are obtained in the following way: Extracting features of the input sample face image through the backbone network in the initial face recognition model to obtain face features; The facial features are segmented to obtain identity features and non-identity features.
6. A face recognition method, include: Obtain an image to be recognized; The image to be recognized is recognized by a pre-trained face recognition model to obtain a face recognition result, wherein the face recognition model is trained by any one of claims 1-5.
7. A training device for a face recognition model, include: A first acquisition unit is configured to acquire a training sample set, wherein the training samples in the training sample set include a sample face image and an identity label corresponding to the identity information of a face object in the sample face image and a non-identity label corresponding to the non-identity information, and the non-identity label of the training sample is obtained based on shallow feature clustering of the sample face image in the training sample set; The training unit is configured to use a machine learning method, take a sample face image as input, and extract the identity features and non-identity features of the input sample face image through an initial face recognition model; obtain a first output result based on the extracted identity features; obtain a second output result based on the extracted non-identity features; calculate the classification loss between the first output result and the identity label corresponding to the input sample face image, and the classification loss between the second output result and the non-identity label corresponding to the input sample face image; update the initial face recognition model based on the classification loss, and obtain the face recognition model through training.
8. The device according to claim 7, in, The training unit is further configured to: Using a machine learning method, with a sample face image as input, the identity features and non-identity features of the input sample face image are extracted through the initial face recognition model, and the identity label and non-identity label corresponding to the input sample face image are used as the expected output of the first output result and the second output result respectively, and with the goal of minimizing the correlation between the extracted identity features and non-identity features, the initial face recognition model is updated to obtain the face recognition model through training.
9. The device according to claim 8, in, The training unit is further configured to: The backbone network for extracting features in the initial face recognition model is fixed, and a machine learning method is used to take a sample face image as input, and the first regularized network and the second regularized network in the face recognition model are trained with the goal of maximizing the correlation between the regularized identity features and the regularized non-identity features, wherein the first regularized network and the second regularized network are used to obtain regularized identity features and regularized non-identity features according to the identity features and non-identity features of the input sample face image, respectively; the first regularized network and the second regularized network are fixed, and a machine learning algorithm is used to take a sample face image as input, and the identity features and non-identity features of the input sample face image are extracted through the initial face recognition model, and the identity label and non-identity label corresponding to the input sample face image are used as the expected outputs of the first output result and the second output result, respectively, and the backbone network is updated with the goal of minimizing the correlation between the extracted identity features and non-identity features.
10. The device according to any one of claims 7 to 9, in, Also includes: A determination unit is configured to extract shallow features from sample face images in the training sample set through a pre-trained feature extraction network; The shallow features of the sample face images in the training sample set are clustered to obtain a preset number of categories; for each sample face image in the training sample set, the category corresponding to the sample face image is determined as the non-identity label corresponding to the sample face image.
11. The device according to any one of claims 7 to 9, in, The training unit is further configured to: The input sample face image is subjected to feature extraction through the backbone network in the initial face recognition model to obtain face features; the face features are segmented to obtain identity features and non-identity features.
12. A face recognition device, include: A second acquisition unit is configured to acquire an image to be recognized; The recognition unit is configured to recognize the image to be recognized through a pre-trained face recognition model to obtain a face recognition result, wherein the face recognition model is obtained by training according to any one of claims 7 to 11.
13. An electronic device, It is characterized in that include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.
14. A non-transitory computer-readable storage medium storing computer instructions, It is characterized in that The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 6.
Citation Information
Patent Citations
Model training method and device, face recognition method and device, equipment and storage medium
CN109993125A
Face recognition method and device, computer equipment and readable storage medium
CN111476216A
