Face recognition model training method, device, electronic device and storage medium

Abstract

The present application provides a face recognition model training method, device, electronic device, and storage medium. The method includes: using a sample set to perform a first-stage training on a basic face recognition model to obtain a first face recognition model; inputting each image sample into the first face recognition model to obtain a feature vector for each image sample, and calculating the quality score corresponding to each image sample; based on the probability and quality score, updating the gradient scaling items of the center vector and the feature vector, and performing a second-stage training to obtain a second face recognition model; based on the probability corresponding to each image sample and the determined type of each image sample, updating the gradient scaling items of the center vector and the feature vector during the third-stage training, and performing a third-stage training to obtain a third face recognition model, and using the third face recognition model as the final model. The present application can fully tap into high-quality difficult samples and improve the training effect of the face recognition model.

Description

Technical Field

[0001] The present application relates to the technical field of face recognition model training, and in particular to a face recognition model training method, device, electronic device and storage medium. Background Art

[0002] In the face recognition model training task, in order to maximize the mining of data information, it is necessary to consider the impact of different quality images and noisy images in the training set on model training, improve their positive impact on model training and reduce their negative impact, and train a better recognition model. This brings difficulties to the design of the recognition algorithm.

[0003] Currently, traditional face recognition algorithms based on fixed margins process all data uniformly, failing to consider the impact of varying quality and noise on model training. Face recognition algorithms based on hard sample mining (such as Curricular Face) fail to distinguish between high-quality and low-quality hard sample data, failing to fully mine quality-related information. Furthermore, face recognition algorithms based on quality cannot fully exploit the value of hard samples, often resulting in suboptimal model results. Consequently, existing face recognition model training algorithms are unable to mine high-quality sample information, making it impossible to use high-quality samples to train better face recognition models, thus reducing the effectiveness of face recognition model training. Summary of the Invention

[0004] In view of this, the embodiments of the present application provide a face recognition model training method, device, electronic device and storage medium to solve the problems existing in the prior art that high-quality sample information cannot be mined, high-quality samples cannot be used to train a better face recognition model, and the training effect of the face recognition model is reduced.

[0005] In a first aspect of an embodiment of the present application, a face recognition model training method is provided, comprising: performing a first-stage training on a pre-configured basic face recognition model using a sample set to obtain a first face recognition model after the first-stage training, wherein the sample set contains image samples of different qualities; inputting each image sample into the first face recognition model in turn to obtain a feature vector corresponding to each image sample, and calculating a quality score corresponding to each image sample based on the feature vector using a quality score algorithm; determining the probability of each image sample being within the category to which it belongs, and updating a center vector and a gradient scaling item of the feature vector in the first face recognition model based on the probability and the quality score of the image sample, and performing a gradient scaling operation on the first face recognition model after the gradient scaling item is updated using the sample set. The second stage of training obtains a second face recognition model after the second stage of training; based on the probability corresponding to each image sample, the type of each image sample is determined. When the image sample is a simple sample, the center vector and the gradient scaling item of the eigenvector of the second face recognition model when the simple sample is used for the third stage of training are set as the initial value; when the image sample is a difficult sample, the center vector and the gradient scaling item of the eigenvector of the second face recognition model when the difficult sample is used for the third stage of training are updated using a predetermined angle; the second face recognition model after the gradient scaling item is updated is trained for the third stage using the sample set to obtain a third face recognition model after the third stage of training, and the third face recognition model is used as the final trained model.

[0006] According to a second aspect of an embodiment of the present application, a face recognition model training device is provided, comprising: a first-stage training module, configured to perform a first-stage training on a pre-configured basic face recognition model using a sample set, to obtain a first face recognition model after the first-stage training, wherein the sample set contains image samples of different qualities; a quality score module, configured to input each image sample into the first face recognition model in sequence, to obtain a feature vector corresponding to each image sample, and to calculate a quality score corresponding to each image sample based on the feature vector using a quality score algorithm; a second-stage training module, configured to determine the probability of each image sample being within the category to which it belongs, and to update a center vector and a gradient scaling item of a feature vector in the first face recognition model based on the probability and the quality score of the image sample, and to use the sample set to update the gradient scaling item of the first face recognition model after the gradient scaling item is updated. The face recognition model is trained in the second stage to obtain a second face recognition model after the second stage training; the update module is configured to determine the type of each image sample based on the probability corresponding to each image sample, and when the image sample is a simple sample, the center vector and the gradient scaling item of the feature vector of the second face recognition model when the simple sample is used to train the third stage are set as the initial value; when the image sample is a difficult sample, the center vector and the gradient scaling item of the feature vector of the second face recognition model when the difficult sample is used to train the third stage are updated using a predetermined angle; the third stage training module is configured to use the sample set to perform the third stage training on the second face recognition model after the gradient scaling item is updated, to obtain a third face recognition model after the third stage training, and use the third face recognition model as the final trained model.

[0007] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the steps of the above method are implemented when the processor executes the program.

[0008] According to a fourth aspect of the embodiments of the present application, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps of the above method are implemented.

[0009] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects:

[0010] The first face recognition model after the first stage training is obtained by using the sample set to perform the first stage training on the pre-configured basic face recognition model, and the sample set contains picture samples of different qualities; each picture sample is input into the first face recognition model in turn to obtain the feature vector corresponding to each picture sample, and the quality score corresponding to each picture sample is calculated based on the feature vector using the quality score algorithm; the probability of each picture sample being in the category to which it belongs is determined, and based on the probability and the quality score of the picture sample, the center vector and the gradient scaling item of the feature vector in the first face recognition model are updated, and the first face recognition model after the gradient scaling item is updated is trained for the second stage using the sample set, and the second stage training is obtained. The second face recognition model after training; based on the probability corresponding to each image sample, the type of each image sample is determined. When the image sample is a simple sample, the center vector and the gradient scaling item of the feature vector of the second face recognition model during the third stage training using the simple sample are set as the initial value; when the image sample is a difficult sample, the center vector and the gradient scaling item of the feature vector of the second face recognition model during the third stage training using the difficult sample are updated using a predetermined angle; the second face recognition model after the gradient scaling item is updated is trained in the third stage using the sample set to obtain the third face recognition model after the third stage training, and the third face recognition model is used as the final trained model. This application can fully tap into high-quality difficult samples, so that high-quality difficult samples can obtain larger gradients, so that the face recognition model can obtain more sufficient training, and improve the training effect of the face recognition model. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0012] Figure 1 This is a flow chart of the face recognition model training method provided in the embodiment of the present application;

[0013] Figure 2 This is a structural diagram of a face recognition model training device provided in an embodiment of the present application;

[0014] Figure 3 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0015] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0016] As introduced in the background technology content above, in the current training task of face recognition models, due to the uneven quality of samples, low-quality samples and noise samples have a greater impact on model training, while high-quality difficult samples have more positive effects on model training. Therefore, it is necessary to distinguish between high-quality difficult sample data and low-quality difficult sample data in the face recognition model training process, and fully explore the information of high-quality difficult samples, thereby fully exploring the value of samples. This is of great significance to improving the training effect of the face recognition model and the accuracy of the recognition results of the face recognition model.

[0017] This application obtains a basic face recognition model and a quality modeling model through basic recognition model training and quality model training, and then further focuses on training high-quality difficult samples by modifying the gradients of samples of different qualities. Finally, by mining difficult samples and abandoning noise samples, critical sample information is further utilized to provide a multi-stage face recognition algorithm based on gradient optimization based on dynamic weighting of quality and angle.

[0018] The purpose of this application is to train a face recognition model that has more accurate recognition capabilities than models trained with traditional algorithms. The input of the basic face recognition model of this application is a face recognition dataset. The face recognition dataset contains multiple pictures and ID information corresponding to each picture (i.e., personal information corresponding to the face picture). The picture data is usually more than 1 million, and the number of IDs is usually more than 100,000. The final output of this application is a face recognition deep learning model. In practical applications, a face picture is input into the face recognition deep learning model, and the model can give a feature vector of length 512 and the ID corresponding to the face picture. The recognition results can be used for subsequent tasks such as face comparison.

[0019] It should be noted that the following embodiments of this application use the CosFace model as the basic face recognition model for training. However, it should be understood that the face recognition model training objects of the embodiments of this application are not limited to the CosFace model. Other neural network-based face recognition models are also applicable to this application. The above application scenarios do not constitute a limitation on the technical solutions of this application.

[0020] Figure 1 It is a flowchart of the face recognition model training method provided in the embodiment of the present application. Figure 1 The face recognition model training method can be executed by the server. Figure 1 As shown, the face recognition model training method may specifically include:

[0021] S101, performing a first-stage training on a pre-configured basic face recognition model using a sample set to obtain a first face recognition model after the first-stage training, wherein the sample set includes image samples of different qualities;

[0022] S102, inputting each image sample into the first face recognition model in sequence to obtain a feature vector corresponding to each image sample, and calculating a quality score corresponding to each image sample based on the feature vector using a quality score algorithm;

[0023] S103, determining the probability of each image sample being within the category to which it belongs, and updating the gradient scaling terms of the center vector and the feature vector in the first face recognition model based on the probability and the quality score of the image sample, and performing a second-stage training on the first face recognition model after the gradient scaling terms are updated using the sample set to obtain a second face recognition model after the second-stage training;

[0024] S104, based on the probability corresponding to each image sample, determining the type of each image sample; when the image sample is a simple sample, setting the center vector and the gradient scaling items of the feature vector when the second face recognition model is trained in the third stage using the simple sample as initial values; when the image sample is a difficult sample, updating the center vector and the gradient scaling items of the feature vector when the second face recognition model is trained in the third stage using the difficult sample using a predetermined angle;

[0025] S105, performing a third stage of training on the second face recognition model after the gradient scaling term is updated using the sample set to obtain a third face recognition model after the third stage of training, and using the third face recognition model as the final trained model.

[0026] Specifically, the sample set of the embodiment of the present application contains a large number of face images, each of which corresponds to an ID tag. However, an ID tag can correspond to multiple face images, and each face image in the sample set can be considered as an image sample. The first stage training, second stage training, and third stage training in the embodiment of the present application are only used to express the training process at different stages, that is, the basic face recognition model first undergoes the first stage training, and then continues to the second stage training. The third stage training is performed on the basis of the face recognition model after the second stage training to obtain the final face recognition model.

[0027] Furthermore, the sample quality mentioned in the embodiments of the present application means that the higher the clarity of an image sample, the higher the quality of the sample is considered to be, and the blurrier the content of the image sample, the lower the quality of the sample is considered to be; in addition, a simple sample refers to a sample whose error with the true value label is smaller during prediction, and a difficult sample refers to a sample whose error with the true value label is larger during prediction. Difficult samples are also referred to as difficult samples.

[0028] According to the technical solutions provided in the embodiments of this application, the face recognition model training process of this application mainly includes three steps: the first step is to train the basic face recognition model and calculate the quality score; the second step is to mine high-quality difficult samples based on the quality score; and the third step is to mine difficult samples based on the similarity of central features. The specific contents of these three steps are described in detail below in conjunction with specific embodiments.

[0029] In some embodiments, the basic face recognition model adopts the CosFace model, and the pre-configured basic face recognition model is trained in the first stage using the sample set to obtain a first face recognition model after the first stage training, including: using the picture samples in the sample set as the input of the CosFace model, and using the picture samples to train the CosFace model in the first stage to obtain the first face recognition model, the first face recognition model is used to automatically identify the category corresponding to the face picture based on the input face picture; wherein, the sample set contains picture samples and category labels corresponding to the picture samples.

[0030] Specifically, the basic face recognition model of the embodiment of the present application adopts the CosFace model structure. In the CosFace model structure, the backbone adopts the IR50 structure. The input of the model is a 112x112 normalized face image, and the output is a feature vector with a length of 512. The training loss function of the CosFace model is:

[0031]

[0032] Among them, x i represents the feature vector of the i-th sample, θ yi Represents x i The angle between the vector and its center, m represents the margin value, s represents the scaling factor, and n represents the total number of categories in the training set.

[0033] Furthermore, the CosFace model was trained using the SGD optimizer, with an initial learning rate set to 0.1, optimizer momentum set to 0.9, and a weight decay factor set to 5e-4. A total of 15 training rounds were performed, and the learning rate was adjusted back to the original 0.1 in the 8th and 12th rounds. Stochastic Gradient Descent (SGD) is an implementation of the gradient descent algorithm. When solving for the minimum value of the loss function, the gradient descent method can be used to iterate step by step to obtain the minimized loss function and the optimal model parameter values. In other words, in the CosFace model, the gradient is calculated using the gradient descent algorithm, and the gradient is used to optimize the model parameters. That is, when optimizing the face recognition model, the gradient is used to update the model parameters.

[0034] Furthermore, during the first stage of model training, the image samples in the obtained sample set are directly input into the pre-trained CosFace model, and the pre-trained CosFace model is trained using the image samples. When the CosFace model is trained, the label corresponding to each image sample is the ID information of the face. In this way, after the first stage of model training is completed, any face image is input into the first face recognition model. The penultimate layer of the first face recognition model can output a feature vector representing the face, and the last layer outputs the ID category corresponding to the face.

[0035] In some embodiments, a quality score algorithm is used to calculate the quality score corresponding to each image sample based on the feature vector, including: based on the feature vector corresponding to each image sample, calculating the first similarity between the feature vector of the image sample and the feature vectors of other image samples in the same category to obtain a first similarity set; calculating the second similarity between the feature vector of the image sample and the feature vectors of multiple randomly sampled heterogeneous samples to obtain a second similarity set; based on the first similarity set, the second similarity set and a preset quality score calculation formula, calculating the quality score corresponding to each image sample.

[0036] Specifically, after the first phase of training results in the first face recognition model, the image samples in the sample set are sequentially input into the first face recognition model to obtain the feature vector corresponding to each image sample. The quality score corresponding to each image sample is then calculated based on the feature vector of each image sample. In practical applications, the quality score of the image sample can be calculated using the SDD method. The quality score calculation process is described in detail below in conjunction with specific embodiments, and may specifically include the following:

[0037] First, each picture in the data set (sample set) is input into the first face recognition model in turn to obtain the feature vector corresponding to each picture; secondly, for the feature vector x of each picture samplei , calculate the cos similarity between it and all other sample feature vectors of this class, and form a set (i.e. the first similarity set), and calculate its cos similarity with the feature vectors of k randomly sampled heterogeneous samples to form a set (i.e. the second similarity set), the quality score is expressed as:

[0038]

[0039] in:

[0040]

[0041]

[0042]

[0043] WD(·) represents the Wasserstein distance between the distributions of the two sets.

[0044] Furthermore, when calculating the quality score corresponding to each image sample, the embodiment of the present application calculates the quality score based on the relationship between the feature vectors of the image samples. That is, the first similarity set is used to represent the cosine similarity between the feature vector of a certain image sample and the feature vectors of other image samples in its category, while the second similarity set is used to represent the cosine similarity between the feature vector of a certain image sample and the feature vectors of other image samples in different categories (i.e., heterogeneous samples). In practical applications, the quality score corresponding to each image sample is between 0 and 100.

[0045] In some embodiments, based on the probability and the quality score of the image sample, the gradient scaling items of the center vector and the feature vector in the first face recognition model are updated, including: based on the probability and the gradient descent algorithm, determining the gradient of the center vector of the category to which the image sample corresponds and the gradient of the feature vector corresponding to the image sample, using the quality score of the image sample to quality-weight the gradient scaling item of the gradient to obtain the quality-weighted gradient scaling item, using the quality-weighted gradient scaling item as the updated gradient scaling item, and using the updated gradient scaling item to update the original gradient scaling item in the first face recognition model.

[0046] Specifically, after the first phase of training to obtain the first face recognition model and calculate the quality score corresponding to each image sample, by combining the CosFace model and sample quality scores after the first phase of training, high-quality sample information is mined by modifying the gradients of samples of different quality. In other words, based on the first face recognition model, the embodiment of the present application updates the gradient of the model, uses the updated gradient to continue to optimize the parameters of the first phase model (i.e., the first face recognition model), and continues to train the second phase model based on the training results of the first phase.

[0047] Furthermore, in the second phase of model training, the CosFace model structure has not changed. The backbone of the CosFace model still uses the IR50 structure. At the beginning of the second phase of model training, all the parameters of the first phase model are loaded first, and the training loss function of the second phase model training still uses the CosFace loss function. In the second phase of training, the following formula is used to calculate the image sample x i The probability of belonging to the jth category (the category to which the image sample belongs):

[0048]

[0049] Furthermore, based on the above probability calculation formula, the gradient of the center vector and the gradient of the feature vector corresponding to the image sample are further derived. The expressions of these two gradients are explained below, specifically:

[0050] Assume that the image sample x i The sample category belongs to the jth category, so the center vector W of the jth category j The gradient expression is:

[0051]

[0052] At the same time, the eigenvector x i The gradient expression is:

[0053]

[0054] In the expressions corresponding to the above two gradients, the expression of the gradient scaling term is:

[0055]

[0056] It should be noted that the center vector W j The gradient expression and eigenvector x i The gradient expressions are derived based on the above probability expressions. The center vector W j The gradient of is used to optimize the center vector, and the eigenvector x iThe gradient of is used to optimize the feature vector, and the gradient scaling terms corresponding to the above two gradients are the same.

[0057] Furthermore, the central vector W is derived using the probability expression j The gradient and eigenvector x i After the gradient of , the embodiment of the present application updates the gradient scaling term of by quality weighting on the basis of the original gradient scaling term, that is, the original gradient scaling term is updated by using the quality score of the image sample to obtain a new gradient scaling term. The specific formula used is:

[0058]

[0059] By multiplying the quality score on the basis of the original gradient scaling term, the original gradient scaling term can be updated, and the original gradient can be adjusted using the updated original gradient scaling term, so that high-quality samples can obtain a larger gradient, and the model can obtain more sufficient training, so that the model can make better use of high-quality sample information.

[0060] In actual applications, in the second stage of model training, by loading the model trained in the first stage (i.e., the first face recognition model), and setting the initial learning rate of the SGD optimizer used in model training to 0.01, and keeping other parameter settings consistent with the first stage training, the second stage model (i.e., the second face recognition model) is obtained by continuing training. The details of the specific training process of the model are not elaborated here.

[0061] In some embodiments, based on the probability corresponding to each image sample, the type of each image sample is determined, including: performing a threshold judgment on the probability corresponding to the image sample, when the probability corresponding to the image sample is greater than or equal to the threshold, treating the image sample as a simple sample, and when the probability corresponding to the image sample is less than the threshold, treating the image sample as a difficult sample.

[0062] Specifically, the second stage of training can mine information about high-quality samples, and the model has achieved good training results. The main purpose of the third stage of training is to mine information about difficult samples at the classification boundary through angle weighting, while ignoring noise samples that may have incorrect labels. Before the start of the third stage of training, the image samples are first classified into types, that is, the probability of each image sample calculated according to the above embodiment is used to determine whether each image sample belongs to a simple sample or a difficult sample (difficult sample).

[0063] Furthermore, when the sample x i The probability P of belonging to the category to which it belongs j (i) ≥0.9, the classification of the image sample is considered to be good, and the image sample is divided into a simple sample; when the sample xi The probability P of belonging to the category to which it belongs j (i) When it is less than 0.9, the image sample is considered to be poorly classified and classified as a difficult sample. Next, different strategies are used to correct the gradient scaling term for easy samples and difficult samples respectively.

[0064] In some embodiments, when the image sample is a simple sample, the gradient scaling items of the center vector and the feature vector when the second face recognition model is trained in the third stage using the simple sample are set to the initial values, including: when the image sample is judged to be a simple sample, when the second face recognition model is trained in the third stage using the simple sample, the gradient scaling items of the center vector and the feature vector corresponding to the simple sample in the second face recognition model are set to the gradient scaling items of the center vector and the feature vector corresponding to the first face recognition model, so that the gradient scaling items of the center vector and the feature vector corresponding to the simple sample in the second face recognition model remain unchanged.

[0065] Specifically, for simple samples, the gradient scaling term is not corrected during the third stage of training, that is, the gradient correction is not performed on samples with good classification (simple samples); that is, when the sample x i The probability P of belonging to this category j (i) When ≥0.9, the classification of the image sample is considered good, and the center vector W of the image sample in the third stage of training is not used. j and the eigenvector x i The gradient scaling term is changed to the same value as the original gradient scaling term used in the first stage of training. Therefore, for simple samples, the gradient does not change during the third stage of training.

[0066] In some embodiments, when the image sample is a difficult sample, the center vector and the gradient scaling items of the feature vector when the second face recognition model is trained in the third stage using the difficult sample are updated using a predetermined angle, including: when the image sample is judged to be a difficult sample, when the second face recognition model is trained in the third stage using the difficult sample, the cosine value of the angle between the feature vector of the difficult sample and the center vector of the sample category corresponding to the difficult sample is obtained, and the cosine value is used to perform angle weighting on the gradient scaling items corresponding to the center vector and the feature vector, so as to update the gradient scaling items of the center vector and the feature vector during the third stage training.

[0067] Specifically, for difficult samples, since the classification of these image samples is not good enough, it is considered that these difficult samples have mining value. By calculating the center vector W of the difficult samples in the third stage training, j and the eigenvector x iThe gradient scaling term is corrected to focus on optimizing the difficult sample information, and the optimization method is also to update the gradient scaling term.

[0068] Furthermore, for samples with suboptimal classification results (i.e., difficult samples), we focus on correcting samples at the edge and reduce the correction of samples far from the center vector (these samples may be noise samples, and their impact on the model needs to be reduced). In actual applications, unlike the second stage, which uses quality weighting to correct the gradient scaling term, the third stage corrects the gradient scaling term through angle weighting. Therefore, we first need to calculate the cosine value of the angle between the feature vector of each difficult sample and the center vector of the category to which it belongs, and then use this cosine value to further correct the gradient scaling term.

[0069] It should be noted that the expressions of the gradient scaling terms corresponding to easy samples and difficult samples are:

[0070]

[0071] The formula on the right side of the equal sign represents the gradient scaling term for simple samples, and the formula below represents the gradient scaling term for difficult samples. It can be seen that when calculating the gradient scaling term for difficult samples, it is necessary to multiply the original gradient scaling term by the cosine value of the angle between the feature vector of the difficult sample and the center vector of the category to which it belongs.

[0072] Furthermore, after determining the gradient scaling terms corresponding to the simple samples and the difficult samples respectively, during the model training of the third stage, the model training results of the second stage (i.e., the second face recognition model) are first loaded, and the initial learning rate of the SGD optimizer used in the model training is set to 0.01. The other parameter settings are consistent with the training of the second stage. The gradient is adjusted by using the gradient scaling terms determined above, and the model parameters are optimized using the adjusted gradient until the third face recognition model is trained.

[0073] According to the technical solution provided in the embodiment of the present application, the present application provides a multi-stage face recognition model training method, wherein the first stage is used to train the basic face recognition model to obtain a first face recognition model; the second stage is used to correct the gradient scaling terms of the center vector and the feature vector using the quality score on the basis of the first face recognition model, and continue to train based on the gradient-adjusted model to obtain a second face recognition model; the third stage is used to correct the gradient scaling terms of the center vector and the feature vector using the angle weighting method on the basis of the second face recognition model, and continue to train based on the gradient-adjusted model to obtain a third face recognition model. The embodiment of the present application can gradually mine the information of ordinary samples, high-quality difficult samples, general difficult samples and noise samples; and can realize the mining of high-quality difficult samples with gradient weighting based on the quality score, and at the same time realize the optimization of difficult samples and noise samples with gradient weighting based on the angle between the feature and the center vector.

[0074] All of the above optional technical solutions can be combined in any way to form optional embodiments of the present application, and will not be described in detail here.

[0075] The following are device embodiments of the present application, which can be used to implement the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

[0076] Figure 2 This is a structural diagram of the face recognition model training device provided in the embodiment of the present application. Figure 2 As shown, the face recognition model training device includes:

[0077] A first-stage training module 201 is configured to perform a first-stage training on a pre-configured basic face recognition model using a sample set to obtain a first face recognition model after the first-stage training, wherein the sample set includes image samples of different qualities;

[0078] The quality score calculation module 202 is configured to input each image sample into the first face recognition model in sequence, obtain a feature vector corresponding to each image sample, and calculate a quality score corresponding to each image sample based on the feature vector using a quality score algorithm;

[0079] The second-stage training module 203 is configured to determine the probability of each image sample being within the category to which it belongs, update the gradient scaling terms of the center vector and the feature vector in the first face recognition model based on the probability and the quality score of the image sample, and perform second-stage training on the first face recognition model after the gradient scaling terms are updated using the sample set to obtain a second face recognition model after the second-stage training;

[0080] The gradient updating module 204 is configured to determine the type of each image sample based on the probability corresponding to each image sample, and when the image sample is a simple sample, set the gradient scaling items of the center vector and the feature vector when the second face recognition model is trained in the third stage using the simple sample to initial values; when the image sample is a difficult sample, update the gradient scaling items of the center vector and the feature vector when the second face recognition model is trained in the third stage using the difficult sample using a predetermined angle;

[0081] The third stage training module 205 is configured to use the sample set to perform the third stage training on the second face recognition model after the gradient scaling term is updated, to obtain the third face recognition model after the third stage training, and to use the third face recognition model as the final trained model.

[0082] In some embodiments, the basic face recognition model adopts the CosFace model. Figure 2 The first-stage training module 201 uses the image samples in the sample set as the input of the CosFace model, uses the image samples to perform the first-stage training of the CosFace model, and obtains a first face recognition model. The first face recognition model is used to automatically identify the category corresponding to the face image based on the input face image; wherein, the sample set contains image samples and category labels corresponding to the image samples.

[0083] In some embodiments, Figure 2 The quality score calculation module 202 calculates a first similarity between the feature vector of each image sample and the feature vectors of other image samples in the same category based on the feature vector corresponding to each image sample, to obtain a first similarity set; calculates a second similarity between the feature vector of the image sample and the feature vectors of multiple randomly sampled heterogeneous samples, to obtain a second similarity set; and calculates the quality score corresponding to each image sample based on the first similarity set, the second similarity set, and a preset quality score calculation formula.

[0084] In some embodiments, Figure 2 The second stage training module 203 determines the gradient of the center vector of the category to which the image sample belongs and the gradient of the feature vector corresponding to the image sample based on probability and gradient descent algorithm, uses the quality score of the image sample to weight the gradient scaling item of the gradient, obtains the quality-weighted gradient scaling item, uses the quality-weighted gradient scaling item as the updated gradient scaling item, and uses the updated gradient scaling item to update the original gradient scaling item in the first face recognition model.

[0085] In some embodiments, Figure 2The gradient update module 204 performs a threshold judgment on the probability corresponding to the image sample. When the probability corresponding to the image sample is greater than or equal to the threshold, the image sample is regarded as a simple sample. When the probability corresponding to the image sample is less than the threshold, the image sample is regarded as a difficult sample.

[0086] In some embodiments, Figure 2 When the gradient update module 204 determines that the image sample is a simple sample, when the simple sample is used to perform the third stage training on the second face recognition model, the gradient scaling items of the center vector and the eigenvector corresponding to the simple sample in the second face recognition model are set to the gradient scaling items of the center vector and the eigenvector corresponding to the first face recognition model, so that the gradient scaling items of the center vector and the eigenvector corresponding to the simple sample in the second face recognition model remain unchanged.

[0087] In some embodiments, Figure 2 When the gradient update module 204 determines that the image sample is a difficult sample, when the difficult sample is used to perform the third stage training on the second face recognition model, the cosine value of the angle between the feature vector of the difficult sample and the center vector of the sample category corresponding to the difficult sample is obtained, and the cosine value is used to perform angle weighting on the gradient scaling items corresponding to the center vector and the feature vector, so as to update the gradient scaling items of the center vector and the feature vector during the third stage training.

[0088] Figure 3 Schematic diagram of the structure of the electronic device 3 provided in the embodiment of the present application. Figure 3 As shown, the electronic device 3 of this embodiment includes: a processor 301, a memory 302, and a computer program 303 stored in the memory 302 and executable on the processor 301. When the processor 301 executes the computer program 303, the steps of the above-mentioned method embodiments are implemented. Alternatively, when the processor 301 executes the computer program 303, the functions of the modules / units in the above-mentioned device embodiments are implemented.

[0089] For example, computer program 303 may be divided into one or more modules / units, which are stored in memory 302 and executed by processor 301 to implement the present application. One or more modules / units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of computer program 303 in electronic device 3.

[0090] The electronic device 3 may be a desktop computer, a notebook, a PDA, a cloud server or other electronic device. The electronic device 3 may include but is not limited to a processor 301 and a memory 302. Those skilled in the art will understand that Figure 3It is only an example of electronic device 3 and does not constitute a limitation of electronic device 3. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0091] The processor 301 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0092] The memory 302 can be an internal storage unit of the electronic device 3, such as a hard drive or memory of the electronic device 3. The memory 302 can also be an external storage device of the electronic device 3, such as a plug-in hard drive, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. equipped on the electronic device 3. Furthermore, the memory 302 can include both an internal storage unit of the electronic device 3 and an external storage device. The memory 302 is used to store computer programs and other programs and data required by the electronic device. The memory 302 can also be used to temporarily store data that has been output or is about to be output.

[0093] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0094] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0095] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments applied for herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0096] In the embodiments provided in this application, it should be understood that the disclosed apparatus / computer equipment and methods can be implemented in other ways. For example, the apparatus / computer equipment embodiments described above are merely schematic. For example, the division of modules or units is merely a logical function division. In actual implementation, there may be other division methods. Multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection of the apparatus or unit, which may be electrical, mechanical or other forms.

[0097] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0098] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0099] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. The computer program may include computer program code, which may be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0100] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A face recognition model training method, characterized in that: include: Performing a first-stage training on a pre-configured basic face recognition model using a sample set to obtain a first face recognition model after the first-stage training, wherein the sample set includes image samples of different qualities; Inputting each of the image samples into the first face recognition model in sequence to obtain a feature vector corresponding to each of the image samples, and calculating a quality score corresponding to each of the image samples based on the feature vector using a quality score algorithm; determining a probability of each of the image samples being within the category to which it belongs, updating a gradient scaling term of a center vector and a feature vector in the first face recognition model based on the probability and a quality score of the image sample, and performing a second-stage training on the first face recognition model after the gradient scaling term is updated using the sample set to obtain a second face recognition model after the second-stage training; determining a type of each image sample based on a probability corresponding to each image sample, and when the image sample is a simple sample, setting a center vector and a gradient scaling item of a feature vector when the second face recognition model is trained in the third stage using the simple sample as initial values; and when the image sample is a difficult sample, updating a center vector and a gradient scaling item of a feature vector when the second face recognition model is trained in the third stage using the difficult sample using a predetermined angle; The second face recognition model after the gradient scaling term is updated is trained in the third stage using the sample set to obtain a third face recognition model after the third stage training, and the third face recognition model is used as the final trained model.

2. The method according to claim 1, characterized in that The basic face recognition model adopts the CosFace model, and the sample set is used to perform the first phase training on the pre-configured basic face recognition model to obtain a first face recognition model after the first phase training, including: The image samples in the sample set are used as input of the CosFace model, and the CosFace model is trained in the first stage using the image samples to obtain a first face recognition model. The first face recognition model is used to automatically identify the category corresponding to the input face image based on the input face image; wherein the sample set contains image samples and category labels corresponding to the image samples.

3. The method according to claim 1, characterized in that The calculating the quality score corresponding to each of the image samples based on the feature vector using the quality score algorithm includes: Based on the feature vector corresponding to each of the image samples, calculating a first similarity between the feature vector of the image sample and the feature vectors of other image samples in the same category to obtain a first similarity set; Calculating a second similarity between the feature vector of the image sample and the feature vectors of a plurality of randomly sampled heterogeneous samples to obtain a second similarity set; The quality score corresponding to each of the image samples is calculated based on the first similarity set, the second similarity set, and a preset quality score calculation formula.

4. The method according to claim 1, wherein The updating of the gradient scaling items of the center vector and the feature vector in the first face recognition model based on the probability and the quality score of the image sample includes: Based on the probability and gradient descent algorithm, determine the gradient of the center vector of the category to which the image sample corresponds and the gradient of the feature vector corresponding to the image sample, use the quality score of the image sample to perform quality weighting on the gradient scaling item of the gradient to obtain a quality-weighted gradient scaling item, use the quality-weighted gradient scaling item as the updated gradient scaling item, and use the updated gradient scaling item to update the original gradient scaling item in the first face recognition model.

5. The method according to claim 1, characterized in that The determining the type of each of the image samples based on the probability corresponding to each of the image samples includes: A threshold is applied to the probability corresponding to the image sample. When the probability corresponding to the image sample is greater than or equal to the threshold, the image sample is regarded as a simple sample. When the probability corresponding to the image sample is less than the threshold, the image sample is regarded as a difficult sample.

6. The method according to claim 5, characterized in that When the picture sample is a simple sample, the gradient scaling items of the center vector and the feature vector when the second face recognition model is trained in the third stage using the simple sample are set to initial values, including: When it is determined that the image sample is a simple sample, when the second face recognition model is trained for the third stage using the simple sample, the gradient scaling items of the center vector and the eigenvector corresponding to the simple sample in the second face recognition model are set to the gradient scaling items of the center vector and the eigenvector corresponding to the first face recognition model, so that the gradient scaling items of the center vector and the eigenvector corresponding to the simple sample in the second face recognition model remain unchanged.

7. The method according to claim 5, characterized in that When the image sample is a difficult sample, using a predetermined angle, updating the center vector and the gradient scaling item of the feature vector when the second face recognition model is trained in the third stage using the difficult sample, including: When the image sample is judged to be a difficult sample, when the second face recognition model is trained for the third stage using the difficult sample, the cosine value of the angle between the feature vector of the difficult sample and the center vector of the sample category corresponding to the difficult sample is obtained, and the cosine value is used to perform angle weighting on the gradient scaling items corresponding to the center vector and the feature vector, so as to update the gradient scaling items of the center vector and the feature vector during the third stage training.

8. A face recognition model training device, characterized in that: include: a first-stage training module configured to perform a first-stage training on a pre-configured basic face recognition model using a sample set, wherein the sample set includes image samples of different qualities, to obtain a first face recognition model after the first-stage training; a quality score calculation module configured to input each of the image samples into the first face recognition model in sequence, obtain a feature vector corresponding to each of the image samples, and calculate a quality score corresponding to each of the image samples based on the feature vector using a quality score algorithm; a second-stage training module configured to determine a probability of each of the image samples being within the category to which it belongs, update the gradient scaling terms of the center vector and the feature vector in the first face recognition model based on the probability and the quality score of the image sample, and perform a second-stage training on the first face recognition model after the gradient scaling terms are updated using the sample set to obtain a second face recognition model after the second-stage training; a gradient update module configured to determine the type of each image sample based on the probability corresponding to each image sample, and when the image sample is a simple sample, set the gradient scaling items of the center vector and the feature vector when the second face recognition model is trained in the third stage using the simple sample to initial values; and when the image sample is a difficult sample, update the gradient scaling items of the center vector and the feature vector when the second face recognition model is trained in the third stage using the difficult sample using a predetermined angle; The third stage training module is configured to use the sample set to perform the third stage training on the second face recognition model after the gradient scaling term is updated, to obtain the third face recognition model after the third stage training, and to use the third face recognition model as the final trained model.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 7 when executing the program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images