Face recognition model training method, recognition method, device, equipment and medium

Abstract

The application provides a face recognition model training method, a recognition method, a device, equipment and a medium. The method comprises the following steps: acquiring a plurality of first sample face images and an identity code corresponding to each first sample face image; performing a preset augmentation processing on the plurality of first sample face images to obtain a plurality of first face images, and performing an augmentation processing on the plurality of first sample face images according to a preset image augmentation model to obtain a plurality of second face images; and training a preset face recognition model according to the plurality of first face images and the plurality of second face images until the face recognition model converges. The preset face recognition model is jointly trained by the first face images and the second face images, so that the trained face recognition model is more accurate.

Description

Technical Field

[0001] The present application relates to the field of face recognition technology, and in particular to a face recognition model training method, recognition method, device, equipment and medium. Background Art

[0002] As a fundamental attribute that distinguishes individuals, faces are frequently recognized in computer vision and multimedia applications. In these applications, face recognition models need to be re-deployed in mobile phones and even smart cameras for use in a variety of areas, including camera autofocus, human-computer interaction, photo management, urban security monitoring, and intelligent driving. Currently, face recognition in open environments often requires recognizing low-resolution facial images, but the accuracy of low-resolution facial recognition is currently poor. Enhancement-based and embedding-based methods are currently used to improve the accuracy of low-resolution facial recognition, but these two processing methods are not ideal and cannot meet user requirements. Therefore, how to improve the accuracy of low-resolution facial recognition is an urgent problem that needs to be solved. Summary of the Invention

[0003] The main purpose of this application is to provide a face recognition model training method, recognition method, device, equipment and medium, aiming to make the trained face recognition model more accurate and improve the accuracy of low-resolution face image recognition.

[0004] In a first aspect, the present application provides a face recognition method, the face recognition method comprising the following steps:

[0005] Obtaining a plurality of first sample facial images and an identity identification code corresponding to each of the first sample facial images;

[0006] performing a preset augmentation process on the plurality of first sample facial images to obtain a plurality of first facial images, and performing augmentation process on the plurality of first sample facial images according to a preset image augmentation model to obtain a plurality of second facial images, wherein the first sample facial images have the same identity identification code as that corresponding to the second facial images, and the image augmentation model is used to perform image fuzzy augmentation on the first facial images;

[0007] A preset face recognition model is trained according to the multiple first face images and the multiple second face images until the face recognition model converges.

[0008] In a second aspect, the present application also provides a face recognition method, comprising:

[0009] Obtain the face image to be recognized;

[0010] Inputting the face image to be identified into a face recognition model to obtain identity features of the person corresponding to the face image to be identified, wherein the face recognition model is trained using the face recognition model training method;

[0011] The identity information of the person corresponding to the face image to be identified is determined based on the identity feature and a preset identity information database.

[0012] In a third aspect, the present application further provides a face recognition model training device, the face recognition model training device comprising a first acquisition module, a generation module and a training module, wherein:

[0013] The first acquisition module is configured to acquire a plurality of first sample facial images and an identity code corresponding to each of the first sample facial images;

[0014] The generating module is configured to perform a preset augmentation process on the plurality of first sample facial images to obtain a plurality of first facial images, and perform augmentation process on the plurality of first sample facial images according to a preset image augmentation model to obtain a plurality of second facial images, wherein the first sample facial images have the same identity identification code as the corresponding second facial images, and the image augmentation model is configured to perform image fuzzy augmentation on the first facial images;

[0015] The first training module is used to train a preset face recognition model according to the multiple first face images and the multiple second face images until the face recognition model converges.

[0016] In a fourth aspect, the present application also provides a terminal device, which includes a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, it implements the steps of the above-mentioned face recognition model training method and / or face recognition method.

[0017] In a fifth aspect, the present application also provides a computer-readable storage medium, on which a computer program is stored, wherein when the computer program is executed by a processor, the steps of the above-mentioned face recognition model training method and / or face recognition method are implemented.

[0018] The present application provides a face recognition model training method, recognition method, apparatus, device and medium. The present application obtains a plurality of first sample face images and an identity code corresponding to each first sample face image; then performs a preset augmentation process on the plurality of first sample face images to obtain a plurality of first face images, and performs augmentation process on the plurality of first sample face images according to a preset image augmentation model to obtain a plurality of second face images, wherein the first sample face images have the same identity code as the corresponding second face images, and the image augmentation model is used to perform image blur augmentation on the first face images; and trains the preset face recognition model according to the plurality of first face images and the plurality of second face images until the face recognition model converges. This solution performs augmentation process on the plurality of first sample face images by using the preset augmentation and image augmentation models, and can obtain a large number of first face images and second face images, greatly increasing the number of training samples. By jointly training the preset face recognition model on the first face images and the second face images, the trained face recognition model is made more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] Figure 1 A flowchart of a face recognition model training method provided in an embodiment of the present application;

[0021] Figure 2 A schematic diagram of a process for training an image augmentation model provided in an embodiment of the present application;

[0022] Figure 3 for Figure 2 Schematic diagram of the sub-step flow of image augmentation model training in ;

[0023] Figure 4 for Figure 1 Schematic diagram of the sub-step flow of the face recognition model training method in ;

[0024] Figure 5 A schematic diagram of the steps of the face recognition method provided in an embodiment of the present application;

[0025] Figure 6 A schematic block diagram of a face recognition model training device provided in an embodiment of the present application;

[0026] Figure 7A schematic block diagram of the submodules of the face recognition model training device provided in an embodiment of the present application;

[0027] Figure 8 A schematic block diagram of an image augmentation model training device provided in an embodiment of the present application;

[0028] Figure 9 A schematic block diagram of a face recognition device provided in an embodiment of the present application;

[0029] Figure 10 A schematic block diagram of the structure of a terminal device provided in an embodiment of the present application.

[0030] The realization of the objectives, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0032] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, combined, or partially merged, so the actual execution order may vary depending on the actual situation.

[0033] The present invention provides a face recognition model training method, recognition method, apparatus, device, and medium. The face recognition model training method can be applied to a terminal device, which can be an electronic device such as a mobile phone, tablet computer, laptop computer, desktop computer, personal digital assistant, or wearable device.

[0034] The following describes some embodiments of the present application in detail with reference to the accompanying drawings. In the absence of conflict, the following embodiments and features therein may be combined with each other.

[0035] Please refer to Figure 1 , Figure 1 A flowchart of a face recognition model training method provided in an embodiment of the present application.

[0036] like Figure 1 As shown, the face recognition model training method includes steps S101 to S103.

[0037] Step S101: Acquire a plurality of first sample facial images and an identity code corresponding to each of the first sample facial images.

[0038] Among them, the identity identification code is the identity identification corresponding to the first sample face image. The identity identification code can be set according to actual conditions, and the embodiment of the present invention does not make specific limitations on this. For example, the identity identification code can be an ID card number. The preset resolution can be set according to actual conditions, and the embodiment of the present invention does not make specific limitations on this. For example, the preset resolution is 720P.

[0039] In one embodiment, multiple first sample facial images and an identity identification code corresponding to each first sample facial image are obtained, and multiple sample facial images and an identity identification code corresponding to each first sample facial image are obtained, and the resolution of each first sample facial image is less than or equal to a preset resolution.

[0040] It should be noted that the method for obtaining the first sample facial image can be selected according to actual circumstances, and the embodiment of the present invention does not impose any specific limitation on this. For example, the first sample facial image can be an image captured from a video, or it can be an image captured by a shooting device. The shooting device can be selected according to actual circumstances, and the embodiment of the present invention does not impose any specific limitation on this. For example, the shooting device can be a camera, a video camera, a mobile phone and other devices.

[0041] Step S102: performing a preset augmentation process on the plurality of first sample facial images to obtain a plurality of first facial images, and performing an augmentation process on the plurality of first sample facial images according to a preset image augmentation model to obtain a plurality of second facial images.

[0042] Preset augmentation processing and image augmentation model processing are performed on multiple first sample face images to obtain more samples.

[0043] In one embodiment, please refer to Figure 2 , Figure 2 A schematic diagram of a process for training an image augmentation model provided in an embodiment of the present application.

[0044] like Figure 2 As shown, the image augmentation model training includes steps S201 to S202.

[0045] Step S201: Acquire multiple second sample facial images, and add noise to each of the second sample facial images to obtain multiple third sample facial images.

[0046] Acquire multiple second sample facial images, where the method for acquiring the multiple second sample facial images can be selected according to actual circumstances, and the embodiments of the present invention do not specifically limit this. For example, the second sample facial images can be images captured from a video, or images collected by a shooting device. The shooting device can be selected according to actual circumstances, and the embodiments of the present invention do not specifically limit this. For example, the shooting device can be a camera, a video camera, a mobile phone, or other devices.

[0047] In one embodiment, preset photon noise, readout noise, and quantization noise are obtained; and based on the resolution of each second sample facial image, photon noise, readout noise, and quantization noise are added to each second sample facial image to obtain a plurality of third sample facial images. The preset photon noise, readout noise, and quantization noise can be set based on actual conditions and are not specifically limited in this embodiment of the present invention. By adding noise to each second sample facial image, the sample images used to train the image augmentation model are more realistic, thereby increasing the accuracy of the trained image augmentation model.

[0048] It should be noted that photon noise is the optical noise generated by the photoelectric effect when photons are converted into electrons during image acquisition; readout noise is the inherent factors of the circuit in the process of converting electrons into voltage when acquiring images, such as the thermal motion of electrons in the device, which causes inaccurate results. The error generated is called readout noise; quantization noise is the information loss caused by converting voltage into digital signals when acquiring images, from continuous signals to digital signals. It is called quantization error or rounding error, that is, quantization noise.

[0049] In one embodiment, photon noise may also be obtained by obtaining the number of photons I received by the sensor in the captured image and fitting the number of photons I using a Poisson distribution to obtain photon noise. Fitting the number of received photons using a Poisson distribution accurately obtains photon noise.

[0050] In one embodiment, readout noise may also be obtained by obtaining errors in the process of converting electrons into voltage during image acquisition, applying Gaussian distribution to the errors, and then processing them using a preset Tukey lambda distribution to generate readout noise. By processing the errors in the process of converting electrons into voltage during image acquisition, readout noise can be accurately obtained.

[0051] In an embodiment, the manner of obtaining the quantization noise can also be: obtaining a quantization noise distribution, the quantization noise distribution being [-0.5q, 0.5q], wherein q is the number of quantization steps. The number of quantization steps is obtained, and the number of quantization steps and the quantization noise distribution are operated to obtain the quantization noise. For example, when the number of quantization steps is 1 (i.e., q is 1), the quantization noise is [-0.5, 0.5]; when the number of quantization steps is 2 (i.e., q is 2), the quantization noise is [-0.1, 0.1].

[0052] In an embodiment, the resolution of each second sample face image is obtained, and according to the resolution of each second sample face image, photon noise, readout noise and quantization noise are added to each second sample face image to obtain a plurality of third sample face images. By adding photon noise, readout noise and quantization noise to the second sample face image, a sample image more consistent with the characteristics of a low-resolution image can be obtained.

[0053] For example, a noise superposition formula N=kN1+N2+N3 is obtained, wherein N is the total noise, k is the photon noise gain value, N1 is the photon noise of each pixel point, N2 is the readout noise of each pixel point, and N3 is the quantization noise of each pixel point. The photon noise, the readout noise and the quantization noise are obtained, and the photon noise, the readout noise and the quantization noise are superimposed based on the noise superposition formula to obtain the total noise. The photon noise gain value is set according to the imaging system, and the photon noise gain value can be set according to actual conditions, which is not limited in the embodiment of the present application. According to the resolution of the second sample face image, the total noise is added to the second sample face image to generate a third sample face image.

[0054] In step S202, the preset image augmentation model is trained according to the plurality of third sample face images until the image augmentation model converges.

[0055] The image augmentation model includes an image down-sampling model and a Gaussian blur model.

[0056] In an embodiment, as shown in Figure 3 Step S202 includes sub-step S2021 to sub-step S2023.

[0057] In sub-step S2021, each third sample face image is processed by the preset image augmentation model to obtain a plurality of third face images.

[0058] Each third sample facial image is downsampled using the image downsampling model, and the downsampled images are processed using the Gaussian blur model to obtain a third facial image corresponding to each third sample facial image. Downsampling the third sample facial images using the image downsampling model can ensure that the sample images fit within the display area and generate corresponding image thumbnails. Gaussian blurring the thumbnails accurately obtains the third facial images.

[0059] Sub-step S2022: Determine whether the image augmentation model converges based on the plurality of second sample facial images and the plurality of third facial images.

[0060] Calculate the facial feature similarity between two second sample facial images that match each identity identification code to obtain at least one facial similarity corresponding to each identity identification code, and establish a first similarity histogram based on each facial similarity; calculate the facial feature similarity between two third facial images that match each identity identification code to obtain at least one facial similarity corresponding to each identity identification code, and establish a second similarity histogram based on each facial similarity; perform curve fitting on the first similarity histogram to obtain a first curve, and perform curve fitting on the second similarity histogram to obtain a second curve; determine a first area enclosed by the first curve and the coordinate axis, and a second area enclosed by the second curve and the coordinate axis; determine that the image augmentation model has converged when the area of ​​the intersection of the first area and the second area is greater than or equal to a preset area threshold; determine that the image augmentation model has converged when the area of ​​the intersection of the first area and the second area is less than the preset area threshold. The preset area threshold can be set according to actual conditions and is not specifically limited in the embodiments of the present invention. By determining the area of ​​the intersection of the first area enclosed by each second sample face image and the coordinate axis, and the second area enclosed by each third face image and the coordinate axis, it is possible to accurately know whether the image augmentation model has converged.

[0061] In one embodiment, the facial feature similarity between two second sample facial images whose identification codes match may be calculated by obtaining two second sample facial images whose identification codes match, performing a cosine distance calculation on the features of the two second sample facial images, and obtaining the similarity between the two second sample facial images. By performing a cosine distance calculation on the features of the two second sample facial images, the similarity between the two second sample facial images can be accurately obtained.

[0062] In one embodiment, the first similarity histogram may be established based on the facial similarities of each person by establishing a rectangular coordinate system with facial similarity as the horizontal axis and the number of identical facial similarities as the vertical axis, and establishing the first similarity histogram based on the facial similarities of each person and the number of identical facial similarities. By establishing the first similarity histogram for each facial similarity, the accuracy of model training can be improved.

[0063] It should be noted that the method of calculating the facial feature similarity between the two third facial images that match each identity identification code to obtain at least one facial similarity corresponding to each identity identification code can refer to the method of calculating the facial feature similarity between the two second sample facial images that match each identity identification code to obtain at least one facial similarity corresponding to each identity identification code; the method of establishing a second similarity histogram based on each facial similarity can refer to the method of establishing a first similarity histogram based on each facial similarity. Therefore, there will be no further elaboration on calculating the facial feature similarity between the two third facial images that match each identity identification code to obtain at least one facial similarity corresponding to each identity identification code, and establishing a second similarity histogram based on each facial similarity.

[0064] In one embodiment, a preset curve fitting method is obtained, and a curve fitting is performed on the first similarity histogram based on the preset curve fitting method to obtain a first curve, and a curve fitting is performed on the second similarity histogram to obtain a second curve. The preset curve fitting method can be selected according to actual conditions, and the embodiment of the present invention does not specifically limit this. For example, the preset curve fitting method can be drawn using the mlab module in matplotlib or the distplot in the seaborn library. Through this curve fitting method, the first curve corresponding to the first similarity histogram and the second curve corresponding to the second similarity histogram can be accurately obtained.

[0065] Sub-step S2023: If the image augmentation model has not converged, adjust the model parameters of the image augmentation model to update the image augmentation model, and continue training the updated image augmentation model until the image augmentation model converges.

[0066] When determining whether the area of ​​the intersection area of ​​the first area and the second area is greater than or equal to the preset area threshold, when the area of ​​the intersection area of ​​the first area and the second area is greater than or equal to the preset area threshold, it is determined that the image augmentation model has converged. When the area of ​​the intersection area of ​​the first area and the second area is less than the preset area threshold, it is determined that the image augmentation model has not converged, wherein the image augmentation model includes an image downsampling model and a Gaussian blur model, adjusting the downsampling parameters of the image downsampling model in the image augmentation model and adjusting the model parameters of the Gaussian blur model to update the image downsampling model and the Gaussian blur model, and continuing to train the updated image downsampling model and the Gaussian blur model until the image downsampling model and the Gaussian blur model converge to obtain a converged image augmentation model. When it is determined that the image augmentation model has not converged, by adjusting the model parameters of the image augmentation model and continuing to train the image augmentation model with the updated model parameters, a converged image augmentation model can be accurately obtained.

[0067] In one embodiment, the method for adjusting the downsampling parameters of the image downsampling model and the model parameters of the Gaussian blur model in the image augmentation model can be: select one parameter from a preset downsampling parameter library and a preset model parameter library as the downsampling parameter for adjusting the image downsampling model and the model parameter for adjusting the Gaussian blur model. The preset downsampling parameter library and the preset model parameter library can be set according to actual conditions, and the embodiment of the present invention does not specifically limit this. For example, the downsampling parameters included in the downsampling parameter library can be parameters such as 10 times, 20 times, and 50 times; the model parameters include Gaussian kernel parameters, and the Gaussian kernel parameters can be parameters such as 0.5, 5, and 8. By selecting downsampling parameters and model parameters from the preset downsampling parameter library and the preset model parameter library, the downsampling parameters of the image downsampling model in the image augmentation model and the model parameters of the Gaussian blur model can be accurately adjusted.

[0068] In one embodiment, a plurality of first sample facial images are subjected to a preset augmentation process to obtain a plurality of first facial images. The preset augmentation can be selected based on actual circumstances and is not specifically described in this embodiment of the present invention. The preset augmentation may include random flipping, brightness and contrast adjustment, image grayscale conversion, and random erasure. By performing the preset augmentation process on the plurality of first sample facial images, the plurality of first facial images are obtained to enrich the sample images.

[0069] In one embodiment, a plurality of first sample facial images are augmented according to a preset image augmentation model to obtain a plurality of second facial images, wherein the first sample facial images have the same identity identification code as the corresponding second facial images. The first sample facial images are augmented according to the image augmentation model to obtain the plurality of second facial images.

[0070] Step S103, training a preset face recognition model according to the plurality of first face images and the plurality of second face images until the face recognition model converges.

[0071] The face recognition model is a neural network model, and the specific type of the neural network model can be selected according to actual conditions, and embodiments of the present application do not make specific limitations thereto. For example, the neural network model can be a knowledge distillation neural network model. For example, the face recognition model can be a face recognition model based on a knowledge distillation neural network and a face recognition model based on a convolutional neural network.

[0072] For example, the structure of the face recognition model can be backbone+L2 norm, the backbone can be selected according to actual conditions, and embodiments of the present application do not make specific limitations thereto. For example, the backbone can be MobileFaceNet, iresnet and vit. For example, the structure of the face recognition model can be MobileFaceNet+L2 norm, iresnet+L2 norm and vit+L2 norm.

[0073] In an embodiment, as shown in Figure 4 Step S103 includes sub-step S1031 to sub-step S1034.

[0074] Sub-step S1031, inputting the first face image into the face recognition model for processing to obtain a first feature vector.

[0075] The first face image is input into the face recognition model for processing to obtain a first feature vector. The face recognition model can accurately obtain the first feature vector corresponding to the first face image.

[0076] Sub-step S1032, inputting the second face image into the face recognition model for processing to obtain a second feature vector.

[0077] The second face image is input into the face recognition model for processing to obtain a second feature vector. The face recognition model can accurately obtain the second feature vector corresponding to the second face image.

[0078] Sub-step S1033, determining a target loss value of the face recognition model according to the first feature vector and the second feature vector, and determining whether the face recognition model converges according to the target loss value.

[0079] A first loss value is generated based on the first eigenvector and the identity code corresponding to the first eigenvector; a second loss value is generated based on the second eigenvector and the first eigenvector; and a weighted sum of the first and second loss values ​​is performed to obtain a target loss value. By determining the first and second loss values ​​and performing a weighted sum of the first and second loss values, the target loss value of the face recognition model can be accurately obtained.

[0080] In one embodiment, the first loss value may be generated according to the first eigenvector and the identity identification code corresponding to the first eigenvector by obtaining a preset first loss value formula, which is: Among them, L1 is the first loss value, N is the number of small batch images, n is the number of identity codes of the first sample face images participating in the training, m is the angle distance, s is the multiplier of the cosine distance of the first eigenvector, is the angle between the first eigenvector and the characteristic prototype of the corresponding identity identification code. Based on the first loss value formula and according to the first eigenvector and the identity identification code corresponding to the first eigenvector, a first loss value is generated. The first loss value can be accurately calculated using the first loss value formula.

[0081] In one embodiment, the second loss value can be generated according to the second eigenvector and the first eigenvector by performing distillation learning on the second eigenvector and the first eigenvector, specifically, setting a triplet for the second eigenvector and each of the first eigenvectors, using the second eigenvector as an anchor, using the first eigenvector that has the same identity code as the second eigenvector and has the smallest feature similarity as the positive, and using the first eigenvector that has a different identity code from the second eigenvector and has the largest feature similarity as the negative, and calculating the triplet loss value based on the second eigenvector corresponding to each anchor, the first eigenvector corresponding to the positive, and the first eigenvector corresponding to the negative, to obtain the second loss value. By constructing a triplet for the second eigenvector and each of the first eigenvectors, the second loss value can be accurately calculated based on the constructed triplet and the triplet loss principle. It should be noted that the triplet loss principle is based on the Euclidean distance formalization principle.

[0082] Exemplarily, a preset second loss value formula is obtained, which is L2=max}d(a,p)-d(a,n)+m,0}, assuming that the sample is x and the mapping function is f (x) , simplify the second loss value formula, and obtain the simplified second loss value formula as Wherein, L2 is a second loss value, N is a total number of samples, n is a sample, a is an anchor, p is a positive, n is a negative, and m is a constant, wherein the constant m and the mapping function f (x) The constant m and the mapping function f can be set according to actual conditions, and embodiments of the present application do not make specific limitations thereto. Based on the second loss value formula, and according to the second characteristic vector and each first characteristic vector, a triple is constructed, and a second loss value is generated.

[0083] In an embodiment, the manner in which the first loss value and the second loss value are summed by weighting to obtain the target loss value can be: obtaining a first weight parameter and a second weight parameter, performing multiplication operation on the first weight parameter and the first loss value to obtain a third loss value, performing multiplication operation on the second weight parameter and the second loss value to obtain a fourth loss value, and performing summation on the third loss value and the fourth loss value to obtain the target loss value. The first weight parameter and the second weight parameter can be set according to actual conditions, and embodiments of the present application do not make specific limitations thereto. By summing the first loss value and the second loss value by weighting, the target loss value can be accurately obtained.

[0084] In an embodiment, after the target loss value is obtained, it is determined whether the target loss value is less than or equal to a preset threshold value. If the target loss value is less than or equal to the preset threshold value, it is determined that the face recognition model has converged. If the target loss value is greater than the preset threshold value, it is determined that the face recognition model has not converged. The preset threshold value can be set according to actual conditions, and embodiments of the present application do not make specific limitations thereto.

[0085] Sub-step S1034: If the face recognition model has not converged, the model parameters of the face recognition model are adjusted to update the face recognition model, and the updated face recognition model is continuously trained. If the face recognition model has converged, the converged face recognition model is obtained.

[0086] It is determined whether the target loss value is less than or equal to a preset threshold value. If the target loss value is less than or equal to the preset threshold value, it is determined that the face recognition model has converged. If the target loss value is greater than the preset threshold value, it is determined that the face recognition model has not converged. The model parameters of the face recognition model are adjusted to update the face recognition model, and the updated face recognition model is continuously trained. If the target loss value of the updated face recognition model is less than or equal to the preset threshold value, it is determined that the face recognition model has converged. When the face recognition model has not converged, the model parameters are updated, and the converged face recognition model can be obtained by continuous training.

[0087] The face recognition model training method provided in the above embodiment obtains a plurality of first sample face images and an identity identification code corresponding to each first sample face image; then performs a preset augmentation process on the plurality of first sample face images to obtain a plurality of first face images, and performs augmentation process on the plurality of first sample face images according to a preset image augmentation model to obtain a plurality of second face images, wherein the first sample face images have the same identity identification code as the corresponding second face images; and trains the preset face recognition model according to the plurality of first face images and the plurality of second face images until the face recognition model converges. This solution performs augmentation process on the plurality of first sample face images through the preset augmentation and image augmentation models, and can obtain a large number of first face images and second face images, greatly increasing the number of training samples. By jointly training the preset face recognition model on the first face images and the second face images, the trained face recognition model is made more accurate.

[0088] Please refer to Figure 5 , Figure 5 A flowchart illustrating the steps of the face recognition method provided in an embodiment of the present application.

[0089] like Figure 5 As shown, the face recognition method includes steps S301 to S303.

[0090] Step S301: Obtain a face image to be recognized.

[0091] A facial image to be recognized is obtained. The facial image may be a facial photo or a facial image frame in a video, which is not specifically limited in the embodiment of the present invention.

[0092] Step S302: Input the face image to be identified into a face recognition model to obtain identity features of the person corresponding to the face image to be identified.

[0093] The face recognition model is trained using the aforementioned face recognition model training method.

[0094] The face image is input into a preset face recognition model to obtain the identity features of the person corresponding to the face image. By inputting the face image into a preset face recognition model, the identity features of the person corresponding to the face image can be accurately obtained.

[0095] Step S303: Determine the identity information of the person corresponding to the face image to be identified based on the identity feature and a preset identity information database.

[0096] The preset identity information database is pre-established based on the identity information of each person, and each identity information in the identity information database maps the preset identity characteristics of each person. The preset identity information database can be established according to actual conditions, and the embodiment of the present invention does not make specific limitations on this.

[0097] In one embodiment, the similarity between the identity feature and each preset identity feature in the identity information database is calculated to obtain the similarity of each preset identity feature. The preset identity feature corresponding to the greatest similarity is selected from the similarity queue as the target identity feature, and the identity information corresponding to the target identity feature is used as the identity information of the person corresponding to the facial image to be identified. By calculating the similarity between the identity feature and each preset identity feature in the identity information database, the identity information of the person corresponding to the facial image to be identified can be accurately determined.

[0098] In one embodiment, the similarity between the identity feature and each preset identity feature in the identity information database is calculated, and the similarity between each preset identity feature of the identity feature can be obtained by obtaining a preset cosine similarity formula, which is: Wherein, L3 is the identity feature similarity, A is the identity feature, and B is the preset identity feature. The identity feature and the preset identity feature are substituted into the cosine similarity formula to obtain the similarity between the identity feature and the preset identity feature.

[0099] The facial recognition method provided in the above embodiment obtains a facial image to be recognized; then inputs the facial image into a facial recognition model to obtain the identity features of the person corresponding to the facial image. The identity information of the person corresponding to the facial image to be recognized is then determined based on the identity features and a preset identity information database. By inputting this facial image into the facial recognition model, even low-resolution images can be accurately recognized, greatly improving the accuracy of facial recognition.

[0100] Please parameter Figure 6 , Figure 6 A schematic block diagram of a face recognition model training device provided in an embodiment of the present application.

[0101] like Figure 6 As shown, the face recognition model training device 400 includes a first acquisition module 410, a generation module 420 and a first training module 430, wherein:

[0102] The first acquisition module 410 is configured to acquire a plurality of first sample facial images and an identity code corresponding to each of the first sample facial images;

[0103] The generating module 420 is configured to perform a preset augmentation process on the plurality of first sample facial images to obtain a plurality of first facial images, and perform augmentation process on the plurality of first sample facial images according to a preset image augmentation model to obtain a plurality of second facial images, wherein the first sample facial images have the same identity identification code as the corresponding second facial images, and the image augmentation model is configured to perform image fuzzy augmentation on the first facial images;

[0104] The first training module 430 is configured to train a preset face recognition model according to the plurality of first face images and the plurality of second face images until the face recognition model converges.

[0105] In one embodiment, if Figure 7 As shown, the first training module 430 further includes a first processing module 431, a second processing module 432, a first determination module 433 and an update module 434, wherein:

[0106] A first processing module 431 is configured to input the first facial image into the facial recognition model for processing to obtain a first feature vector;

[0107] A second processing module 432 is configured to input the second facial image into the facial recognition model for processing to obtain a second eigenvector;

[0108] A first determining module 433 is configured to determine a target loss value of the face recognition model based on the first eigenvector and the second eigenvector, and determine whether the face recognition model has converged based on the target loss value;

[0109] The update module 434 is used to adjust the model parameters of the face recognition model to update the face recognition model if the face recognition model has not converged, and continue to train the updated face recognition model. If the face recognition model converges, a converged face recognition model is obtained.

[0110] In one embodiment, the first determining module 433 is further configured to:

[0111] generating a first loss value according to the first feature vector and the identity identification code corresponding to the first feature vector;

[0112] generating a second loss value according to the second eigenvector and the first eigenvector;

[0113] A weighted sum is performed on the first loss value and the second loss value to obtain a target loss value.

[0114] In one embodiment, see Figure 8 , Figure 8This is a schematic block diagram of an image augmentation model training device provided in an embodiment of the present application. The image augmentation model training device 500 includes a second acquisition module 510, an addition module 520, and a second training model 530, wherein:

[0115] A second acquisition module 510 is configured to acquire a plurality of second sample face images;

[0116] an adding module 520, configured to add noise to each of the second sample facial images to obtain a plurality of third sample facial images;

[0117] The second training model 530 is used to train a preset image augmentation model according to the plurality of third sample face images until the image augmentation model converges.

[0118] In one embodiment, the adding module 520 is further configured to:

[0119] Obtain preset photon noise, readout noise and quantization noise;

[0120] According to the resolution of each of the second sample facial images, the photon noise, readout noise and quantization noise are added to each of the second sample facial images to obtain a plurality of third sample facial images.

[0121] In one embodiment, the second training module 530 is further configured to:

[0122] Processing each of the third sample facial images using a preset image augmentation model to obtain a plurality of third facial images;

[0123] determining whether the image augmentation model converges based on the plurality of second sample facial images and the plurality of third facial images;

[0124] If the image augmentation model has not converged, adjusting the model parameters of the image augmentation model to update the image augmentation model, and continuing to train the updated image augmentation model until the image augmentation model converges.

[0125] In one embodiment, the second training module 530 is further configured to:

[0126] Calculating the facial feature similarity between the two second sample face images that match each identity code to obtain at least one facial similarity corresponding to each identity code, and establishing a first similarity histogram based on each facial similarity;

[0127] Calculating the facial feature similarity between the two third facial images that match each identity identification code to obtain at least one facial similarity corresponding to each identity identification code, and establishing a second similarity histogram based on each facial similarity;

[0128] Performing curve fitting on the first similarity histogram to obtain a first curve, and performing curve fitting on the second similarity histogram to obtain a second curve;

[0129] determining a first area enclosed by the first curve and the coordinate axis and a second area enclosed by the second curve and the coordinate axis;

[0130] When the area of ​​the intersection region of the first region and the second region is greater than or equal to a preset area threshold, it is determined that the image augmentation model has converged.

[0131] It should be noted that technical personnel in the relevant field can clearly understand that for the convenience and conciseness of description, the specific working process of the above-mentioned face recognition model training device can refer to the corresponding process in the aforementioned face recognition model training method embodiment, and will not be repeated here.

[0132] See also Figure 9 , Figure 9 This is a schematic block diagram of a face recognition device provided in an embodiment of the present application. The face recognition device 600 includes a third acquisition module 610, a recognition module 620, and a second determination module 630, wherein:

[0133] The third acquisition module 610 is used to acquire a face image to be recognized;

[0134] The recognition module 620 is configured to input the face image to be recognized into a face recognition model to obtain the identity features of the person corresponding to the face image to be recognized;

[0135] The second determining module 630 is configured to determine the identity information of the person corresponding to the face image to be identified based on the identity feature and a preset identity information database.

[0136] It should be noted that those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process of the above-mentioned face recognition model training device can refer to the corresponding process in the aforementioned face recognition method embodiment, and will not be repeated here.

[0137] See also Figure 10 , Figure 10 A schematic block diagram of the structure of a terminal device provided in an embodiment of the present application.

[0138] like Figure 10 As shown, the terminal device 700 includes a processor 701 and a memory 702, and the processor 701 and the memory 702 are connected via a bus 703, such as an I2C (Inter-integrated Circuit) bus.

[0139] Specifically, the processor 701 is configured to provide calculation and control capabilities to support the operation of the entire terminal device 700. The processor 701 can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0140] Specifically, the memory 702 can be a Flash chip, a read-only memory (ROM) disk, an optical disk, a U disk or a mobile hard disk, etc.

[0141] Those skilled in the art can understand that, Figure 10 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the terminal device to which the scheme of the present application is applied. The specific terminal device 700 can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement. In one embodiment, the processor 701 is configured to run a computer program stored in the memory to perform the following steps:

[0142] obtain a plurality of first sample face images and an identity code corresponding to each of the first sample face images;

[0143] perform a preset augmentation process on the plurality of first sample face images to obtain a plurality of first face images, and perform an augmentation process on the plurality of first sample face images according to a preset image augmentation model to obtain a plurality of second face images. The identity code corresponding to the second face image is the same as that of the first sample face image. The image augmentation model is configured to perform image blur augmentation on the first face image.

[0144] According to the plurality of first face images and the plurality of second face images, a preset face recognition model is trained until the face recognition model converges.

[0145] In one embodiment, the processor 701, when implementing the training of a preset face recognition model according to the multiple first face images and the multiple second face images until the face recognition model converges, is configured to implement:

[0146] Inputting the first facial image into the face recognition model for processing to obtain a first feature vector;

[0147] Inputting the second facial image into the face recognition model for processing to obtain a second eigenvector;

[0148] determining a target loss value of the face recognition model according to the first eigenvector and the second eigenvector, and determining whether the face recognition model has converged according to the target loss value;

[0149] If the face recognition model has not converged, the model parameters of the face recognition model are adjusted to update the face recognition model, and the updated face recognition model is continued to be trained. If the face recognition model converges, a converged face recognition model is obtained.

[0150] In one embodiment, when determining the target loss value of the face recognition model based on the first eigenvector and the second eigenvector, the processor 701 is configured to implement:

[0151] generating a first loss value according to the first feature vector and the identity identification code corresponding to the first feature vector;

[0152] generating a second loss value according to the second eigenvector and the first eigenvector;

[0153] A weighted sum is performed on the first loss value and the second loss value to obtain a target loss value.

[0154] In one embodiment, before obtaining the plurality of first sample facial images and the identity code corresponding to each of the first sample facial images, the processor 701 is further configured to:

[0155] Acquire a plurality of second sample facial images, and add noise to each of the second sample facial images to obtain a plurality of third sample facial images;

[0156] The preset image augmentation model is trained according to the plurality of third sample facial images until the image augmentation model converges.

[0157] In one embodiment, when implementing the adding noise to each of the second sample facial images to obtain a plurality of third sample facial images, the processor 701 is configured to implement:

[0158] Obtain preset photon noise, readout noise and quantization noise;

[0159] According to the resolution of each of the second sample facial images, the photon noise, readout noise and quantization noise are added to each of the second sample facial images to obtain a plurality of third sample facial images.

[0160] In one embodiment, the processor 701, when implementing the training of the preset image augmentation model according to the plurality of third sample facial images until the image augmentation model converges, is configured to implement:

[0161] Processing each of the third sample facial images using a preset image augmentation model to obtain a plurality of third facial images;

[0162] determining whether the image augmentation model converges based on the plurality of second sample facial images and the plurality of third facial images;

[0163] If the image augmentation model has not converged, adjusting the model parameters of the image augmentation model to update the image augmentation model, and continuing to train the updated image augmentation model until the image augmentation model converges.

[0164] In one embodiment, when determining whether the image augmentation model has converged based on the plurality of second sample facial images and the plurality of third facial images, the processor 701 is configured to implement:

[0165] Calculating the facial feature similarity between the two second sample face images that match each identity code to obtain at least one facial similarity corresponding to each identity code, and establishing a first similarity histogram based on each facial similarity;

[0166] Calculating the facial feature similarity between the two third facial images that match each identity identification code to obtain at least one facial similarity corresponding to each identity identification code, and establishing a second similarity histogram based on each facial similarity;

[0167] Performing curve fitting on the first similarity histogram to obtain a first curve, and performing curve fitting on the second similarity histogram to obtain a second curve;

[0168] determining a first area enclosed by the first curve and the coordinate axis and a second area enclosed by the second curve and the coordinate axis;

[0169] When the area of ​​the intersection region of the first region and the second region is greater than or equal to a preset area threshold, it is determined that the image augmentation model has converged.

[0170] In one embodiment, the processor 701 is configured to implement:

[0171] Obtain the face image to be recognized;

[0172] Inputting the face image to be identified into a face recognition model to obtain identity features of the person corresponding to the face image to be identified;

[0173] The identity information of the person corresponding to the face image to be identified is determined based on the identity feature and a preset identity information database.

[0174] It should be noted that technical personnel in the relevant field can clearly understand that for the convenience and conciseness of description, the specific working process of the terminal device described above can refer to the corresponding process in the aforementioned face recognition model training method and / or face recognition method embodiment, and will not be repeated here.

[0175] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. The computer program includes program instructions. The method implemented when the program instructions are executed can refer to the various embodiments of the face recognition method of the present application.

[0176] The computer-readable storage medium may be an internal storage unit of the computer device described in the aforementioned embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may be non-volatile or volatile. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash memory card, etc., equipped on the computer device.

[0177] It should be understood that the terms used in this specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0178] It should also be understood that the term "and / or" used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system that includes a series of elements includes not only those elements, but also other elements that are not explicitly listed, or also includes elements that are inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system that includes the element.

[0179] The serial numbers of the embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments. The above description is only a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A face recognition model training method, characterized in that: include: Acquire a plurality of second sample facial images, and add noise to each of the second sample facial images to obtain a plurality of third sample facial images; Training a preset image augmentation model according to the plurality of third sample facial images until the image augmentation model converges; Obtaining a plurality of first sample facial images and an identity identification code corresponding to each of the first sample facial images; performing a preset augmentation process on the plurality of first sample facial images to obtain a plurality of first facial images, and performing augmentation process on the plurality of first sample facial images according to a preset image augmentation model to obtain a plurality of second facial images, wherein the first sample facial images have the same identity identification code as that corresponding to the second facial images, and the image augmentation model is used to perform image fuzzy augmentation on the first facial images; Training a preset face recognition model according to the plurality of first face images and the plurality of second face images until the face recognition model converges; In which, the convergence of the image augmentation model is determined by a first similarity histogram constructed based on the facial feature similarity between the two second sample facial images that match each identity identification code, and a second similarity histogram constructed based on the facial feature similarity between the two third facial images that match each identity identification code, and the third facial image is determined based on multiple third sample facial images.

2. The face recognition model training method according to claim 1, wherein: The step of training a preset face recognition model according to the plurality of first face images and the plurality of second face images until the face recognition model converges comprises: Inputting the first facial image into the face recognition model for processing to obtain a first feature vector; Inputting the second facial image into the face recognition model for processing to obtain a second feature vector; determining a target loss value of the face recognition model according to the first eigenvector and the second eigenvector, and determining whether the face recognition model has converged according to the target loss value; If the face recognition model has not converged, the model parameters of the face recognition model are adjusted to update the face recognition model, and the updated face recognition model is continued to be trained. If the face recognition model converges, a converged face recognition model is obtained.

3. The face recognition model training method according to claim 2, wherein: Determining a target loss value of the face recognition model according to the first eigenvector and the second eigenvector includes: generating a first loss value according to the first feature vector and the identity identification code corresponding to the first feature vector; generating a second loss value according to the second eigenvector and the first eigenvector; A weighted sum is performed on the first loss value and the second loss value to obtain a target loss value.

4. The face recognition model training method according to claim 1, wherein: Adding noise to each of the second sample facial images to obtain a plurality of third sample facial images includes: Obtain preset photon noise, readout noise and quantization noise; According to the resolution of each of the second sample facial images, the photon noise, readout noise and quantization noise are added to each of the second sample facial images to obtain a plurality of third sample facial images.

5. The face recognition model training method according to claim 1, wherein: The step of training a preset image augmentation model according to the plurality of third sample facial images until the image augmentation model converges includes: Processing each of the third sample facial images using a preset image augmentation model to obtain a plurality of third facial images; determining whether the image augmentation model converges based on the plurality of second sample facial images and the plurality of third facial images; If the image augmentation model has not converged, adjusting the model parameters of the image augmentation model to update the image augmentation model, and continuing to train the updated image augmentation model until the image augmentation model converges.

6. The face recognition model training method according to claim 5, wherein: The determining, based on the plurality of second sample facial images and the plurality of third facial images, whether the image augmentation model converges comprises: Calculating the facial feature similarity between the two second sample face images that match each identity code to obtain at least one facial similarity corresponding to each identity code, and establishing a first similarity histogram based on each facial similarity; Calculating the facial feature similarity between the two third facial images that match each identity identification code to obtain at least one facial similarity corresponding to each identity identification code, and establishing a second similarity histogram based on each facial similarity; Performing curve fitting on the first similarity histogram to obtain a first curve, and performing curve fitting on the second similarity histogram to obtain a second curve; determining a first area enclosed by the first curve and the coordinate axis and a second area enclosed by the second curve and the coordinate axis; When the area of ​​the intersection region of the first region and the second region is greater than or equal to a preset area threshold, it is determined that the image augmentation model has converged.

7. A face recognition method, characterized in that: include: Obtaining a face image to be recognized; Inputting the face image to be identified into a face recognition model to obtain identity features of the person corresponding to the face image to be identified, wherein the face recognition model is trained using the face recognition model training method according to any one of claims 1 to 6; The identity information of the person corresponding to the face image to be identified is determined based on the identity feature and a preset identity information database.

8. A face recognition model training device, characterized in that: The face recognition model training device includes a first acquisition module, a generation module, a first training module, a second acquisition module, an addition module and a second training model, wherein: The second acquisition module is used to acquire a plurality of second sample face images; The adding module is configured to add noise to each of the second sample facial images to obtain a plurality of third sample facial images; the second training model being configured to train a preset image augmentation model based on the plurality of third sample facial images until the image augmentation model converges, wherein the convergence of the image augmentation model is determined by a first similarity histogram constructed based on facial feature similarity between two second sample facial images that match respective identity identification codes, and a second similarity histogram constructed based on facial feature similarity between two third facial images that match respective identity identification codes, the third facial image being determined based on the plurality of third sample facial images; The first acquisition module is configured to acquire a plurality of first sample facial images and an identity identification code corresponding to each of the first sample facial images; The generating module is configured to perform a preset augmentation process on the plurality of first sample facial images to obtain a plurality of first facial images, and perform augmentation process on the plurality of first sample facial images according to a preset image augmentation model to obtain a plurality of second facial images, wherein the first sample facial images have the same identity identification code as the corresponding second facial images, and the image augmentation model is configured to perform image fuzzy augmentation on the first facial images; The first training module is used to train a preset face recognition model according to the multiple first face images and the multiple second face images until the face recognition model converges.

9. A terminal device, characterized in that: The terminal device includes a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, it implements the steps of the face recognition model training method according to any one of claims 1 to 6 and / or the face recognition method according to claim 7.

10. A storage medium, characterized in that: The storage medium stores a computer program, wherein when the computer program is executed by the processor, the steps of the face recognition model training method according to any one of claims 1 to 6 and / or the face recognition method according to claim 7 are implemented.

Images