Method, device and computer-readable storage medium for identifying forged human faces

Abstract

The present disclosure relates to a method, device and computer-readable storage medium for identifying forged faces, and relates to the field of computer technology. The method disclosed in the present disclosure includes: obtaining a small sample set of forged face images of a new category; using the small sample set of forged face images of a new category to update the parameters of a pre-trained face anti-counterfeiting model; wherein the parameters of the pre-trained face anti-counterfeiting model are determined according to multiple sets of parameters of the face anti-counterfeiting model respectively according to multiple tasks, and re-determined according to the multiple sets of parameters; the tasks include: one or more small sample sets consisting of forged face images of existing categories and real face images; inputting the face image to be identified into the updated face anti-counterfeiting model to determine whether the face image to be identified is a forged face image.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a method and device for identifying a forged face, and a computer-readable storage medium. Background Art

[0002] Face anti-counterfeiting recognition refers to the effective identification of forged facial features. For example, in a face recognition system, someone may create some forged faces (such as videos, photos, and masks, etc.) to attack the face recognition system. Face anti-counterfeiting recognition is to identify forged facial features. Generally, in a face recognition system, users need to pass the face anti-counterfeiting recognition test, which can also be called liveness detection.

[0003] A method for identifying forged faces known to the inventors is to perform deep learning training on a classification model based on a large number of forged face images of known categories. The trained classification model can identify forged faces that belong to the same category as the forged face images used for training. Summary of the invention

[0004] The inventors found that existing forged face recognition methods all require a large number of samples to train the model, and only the trained model can accurately recognize forged faces. When a new type of forged face image appears, the number of forged face image samples is very small, and the model cannot be retrained, resulting in the model being unable to recognize the new type of forged face in time, and unable to effectively prevent the attack of the new type of forged face in time.

[0005] A technical problem to be solved by the present disclosure is: how to achieve timely and effective recognition of new categories of forged facial images when a small number of new categories of forged facial images appear, thereby improving the security of the facial recognition system.

[0006] According to some embodiments of the present disclosure, a method for identifying a forged face is provided, comprising: obtaining a small sample set of forged face images of a new category; updating the parameters of a pre-trained face anti-counterfeiting model using the small sample set of forged face images of the new category; wherein the parameters of the pre-trained face anti-counterfeiting model are determined by respectively determining multiple sets of parameters of the face anti-counterfeiting model according to multiple tasks, and re-determined based on the multiple sets of parameters; the tasks include: one or more small sample sets consisting of forged face images of existing categories and real face images; inputting the face image to be identified into the updated face anti-counterfeiting model to determine whether the face image to be identified is a forged face image.

[0007] In some embodiments, updating the parameters of a face anti-counterfeiting model using a small sample set of forged face images of a new category includes: when the face anti-counterfeiting model is a classification model, updating the parameters of the face anti-counterfeiting model according to the forged face images in the small sample set of the new category and the corresponding fine classification labels; or, when the face anti-counterfeiting model is a regression model, updating the parameters of the face anti-counterfeiting model according to the forged face images in the small sample set of the new category and the corresponding coarse classification labels; wherein the fine classification labels are used to mark the categories of forged face images, and the coarse classification labels are used to mark the forged face images as positive samples or negative samples.

[0008] In some embodiments, when the face anti-counterfeiting model is a classification model, updating the parameters of the face anti-counterfeiting model according to the forged face images in a small sample set of new categories and the corresponding fine classification labels includes: when the face anti-counterfeiting model is a classification model, inputting the forged face images in a small sample set of new categories into the face anti-counterfeiting model to obtain the probability that each forged face image belongs to a different category, determining a first gradient according to the probability that each forged face image belongs to a different category and the corresponding fine classification labels, and updating the parameters of the face anti-counterfeiting model according to the first gradient; or, when the face anti-counterfeiting model is a regression model, updating the parameters of the face anti-counterfeiting model according to the forged face images in a small sample set of new categories and the corresponding coarse classification labels includes: when the face anti-counterfeiting model is a regression model, inputting the forged face images in a small sample set of new categories into the face anti-counterfeiting model to obtain the output values ​​corresponding to each forged face image, determining a second gradient according to the output values ​​corresponding to each forged face image and the corresponding coarse classification labels, and updating the parameters of the face anti-counterfeiting model according to the second gradient.

[0009] In some embodiments, the method also includes: dividing the existing categories of forged face images and real face images into multiple tasks, wherein each task includes a real face image category and a first preset number of existing categories; each category includes a small sample set consisting of a second preset number of supporting sample images and a third preset number of query sample images; each time selecting the fourth preset number of tasks into a batch, inputting the batch into a face anti-counterfeiting model, determining multiple sets of parameters of the face anti-counterfeiting model according to the multiple tasks of the batch, and re-determining the parameters of the face anti-counterfeiting model according to the multiple sets of parameters; until the face anti-counterfeiting model reaches the convergence condition, the pre-training of the face anti-counterfeiting model is completed.

[0010] In some embodiments, multiple groups of parameters of the face anti-counterfeiting model are determined respectively according to multiple tasks of the batch, and the parameters of the face anti-counterfeiting model are re-determined according to the multiple groups of parameters, including: for each task in the batch, a third gradient corresponding to the task is determined according to the supporting sample images and the corresponding labels, and a group of parameters of the face anti-counterfeiting model is determined according to the third gradient; multiple groups of parameters are determined respectively according to the multiple tasks in the batch, the query sample images of the multiple tasks and the labels corresponding to the query sample images, and the fourth gradient corresponding to the batch is determined; the parameters of the face anti-counterfeiting model are re-determined according to the fourth gradient corresponding to the batch.

[0011] In some embodiments, for each task in the batch, determining the third gradient corresponding to the task based on the supporting sample images and the corresponding labels includes: when the face anti-counterfeiting model is a classification model, for each task in the batch, determining the third gradient corresponding to the task based on the supporting sample images and the corresponding fine classification labels; or, when the face anti-counterfeiting model is a regression model, for each task in the batch, determining the third gradient corresponding to the task based on the supporting sample images and the corresponding coarse classification labels; wherein the fine classification labels are used to mark the categories of forged face images, and the coarse classification labels are used to mark the forged face images as positive samples or negative samples.

[0012] In some embodiments, the initial parameters of the face anti-counterfeiting model before pre-training are determined according to a deep learning training method.

[0013] In some embodiments, the existing categories include: at least one of photo category, video category, and mask category; the existing categories of fake facial images include: existing categories of fake facial images in at least one of depth modality, near infrared modality, and red, green, and blue (RGB) modality; the real facial images include: real facial images in at least one of depth modality, near infrared modality, and red, green, and blue (RGB) modality; the modality of the fake facial images of the new category is different from the modalities of the fake facial images and real facial images of the existing categories.

[0014] According to some other embodiments of the present disclosure, a device for identifying a fake face is provided, including: a small sample acquisition module, used to obtain a small sample set of fake face images of a new category; a model adjustment module, used to update the parameters of a pre-trained face anti-counterfeiting model using the small sample set of fake face images of the new category; wherein the parameters of the pre-trained face anti-counterfeiting model are determined by respectively determining multiple groups of parameters of the face anti-counterfeiting model according to multiple tasks, and re-determined based on the multiple groups of parameters; the tasks include: one or more small sample sets consisting of fake face images of existing categories and real face images; a face recognition module, used to input the face image to be identified into the updated face anti-counterfeiting model, and determine whether the face image to be identified is a fake face image.

[0015] In some embodiments, the model adjustment module is used to update the parameters of the face anti-counterfeiting model according to the forged face images in a small sample set of new categories and the corresponding fine classification labels when the face anti-counterfeiting model is a classification model; or, when the face anti-counterfeiting model is a regression model, update the parameters of the face anti-counterfeiting model according to the forged face images in a small sample set of new categories and the corresponding coarse classification labels; wherein the fine classification labels are used to mark the categories of forged face images, and the coarse classification labels are used to mark forged face images as positive samples or negative samples.

[0016] In some embodiments, the model adjustment module is used to, when the face anti-counterfeiting model is a classification model, input the forged face images in a small sample set of the new category into the face anti-counterfeiting model to obtain the probability that each forged face image belongs to different categories, determine a first gradient based on the probability that each forged face image belongs to different categories and the corresponding detailed classification labels, and update the parameters of the face anti-counterfeiting model based on the first gradient; or, when the face anti-counterfeiting model is a regression model, input the forged face images in a small sample set of the new category into the face anti-counterfeiting model to obtain the output values ​​corresponding to each forged face image, determine a second gradient based on the output values ​​corresponding to each forged face image and the corresponding coarse classification labels, and update the parameters of the face anti-counterfeiting model based on the second gradient.

[0017] In some embodiments, the device also includes: a pre-training module, used to divide existing categories of forged face images and real face images into multiple tasks, wherein each task includes a real face image category and a first preset number of existing categories; each category includes a small sample set consisting of a second preset number of supporting sample images and a third preset number of query sample images; each time the fourth preset number of tasks are selected into a batch, the batch is input into the face anti-counterfeiting model, multiple sets of parameters of the face anti-counterfeiting model are determined according to the multiple tasks of the batch, and the parameters of the face anti-counterfeiting model are re-determined according to the multiple sets of parameters; until the face anti-counterfeiting model reaches the convergence condition, the pre-training of the face anti-counterfeiting model is completed.

[0018] In some embodiments, the pre-training module is used to determine, for each task in a batch, a third gradient corresponding to the task based on the supporting sample images and the corresponding labels, and determine a set of parameters of the face anti-counterfeiting model based on the third gradient; determine multiple sets of parameters respectively according to multiple tasks in the batch, the query sample images of the multiple tasks and the labels corresponding to the query sample images, and determine the fourth gradient corresponding to the batch; and re-determine the parameters of the face anti-counterfeiting model based on the fourth gradient corresponding to the batch.

[0019] In some embodiments, the pre-training module is used to determine, for each task in the batch, a third gradient corresponding to the task based on the supporting sample images and the corresponding fine classification labels when the face anti-counterfeiting model is a classification model; or, when the face anti-counterfeiting model is a regression model, for each task in the batch, a third gradient corresponding to the task based on the supporting sample images and the corresponding coarse classification labels; wherein the fine classification labels are used to mark the categories of forged face images, and the coarse classification labels are used to mark forged face images as positive samples or negative samples.

[0020] In some embodiments, the initial parameters of the face anti-counterfeiting model before pre-training are determined according to a deep learning training method.

[0021] In some embodiments, the existing categories include: at least one of photo category, video category, and mask category; the existing categories of fake facial images include: existing categories of fake facial images in at least one of depth modality, near infrared modality, and red, green, and blue (RGB) modality; the real facial images include: real facial images in at least one of depth modality, near infrared modality, and red, green, and blue (RGB) modality; the modality of the fake facial images of the new category is different from the modalities of the fake facial images and real facial images of the existing categories.

[0022] According to some further embodiments of the present disclosure, a device for identifying a forged face is provided, comprising: a memory; and a processor coupled to the memory, wherein the processor is configured to execute the steps of the method for identifying a forged face as described in any of the aforementioned embodiments based on instructions stored in the memory.

[0023] According to some further embodiments of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, wherein when the program is executed by a processor, the steps of the method for identifying a forged face of any of the aforementioned embodiments are implemented.

[0024] In the present disclosure, a small sample set of forged face images of a new category is directly used to update the parameters of the pre-trained face anti-counterfeiting model, and the updated face anti-counterfeiting model can identify multiple forged face images of existing categories and new categories. This is because the pre-trained face anti-counterfeiting model is trained using one or more small sample sets consisting of forged face images of existing categories and real face images, and multiple sets of parameters of the face anti-counterfeiting model are determined by multiple tasks during the training process, and then the parameters of the face anti-counterfeiting model are re-determined based on the multiple sets of parameters. Such a training process includes two stages, one is to use multiple small sample sets of forged face images of existing categories and real face images to train the face anti-counterfeiting model respectively, which is the stage of the model learning the features of the small sample set, or the stage of learning to distinguish forged faces from real faces; the second is to summarize and re-update the parameters of the face anti-counterfeiting model based on multiple training results, which is the stage of the model learning how to learn the features of the small sample set, or the stage of learning how to distinguish forged faces from real faces.

[0025] Therefore, after learning multiple tasks of a small sample set of forged face images and real face images of existing categories, the face anti-counterfeiting model has mastered how to learn a small sample set, or how to learn to distinguish forged faces from real faces or how to learn to find forgery clues, and obtained good initialization parameters. When a new category of forged face images is a small sample set, the face anti-counterfeiting model is updated. Thanks to the fast learning ability learned previously, the face anti-counterfeiting model can also converge quickly, quickly learn the new category of forged face features, and update the face anti-counterfeiting model in a timely and effective manner, so as to timely and effectively identify new categories of forged face images and timely and effectively defend against attacks from new categories of forged faces.

[0026] Other features and advantages of the present disclosure will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0028] Figure 1 A schematic flow chart showing a method for identifying a forged face according to some embodiments of the present disclosure.

[0029] Figure 2 A schematic flow chart showing a method for identifying a forged face according to other embodiments of the present disclosure.

[0030] Figure 3 A schematic diagram showing the structure of a forged face recognition device according to some embodiments of the present disclosure.

[0031] Figure 4 A schematic diagram showing the structure of a forged face recognition device according to some other embodiments of the present disclosure.

[0032] Figure 5 A schematic diagram showing the structure of a forged face recognition device according to some other embodiments of the present disclosure. DETAILED DESCRIPTION

[0033] The technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present disclosure and its application or use. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0034] This solution is proposed to solve the problem that when a new type of fake face image appears and the number of samples is very small, the model cannot effectively identify the new type of fake face in a timely manner and cannot effectively prevent the attack of the new type of fake face in a timely manner. Figure 1 Some embodiments of the method for identifying forged faces disclosed in the present invention are described.

[0035] Figure 1 Flowcharts of some embodiments of the forged face recognition method disclosed in the present invention. Figure 1 As shown, the method of this embodiment includes: steps S102 to S106.

[0036] In step S102, a small sample set of forged face images of a new category is obtained.

[0037] In the present disclosure, the number of images in the small sample set is less than a threshold value, and the threshold value is, for example, 10. This threshold value makes it impossible to converge the model by updating the face anti-counterfeiting model using a small sample set according to the deep learning training method, and the new category of forged face images is different from the category of forged face images used when pre-training the face anti-counterfeiting model. The category of forged face images used when pre-training the face anti-counterfeiting model can be called an existing category. For example, the existing categories include: at least one of: photo category, video category, and mask category. The new category can be a forged face of other categories.

[0038] In step 104, the parameters of the pre-trained face anti-counterfeiting model are updated using a small sample set of forged face images of a new category. The parameters of the pre-trained face anti-counterfeiting model are determined by determining multiple sets of parameters of the face anti-counterfeiting model according to multiple tasks, and are re-determined according to the multiple sets of parameters; the tasks include: one or more small sample sets consisting of forged face images of existing categories and real face images.

[0039] The forged face images in the small sample set of the new category can be input into the pre-trained face anti-counterfeiting model, and the gradient of the parameters of the face anti-counterfeiting model can be determined according to the output results, the labels corresponding to the forged face images and the preset loss function, and the parameters of the pre-trained face anti-counterfeiting model can be updated according to the gradient. In the case where the face anti-counterfeiting model is a classification model, the preset loss function can adopt the MSE (mean square error) loss function. In the case where the face anti-counterfeiting model is a regression model, the preset loss function can adopt the cross entropy loss function, which is not limited to the examples given. The method for updating the parameters of the pre-trained face anti-counterfeiting model according to the gradient can select the existing gradient descent method according to the needs, which will not be repeated here.

[0040] A task can be constructed based on a small sample set of forged face images of a new category, and the forged face images in the small sample set of the new category are used as support sample images for the task. This is similar to the task setting form in the pre-training method of the face forgery model to be described later, which will be described in detail later. The task can include one or more different new categories, and each new category has a corresponding small sample set. The task can also include a small sample set of real face images, etc.

[0041] In some embodiments, the face anti-counterfeiting model can adopt a classification model, and the parameters of the face anti-counterfeiting model can be updated according to the forged face images in the small sample set of the new category and the corresponding sub-category labels. The sub-category labels are used to mark the category of the forged face images. For example, the sub-category label of the real face image is 0, the sub-category label of the mask-type forged face image is 1, and the sub-category label of the new category is 2...

[0042] In the process of updating the parameters of the face anti-counterfeiting model, fine classification labels can be used to calculate the loss function value, gradient value, etc., which can improve the accuracy of the model. When the face anti-counterfeiting model classifies the image and outputs the result, it is only necessary to divide the image into two categories: real face image and forged face image. In the test process after pre-training the face anti-counterfeiting model, the coarse classification label can be used to test the accuracy of the face anti-counterfeiting model in identifying whether the image is forged or real. The coarse classification label is used to mark the forged face image as a positive sample or a negative sample. For example, the coarse classification label representing the forged face image is 0, and the coarse classification label representing the real face image is 1.

[0043] Furthermore, in the case that the face anti-counterfeiting model is a classification model, the forged face images in a small sample set of the new category are input into the face anti-counterfeiting model to obtain the probability that each forged face image belongs to different categories, and the first gradient is determined according to the probability that each forged face image belongs to different categories and the corresponding detailed classification labels, and the parameters of the face anti-counterfeiting model are updated according to the first gradient.

[0044] In some embodiments, the face anti-counterfeiting model can adopt a regression model, and the parameters of the face anti-counterfeiting model can be updated according to the forged face images in the small sample set of the new category and the corresponding coarse classification labels. The regression model will finally output a numerical value for the input image, and a corresponding threshold can be set. If the numerical value is higher than the threshold, it means that the image is a real face image, and if it is lower than the threshold, it means that the image is a forged face image. Therefore, the coarse classification label of the image can be used in the regression model.

[0045] Furthermore, when the face anti-counterfeiting model is a regression model, the forged face images in the small sample set of the new category are input into the face anti-counterfeiting model to obtain the output value corresponding to each forged face image, and the second gradient is determined according to the output value corresponding to each forged face image and the corresponding coarse classification label, and the parameters of the face anti-counterfeiting model are updated according to the second gradient.

[0046] The parameters of the pre-trained face anti-counterfeiting model are trained using the following method: multiple sets of parameters of the face anti-counterfeiting model are determined according to multiple tasks, and then the parameters of the face anti-counterfeiting model are re-determined according to the multiple sets of parameters. The tasks include: one or more small sample sets consisting of forged face images and real face images of existing categories. That is, small sample sets consisting of forged face images and real face images of existing categories are assigned to multiple tasks, and different small sample sets are used to train the face anti-counterfeiting model, but after training, the parameters of the face anti-counterfeiting model will be re-determined according to the results of multiple trainings. The process of training the face anti-counterfeiting model with different small sample sets is the process of model learning, and the process of re-determining parameters according to multiple training results is the process of the model "learning how to learn".

[0047] After two processes, the model not only learns the features of a small sample set, but also learns how to learn the features of a small sample set. In other words, the face anti-counterfeiting model finally learns "how to learn to distinguish fake faces from real faces" and "how to learn to find forgery clues", instead of just learning how to distinguish fake faces from real faces, or just learning forgery clues. Therefore, when a new category of fake face images is a small sample set, the face forgery model can also converge quickly and quickly learn the features of the new category of fake faces to provide effective and timely defense.

[0048] In step S106, the facial image to be identified is input into the updated facial anti-counterfeiting model to determine whether the facial image to be identified is a forged facial image.

[0049] The face image to be identified can be a new category or a forged face image of a known category, and can be identified by the face anti-counterfeiting model.

[0050] The method of the above embodiment directly uses a small sample set of forged face images of a new category to update the parameters of the pre-trained face anti-counterfeiting model, and the updated face anti-counterfeiting model can identify multiple forged face images of existing categories and new categories. This is because the pre-trained face anti-counterfeiting model is trained using one or more small sample sets consisting of forged face images of existing categories and real face images, and multiple tasks determine multiple sets of parameters of the face anti-counterfeiting model during the training process, and then the parameters of the face anti-counterfeiting model are re-determined based on the multiple sets of parameters. Such a training process includes two stages, one is to use multiple small sample sets of forged face images of existing categories and real face images to train the face anti-counterfeiting model respectively, which is the stage of the model learning the features of the small sample set, or the stage of learning to distinguish forged faces from real faces; the second is to summarize and re-update the parameters of the face anti-counterfeiting model based on multiple training results, which is the stage of the model learning how to learn the features of the small sample set, or the stage of learning how to distinguish forged faces from real faces.

[0051] Therefore, after learning multiple tasks of a small sample set of forged face images and real face images of existing categories, the face anti-counterfeiting model has mastered how to learn a small sample set, or how to learn to distinguish forged faces from real faces or how to learn to find forgery clues, and obtained good initialization parameters. When a new category of forged face images is a small sample set, the face anti-counterfeiting model is updated. Thanks to the fast learning ability learned previously, the face anti-counterfeiting model can also converge quickly, quickly learn the new category of forged face features, and update the face anti-counterfeiting model in a timely and effective manner, so as to timely and effectively identify new categories of forged face images and timely and effectively defend against attacks from new categories of forged faces.

[0052] Combine the following Figure 2 Other embodiments of the forged face recognition method disclosed in the present invention are described, including a pre-training method and a use method of a face anti-forgery model.

[0053] Figure 2 Flowcharts of other embodiments of the forged face recognition method disclosed in the present invention. Figure 2 As shown, the method of this embodiment includes: steps S202 to S214.

[0054] In step S202, the face anti-counterfeiting model is initially trained using the forged face images and real face images of existing categories according to the deep learning training method to determine the initial parameters of the face anti-counterfeiting model before pre-training.

[0055] First, the existing deep learning training method is used to perform initial training on the face anti-counterfeiting model, and then a subsequent pre-training process is performed, so that the face anti-counterfeiting model has both good discrimination and good generalization after double training. Because the deep learning training method enables the face anti-counterfeiting model to discriminate images more accurately, and the pre-training method in the present disclosure can generalize the model to different categories of forged face images. After two trainings, the face anti-counterfeiting model can not only quickly adapt to new tasks, but also has good discrimination capabilities for known tasks or similar tasks.

[0056] In step S204, the existing categories of forged face images and real face images are divided into multiple tasks, wherein each task includes a real face image category and a first preset number of existing categories; each category includes a small sample set consisting of a second preset number of support sample images and a third preset number of query sample images.

[0057] A task (Task) includes multiple categories (also called paths (Way)), and the multiple categories include both real face image categories and one or more existing categories. Each category includes a second preset number of support (Support) sample images and a third preset number of query (Query) sample images. The support sample images form a small sample set, that is, the support sample images of each category in each task are less than a threshold, and the threshold is, for example, 10. The number of images in the small sample set formed by the support sample images during the pre-training process can be the same as or different from the number of images in the small sample set of the new category. In the case that the number of images in the small sample set formed by the support sample images during the pre-training process is less than or equal to the number of images in the small sample set of the new category, the recognition effect of the model will be better after the face anti-counterfeiting model is subsequently updated according to the small sample set of the new category. This is because the face anti-counterfeiting model has learned how to learn a smaller number of samples during the pre-training process. When the number of images of the new category is more during the update, the parameter update of the model can be better completed.

[0058] Each time, a first preset number of existing categories can be randomly selected, and then a second preset number of support sample images and a third preset number of query sample images are randomly extracted from the fake face images and real face images of the selected existing categories to generate a task. The images in different tasks cannot be exactly the same.

[0059] In step S206, a fourth preset number of tasks are selected into a batch, the batch is input into the face anti-counterfeiting model, multiple groups of parameters of the face anti-counterfeiting model are determined according to the multiple tasks in the batch, and the parameters of the face anti-counterfeiting model are re-determined according to the multiple groups of parameters.

[0060] A fourth preset number of tasks can be randomly selected into a batch (Batch), for example, 5 tasks are selected as a batch. For each batch, the parameters of the face anti-counterfeiting model will be updated twice. The first update is to update the parameters of the face anti-counterfeiting model respectively using the supporting sample images in each task. The second update is to update the parameters of the face anti-counterfeiting model using the query sample set in each task. That is the learning process mentioned above, and the process of "learning how to learn". Unlike existing deep learning methods, each batch is directly input into the model for a parameter update.

[0061] In some embodiments, for each task in the batch, the third gradient corresponding to the task is determined based on the supporting sample images and the corresponding labels (first gradient update), and a set of parameters of the face anti-counterfeiting model is determined based on the third gradient. The multiple sets of parameters determined by multiple tasks are intermediate parameters, and the parameters of the face anti-counterfeiting model will not be actually updated, but will be stored for the next update. Further, multiple sets of parameters are determined respectively according to the multiple tasks in the batch, the query sample images of the multiple tasks and the labels corresponding to the query sample images, and the fourth gradient corresponding to the batch is determined (second gradient update). According to the fourth gradient corresponding to the batch, the parameters of the face anti-counterfeiting model are re-determined. The parameters of the face anti-counterfeiting model determined for the second time are used to update the parameters of the face anti-counterfeiting model.

[0062] For example, multiple tasks τ in a batch i ∈T, i is a positive integer representing the task number, and T is a set of tasks in a batch. For each task τ i , calculate the third gradient of the parameters of the face anti-counterfeiting model corresponding to the task θ represents the current parameters of the face anti-counterfeiting model, Represents the task τ i Using loss functions that support sample image calculations Gradient relative to the current parameter θ. For task τ i The parameters of the determined face anti-counterfeiting model can be expressed as α represents the first step length. Multiple tasks in a batch can determine multiple θ′ i (can be a weight parameter matrix), multiple θ′ i is stored.

[0063] Furthermore, the fourth gradient corresponding to a batch can be expressed as It means that the losses of multiple tasks in the batch are summed first, and then the gradient is calculated. The loss and the fourth gradient are calculated based on the task τ i The query sample image and the corresponding θ′ i Calculated. Update the parameters of the face anti-counterfeiting model for this batch β is the second step length. It can be seen that θ′ i It is an intermediate variable used to calculate the fourth gradient, and the parameters of the model are updated in each batch based on the initial θ of the batch. Using the query sample image to update the face anti-counterfeiting model can enhance the generalization ability of the model in the task and avoid overfitting the support sample image.

[0064] When the face anti-counterfeiting model is a classification model, for each task in the batch, the third gradient corresponding to the task is determined according to the supporting sample images and the corresponding fine classification labels. Alternatively, when the face anti-counterfeiting model is a regression model, for each task in the batch, the third gradient corresponding to the task is determined according to the supporting sample images and the corresponding coarse classification labels.

[0065] In step S208, it is determined whether the face anti-counterfeiting model has reached the convergence condition. If so, the pre-training of the face anti-counterfeiting model is completed. Otherwise, the process returns to step S206 and starts again.

[0066] The convergence condition is, for example, that the loss function value reaches a minimum or reaches a preset loss function value, etc. If the loss function value does not meet the convergence condition, the process may return to step S206 and select another batch to train the face anti-counterfeiting model.

[0067] The above steps belong to the pre-training process of the face anti-counterfeiting model. It should be noted that although the number of supporting sample images of each category in each task is small and belongs to a small sample set, a large number of samples are used in the overall pre-training process (including a large number of sample images of existing categories and real face categories), and the more samples the better, but many tasks and many small sample sets are divided so that the model can "learn how to learn" in the pre-training process and improve the accuracy of the model.

[0068] In step S210, a small sample set of forged face images of a new category is obtained.

[0069] A small sample set of fake face images of a new category can be used as support sample images in a task. The task can contain one or more new categories, each of which corresponds to a small sample set.

[0070] In step S212, the parameters of the pre-trained face anti-counterfeiting model are updated using a small sample set of forged face images of a new category.

[0071] The method for updating the parameters of the pre-trained face anti-counterfeiting model using a small sample set of forged face images of a new category is similar to the method for updating the parameters of the pre-trained face anti-counterfeiting model using a task τ in step S206. i The supporting sample images determine the parameters θ′ of the face anti-counterfeiting model i The process is similar. For example, a small sample set of fake face images of a new category is used as the task τ j The support sample set of this task is calculated j The third gradient of the parameters of the corresponding face anti-counterfeiting model For the task τ j The parameters of the determined face anti-counterfeiting model can be expressed as α represents the first step length. That is, the parameter of the updated face anti-counterfeiting model is θ′ j The updating process using the query sample image is no longer performed.

[0072] In step S214, the facial image to be identified is input into the updated facial anti-counterfeiting model to determine whether the facial image to be identified is a forged facial image.

[0073] The disclosed solution can be used not only to identify forged face images of new categories, but also to identify forged face images of new modalities. When pre-training a face anti-counterfeiting model, the task can include forged face images and real face images of existing modalities, such as depth modality, near infrared (NIR) modality, and red, green, and blue (RGB) modality.

[0074] In some embodiments, a small sample set of forged face images of a new modality is obtained. The parameters of the pre-trained face anti-counterfeiting model are updated using the small sample set of forged face images of the new modality. The parameters of the pre-trained face anti-counterfeiting model are determined by determining multiple sets of parameters of the face anti-counterfeiting model according to multiple tasks, and re-determined according to the multiple sets of parameters; the tasks include: one or more small sample sets consisting of forged face images of existing modalities and real face images of existing modalities; the face image to be identified is input into the updated face anti-counterfeiting model to determine whether the face image to be identified is a forged face image.

[0075] The forged face images of the new modality may be of the same or different categories as the forged face images of the existing modalities, for example, forged face images of the new modality and the new category. The pre-training process is similar to that in the aforementioned embodiment. During the pre-training process, a task may include: a category of real face images and a first preset number of existing categories; each category includes a small sample set consisting of a second preset number of supporting sample images and a third preset number of query sample images. Real face images and forged face images may belong to a fifth preset number of modalities. For example, there are three known modalities, three known categories and real category face images, then a generated task may include up to twelve small sample sets. Of course, a task may also include a real face category and an existing category, each category corresponds to images of multiple existing modalities, and the new small sample set contains images of the same existing categories as the small sample set in the pre-training process but belonging to the new modality.

[0076] In the above embodiment, the pre-trained face anti-counterfeiting model is trained using one or more small sample sets consisting of forged face images and real face images of existing modalities, and multiple tasks determine multiple sets of parameters of the face anti-counterfeiting model during the training process, and then the parameters of the face anti-counterfeiting model are re-determined based on the multiple sets of parameters. Such a training process includes two stages. The first is to train the face anti-counterfeiting model using multiple small sample sets of forged face images and real face images of existing categories. This is the stage where the model learns the characteristics of the small sample sets, or the stage where it learns to distinguish forged faces of different modalities from real faces; the second is to re-update the parameters of the face anti-counterfeiting model based on the summary of multiple training results. This is the stage where the model learns how to learn the characteristics of the small sample sets, or the stage where it learns how to distinguish forged faces of different modalities from real faces.

[0077] Therefore, after pre-training, the face anti-counterfeiting model has mastered how to learn a small sample set, or how to learn to distinguish between fake faces in different modalities and real faces, and has obtained good initialization parameters. When the fake face image of the new modality is a small sample set, the face anti-counterfeiting model is updated. Thanks to the fast learning ability learned before, the face anti-counterfeiting model can also converge quickly, quickly learn the fake face features of the new modality, and update the face anti-counterfeiting model in a timely and effective manner, so as to timely and effectively identify the fake face images of the new modality, and timely and effectively defend against the attacks of fake faces of the new modality.

[0078] The present disclosure also provides a forged face recognition device, which is combined with Figure 3 Give a description.

[0079] Figure 3 FIG. 1 is a structural diagram of some embodiments of the forged face recognition device disclosed in the present invention. Figure 3As shown, the device 30 of this embodiment includes: a small sample acquisition module 302, a model adjustment module 304, and a face recognition module 306.

[0080] The small sample acquisition module 302 is used to acquire a small sample set of forged face images of a new category.

[0081] The model adjustment module 304 is used to update the parameters of the pre-trained face anti-counterfeiting model using a small sample set of new categories of forged face images; wherein the parameters of the pre-trained face anti-counterfeiting model are determined by multiple sets of parameters of the face anti-counterfeiting model according to multiple tasks, and re-determined based on the multiple sets of parameters; the tasks include: one or more small sample sets consisting of forged face images of existing categories and real face images.

[0082] In some embodiments, the model adjustment module 304 is used to update the parameters of the face anti-counterfeiting model according to the forged face images in the small sample set of the new category and the corresponding fine classification labels when the face anti-counterfeiting model is a classification model; or, when the face anti-counterfeiting model is a regression model, update the parameters of the face anti-counterfeiting model according to the forged face images in the small sample set of the new category and the corresponding coarse classification labels. The fine classification labels are used to mark the categories of the forged face images, and the coarse classification labels are used to mark the forged face images as positive samples or negative samples.

[0083] In some embodiments, the model adjustment module 304 is used to input the forged face images in a small sample set of the new category into the face anti-counterfeiting model when the face anti-counterfeiting model is a classification model, obtain the probability that each forged face image belongs to different categories, determine a first gradient according to the probability that each forged face image belongs to different categories and the corresponding detailed classification labels, and update the parameters of the face anti-counterfeiting model according to the first gradient; or, when the face anti-counterfeiting model is a regression model, input the forged face images in a small sample set of the new category into the face anti-counterfeiting model, obtain the output values ​​corresponding to each forged face image, determine a second gradient according to the output values ​​corresponding to each forged face image and the corresponding coarse classification labels, and update the parameters of the face anti-counterfeiting model according to the second gradient.

[0084] The face recognition module 306 is used to input the face image to be recognized into the updated face anti-counterfeiting model to determine whether the face image to be recognized is a forged face image.

[0085] In some embodiments, the device 30 further includes: a pre-training module 308. The pre-training module 308 is used to divide the existing categories of forged face images and real face images into multiple tasks, wherein each task includes a real face image category and a first preset number of existing categories; each category includes a small sample set consisting of a second preset number of supporting sample images and a third preset number of query sample images; each time the fourth preset number of tasks are selected into a batch, the batch is input into the face anti-counterfeiting model, multiple groups of parameters of the face anti-counterfeiting model are determined according to the multiple tasks of the batch, and the parameters of the face anti-counterfeiting model are re-determined according to the multiple groups of parameters; until the face anti-counterfeiting model reaches the convergence condition, the pre-training of the face anti-counterfeiting model is completed.

[0086] In some embodiments, the pre-training module 308 is used to determine, for each task in a batch, a third gradient corresponding to the task based on the supporting sample images and the corresponding labels, and determine a set of parameters of the face anti-counterfeiting model based on the third gradient; determine multiple sets of parameters respectively according to multiple tasks in the batch, the query sample images of the multiple tasks and the labels corresponding to the query sample images, and determine the fourth gradient corresponding to the batch; and re-determine the parameters of the face anti-counterfeiting model based on the fourth gradient corresponding to the batch.

[0087] In some embodiments, the pre-training module 308 is used to determine the third gradient corresponding to each task in the batch according to the supporting sample images and the corresponding fine classification labels when the face anti-counterfeiting model is a classification model; or, when the face anti-counterfeiting model is a regression model, determine the third gradient corresponding to each task in the batch according to the supporting sample images and the corresponding coarse classification labels. The fine classification labels are used to mark the categories of forged face images, and the coarse classification labels are used to mark forged face images as positive samples or negative samples.

[0088] In some embodiments, the initial parameters of the face anti-counterfeiting model before pre-training are determined according to a deep learning training method.

[0089] In some embodiments, the existing categories include: at least one of a photo category, a video category, and a mask category. The existing categories of forged face images include: existing categories of forged face images of at least one of a depth modality, a near infrared modality, and a red, green, and blue (RGB) modality. The real face images include: real face images of at least one of a depth modality, a near infrared modality, and a red, green, and blue (RGB) modality. The modality of the forged face images of the new category is different from the modalities of the forged face images and real face images of the existing categories.

[0090] The forged face recognition device in the embodiments of the present disclosure can be implemented by various computing devices or computer systems. Figure 4 as well as Figure 5Give a description.

[0091] Figure 4 FIG. 1 is a structural diagram of some embodiments of the forged face recognition device disclosed in the present invention. Figure 4 As shown, the device 40 of this embodiment includes: a memory 410 and a processor 420 coupled to the memory 410 , and the processor 420 is configured to execute the method for recognizing a forged face in any of some embodiments of the present disclosure based on instructions stored in the memory 410 .

[0092] The memory 410 may include, for example, a system memory, a fixed non-volatile storage medium, etc. The system memory may store, for example, an operating system, an application program, a boot loader, a database, and other programs.

[0093] Figure 5 FIG. 1 is a structural diagram of some other embodiments of the forged face recognition device disclosed in the present invention. Figure 5 As shown, the device 50 of this embodiment includes: a memory 510 and a processor 520, which are similar to the memory 410 and the processor 420 respectively. It can also include an input and output interface 530, a network interface 540, a storage interface 550, etc. These interfaces 530, 540, 550 and the memory 510 and the processor 520 can be connected, for example, through a bus 560. Among them, the input and output interface 530 provides a connection interface for input and output devices such as a display, a mouse, a keyboard, and a touch screen. The network interface 540 provides a connection interface for various networked devices, for example, it can be connected to a database server or a cloud storage server. The storage interface 550 provides a connection interface for external storage devices such as SD cards and USB flash drives.

[0094] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer-usable non-transient storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0095] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0096] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0097] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0098] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure should be included in the protection scope of the present disclosure.

Claims

1. A method for identifying a forged face, comprising: Divide the fake face images and real face images of existing categories into multiple tasks, wherein each task includes a real face image category and a first preset number of existing categories, and each category includes a small sample set consisting of a second preset number of support sample images and a third preset number of query sample images; Each time, a fourth preset number of tasks are selected into a batch, the batch is input into the face anti-counterfeiting model, for each task in the batch, a third gradient corresponding to the task is determined according to the supporting sample images and the corresponding labels, and a set of parameters of the face anti-counterfeiting model is determined according to the third gradient; Determine multiple groups of parameters, query sample images of multiple tasks and labels corresponding to the query sample images according to the multiple tasks in the batch, and determine a fourth gradient corresponding to the batch; Re-determining the parameters of the face anti-counterfeiting model according to the fourth gradient corresponding to the batch; Until the face anti-counterfeiting model reaches a convergence condition, the pre-training of the face anti-counterfeiting model is completed.

2. The identification method according to claim 1, further comprising: Obtain a small sample set of fake face images of a new category; The parameters of the pre-trained face anti-counterfeiting model are updated using the small sample set of the new category of forged face images; wherein the parameters of the pre-trained face anti-counterfeiting model are determined by respectively determining multiple groups of parameters of the face anti-counterfeiting model according to multiple tasks, and are re-determined according to the multiple groups of parameters; the tasks include: one or more small sample sets consisting of forged face images and real face images of existing categories; The face image to be identified is input into the updated face anti-counterfeiting model to determine whether the face image to be identified is a forged face image.

3. The identification method according to claim 2, wherein: The updating of the parameters of the face anti-counterfeiting model using the small sample set of the new category of forged face images includes: In the case where the face anti-counterfeiting model is a classification model, updating the parameters of the face anti-counterfeiting model according to the forged face images in the small sample set of the new category and the corresponding detailed classification labels; Alternatively, when the face anti-counterfeiting model is a regression model, the parameters of the face anti-counterfeiting model are updated according to the forged face images in the small sample set of the new category and the corresponding coarse classification labels; The fine classification label is used to mark the category of the forged face image, and the coarse classification label is used to mark the forged face image as a positive sample or a negative sample.

4. The identification method according to claim 3, wherein: When the face anti-counterfeiting model is a classification model, updating the parameters of the face anti-counterfeiting model according to the forged face images in the small sample set of the new category and the corresponding detailed classification labels includes: In the case where the face anti-counterfeiting model is a classification model, forged face images in the small sample set of the new category are input into the face anti-counterfeiting model to obtain the probability that each forged face image belongs to a different category, determine a first gradient according to the probability that each forged face image belongs to a different category and the corresponding detailed classification label, and update the parameters of the face anti-counterfeiting model according to the first gradient; Alternatively, when the face anti-counterfeiting model is a regression model, updating the parameters of the face anti-counterfeiting model according to the forged face images in the small sample set of the new category and the corresponding coarse classification labels includes: When the face anti-counterfeiting model is a regression model, the forged face images in the small sample set of the new category are input into the face anti-counterfeiting model to obtain output values ​​corresponding to each forged face image, and a second gradient is determined according to the output values ​​corresponding to each forged face image and the corresponding coarse classification labels, and the parameters of the face anti-counterfeiting model are updated according to the second gradient.

5. The identification method according to claim 1, wherein: For each task in the batch, determining the third gradient corresponding to the task according to the supporting sample images and the corresponding labels includes: In the case where the face anti-counterfeiting model is a classification model, for each task in the batch, determining a third gradient corresponding to the task according to the supporting sample images and the corresponding fine classification labels; Alternatively, when the face anti-counterfeiting model is a regression model, for each task in the batch, a third gradient corresponding to the task is determined according to the supporting sample images and the corresponding coarse classification labels; The fine classification label is used to mark the category of the forged face image, and the coarse classification label is used to mark the forged face image as a positive sample or a negative sample.

6. The identification method according to claim 1, wherein: The initial parameters of the face anti-counterfeiting model before pre-training are determined according to a deep learning training method.

7. The identification method according to any one of claims 2 to 4, wherein: The existing categories include: at least one of a photo category, a video category, and a mask category; The existing categories of forged face images include: existing categories of forged face images of at least one of the depth modality, near infrared modality, and red, green, and blue (RGB) modality; The real face image includes: a real face image of at least one of a depth modality, a near infrared modality, and a red, green, and blue (RGB) modality; The modality of the new category of forged face images is different from the modalities of the existing categories of forged face images and real face images.

8. A forged face recognition device, comprising: A pre-training module is used to divide the forged face images and real face images of existing categories into multiple tasks, wherein each task includes a real face image category and a first preset number of existing categories, and each category includes a small sample set composed of a second preset number of supporting sample images and a third preset number of query sample images; each time a fourth preset number of tasks are selected into a batch, and the batch is input into a face anti-counterfeiting model, for each task in the batch, a third gradient corresponding to the task is determined according to the supporting sample images and the corresponding labels, and a set of parameters of the face anti-counterfeiting model is determined according to the third gradient; multiple sets of parameters are determined respectively according to multiple tasks in the batch, and a fourth gradient corresponding to the batch is determined according to the query sample images of the multiple tasks and the labels corresponding to the query sample images; the parameters of the face anti-counterfeiting model are re-determined according to the fourth gradient corresponding to the batch; until the face anti-counterfeiting model reaches the convergence condition, the pre-training of the face anti-counterfeiting model is completed.

9. The identification device according to claim 8, further comprising: A small sample acquisition module, used to obtain a small sample set of forged face images of a new category; A model adjustment module, used to update the parameters of the pre-trained face anti-counterfeiting model using the small sample set of the new category of forged face images; wherein the parameters of the pre-trained face anti-counterfeiting model are determined according to multiple tasks, respectively, and are re-determined according to the multiple sets of parameters; the tasks include: one or more small sample sets consisting of forged face images and real face images of existing categories; The face recognition module is used to input the face image to be recognized into the updated face anti-counterfeiting model to determine whether the face image to be recognized is a forged face image.

10. The identification device according to claim 9, wherein: The model adjustment module is used to update the parameters of the face anti-counterfeiting model according to the forged face images in the small sample set of the new category and the corresponding fine classification labels when the face anti-counterfeiting model is a classification model; or, if the face anti-counterfeiting model is a regression model, update the parameters of the face anti-counterfeiting model according to the forged face images in the small sample set of the new category and the corresponding coarse classification labels; The fine classification label is used to mark the category of the forged face image, and the coarse classification label is used to mark the forged face image as a positive sample or a negative sample.

11. The identification device according to claim 10, wherein: The model adjustment module is used to, when the face anti-counterfeiting model is a classification model, input the forged face images in the small sample set of the new category into the face anti-counterfeiting model to obtain the probability that each forged face image belongs to different categories, determine a first gradient according to the probability that each forged face image belongs to different categories and the corresponding detailed classification labels, and update the parameters of the face anti-counterfeiting model according to the first gradient; or, when the face anti-counterfeiting model is a regression model, input the forged face images in the small sample set of the new category into the face anti-counterfeiting model to obtain the output values ​​corresponding to each forged face image, determine a second gradient according to the output values ​​corresponding to each forged face image and the corresponding coarse classification labels, and update the parameters of the face anti-counterfeiting model according to the second gradient.

12. The identification device according to claim 8, wherein: The pre-training module is used for determining, for each task in the batch, the third gradient corresponding to the task according to the supporting sample images and the corresponding fine classification labels when the face anti-counterfeiting model is a classification model; or, for each task in the batch, the third gradient corresponding to the task according to the supporting sample images and the corresponding coarse classification labels when the face anti-counterfeiting model is a regression model; The fine classification label is used to mark the category of the forged face image, and the coarse classification label is used to mark the forged face image as a positive sample or a negative sample.

13. The identification device according to claim 8, wherein: The initial parameters of the face anti-counterfeiting model before pre-training are determined according to a deep learning training method.

14. The identification device according to any one of claims 9 to 11, wherein: The existing categories include: at least one of a photo category, a video category, and a mask category; The existing categories of forged face images include: existing categories of forged face images of at least one of the depth modality, near infrared modality, and red, green, and blue (RGB) modality; The real face image includes: a real face image of at least one of a depth modality, a near infrared modality, and a red, green, and blue (RGB) modality; The modality of the new category of forged face images is different from the modalities of the existing categories of forged face images and real face images.

15. A forged face recognition device, comprising: Memory; as well as A processor coupled to the memory, the processor being configured to execute the forged face recognition method according to any one of claims 1 to 7 based on instructions stored in the memory.

16. A computer-readable storage medium having a computer program stored thereon, wherein: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images