Image Processing and Model Acquisition Method, Device, Electronic Device and Storage Medium

Abstract

The present disclosure provides an image processing method, a model acquisition method, an apparatus, an electronic device, and a storage medium, which relate to the fields of artificial intelligence such as computer vision, augmented reality, virtual reality, and deep learning, and can be applied to scenarios such as smart cities. The image processing method may include: obtaining an image to be processed, using it as an input to a first model, and obtaining an output face liveness recognition result and a forged face discrimination result. The first model is trained through M training stages according to a first training data set and a second training data set. Different training stages correspond to different training methods. The first training data set is a face liveness recognition binary classification data set, and the second training data set is a forged face discrimination binary classification data set. Each training data is image data. By applying the solution described in the present disclosure, the accuracy of the face liveness recognition result and the forged face discrimination result can be improved, etc.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and particularly to image processing and model acquisition methods, devices, electronic devices, and storage media in the fields of computer vision, augmented reality, virtual reality, and deep learning. Background Art

[0002] Currently, face recognition has been widely used in identity verification in daily life, and both face liveness recognition and forged face identification are important components of a face recognition system and are very crucial defense means. The attack methods of face liveness mainly include print attacks, screen attacks, mask attacks, and head mold attacks, etc. For forged faces, they mainly include face swapping, expression driving, and generation by generative adversarial networks (GANs), etc. Once an attack or forged face breaks through the defense system, it is very likely to cause serious consequences such as property losses. Summary of the Invention

[0003] The present disclosure provides an image processing method, device, electronic device, and storage medium, as well as a model acquisition method.

[0004] An image processing method includes:

[0005] Obtaining an image to be processed;

[0006] Using the image to be processed as the input of a first model to obtain an output face liveness recognition result and a forged face identification result, where the first model is trained through M training stages according to a first training data set and a second training data set, M is a positive integer, different training stages correspond to different training methods, the first training data set is a face liveness recognition binary data set, which includes real person training data as positive samples and attack training data as negative samples, the second training data set is a forged face identification binary data set, which includes real person training data as positive samples and forged training data as negative samples, and each training data is image data.

[0007] A model acquisition method includes:

[0008] Obtaining a first training data set and a second training data set, where the first training data set is a face liveness recognition binary data set, which includes real person training data as positive samples and attack training data as negative samples, the second training data set is a forged face identification binary data set, which includes real person training data as positive samples and forged training data as negative samples, and each training data is image data;

[0009] Using the first training dataset and the second training dataset, a first model is trained through M training phases, where M is a positive integer, and different training phases correspond to different training methods. The first model is used to output the face liveness recognition result and the forged face discrimination result corresponding to the image to be processed.

[0010] An image processing device includes: an image acquisition module and an image processing module;

[0011] The image acquisition module is used to acquire the image to be processed;

[0012] The image processing module is used to use the image to be processed as the input of the first model to obtain the output face liveness recognition result and the forged face discrimination result. Among them, the first model is trained through M training phases according to the first training dataset and the second training dataset, where M is a positive integer, and different training phases correspond to different training methods. The first training dataset is a face liveness recognition binary classification dataset, which includes real person training data as positive samples and attack training data as negative samples. The second training dataset is a forged face discrimination binary classification dataset, which includes real person training data as positive samples and forged training data as negative samples. Each training data is image data.

[0013] A model acquisition device includes: a data acquisition module and a model training module;

[0014] The data acquisition module is used to acquire the first training dataset and the second training dataset. The first training dataset is a face liveness recognition binary classification dataset, which includes real person training data as positive samples and attack training data as negative samples. The second training dataset is a forged face discrimination binary classification dataset, which includes real person training data as positive samples and forged training data as negative samples. Each training data is image data;

[0015] The model training module is used to use the first training dataset and the second training dataset to train a first model through M training phases, where M is a positive integer, and different training phases correspond to different training methods. The first model is used to output the face liveness recognition result and the forged face discrimination result corresponding to the image to be processed.

[0016] An electronic device includes:

[0017] At least one processor; and

[0018] A memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method as described above.

[0020] A non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the method as described above.

[0021] A computer program product, comprising a computer program / instructions which, when executed by a processor, implement the method as described above.

[0022] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The drawings are used to better understand the solution and do not constitute a limitation to the present disclosure. Among them:

[0024] Figure 1 is a flowchart of an embodiment of the image processing method described in the present disclosure;

[0025] Figure 2 is a flowchart of an embodiment of the model acquisition method described in the present disclosure;

[0026] Figure 3 is a schematic structural diagram of the composition of Embodiment 300 of the image processing apparatus described in the present disclosure;

[0027] Figure 4 is a schematic structural diagram of the composition of Embodiment 400 of the model acquisition apparatus described in the present disclosure;

[0028] Figure 5 shows a schematic block diagram of an electronic device 500 that can be used to implement the embodiments of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0029] The following describes exemplary embodiments of the present disclosure with reference to the accompanying drawings. Various details of the embodiments of the present disclosure are included to assist in understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0030] In addition, it should be understood that the term "and / or" in this text is merely a description of the association relationship between associated objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this text generally indicates that the associated objects before and after are in an "or" relationship.

[0031] Figure 1 This is a flowchart of an embodiment of the image processing method described in this disclosure. As Figure 1 shown, it includes the following specific implementation manners.

[0032] In step 101, an image to be processed is obtained.

[0033] In step 102, the image to be processed is used as the input of a first model, and a face liveness recognition result and a forged face discrimination result are obtained. The first model is trained through M training stages according to a first training dataset and a second training dataset, where M is a positive integer, and different training stages correspond to different training methods. The first training dataset is a face liveness recognition binary classification dataset, which includes real person training data as positive samples and attack training data as negative samples. The second training dataset is a forged face discrimination binary classification dataset, which includes real person training data as positive samples and forged training data as negative samples. Each training data is image data.

[0034] In the traditional method, for the two tasks of face liveness recognition and forged face discrimination, a model is trained separately. In actual use, for an image, face liveness recognition and forged face discrimination need to be performed separately. Correspondingly, two models need to be run separately to obtain the face liveness recognition result and the forged face discrimination result respectively. However, this method has at least the following problems: the two tasks are completely independent, ignoring the association between them, so that the two models do not affect each other, thus reducing the training effect, and correspondingly reducing the accuracy of the face liveness recognition result and the forged face discrimination result. Moreover, for the same image, two models need to be run separately, which increases the time consumption and reduces the processing efficiency, etc.

[0035] By adopting the solution described in the above method embodiment, the two tasks share one model, that is, a multi-task training method is adopted, so that the two tasks can influence and promote each other during the training process, thereby improving the training effect, and correspondingly improving the accuracy of the face liveness recognition result and the forged face discrimination result. Moreover, in actual use, only one model needs to be run to obtain the face liveness recognition result and the forged face discrimination result simultaneously, thereby reducing the time consumption and improving the processing efficiency, etc.

[0036] The image to be processed can be any image. The image to be processed can be used as the input of the first model, so as to obtain the face liveness recognition result and the forged face discrimination result output by the first model at the same time.

[0037] It can be seen that the implementation of the above method depends on the first model obtained by pre-training. The method for obtaining the first model will be described below. In addition, in the solution described in the present disclosure, the specific type and specific structure of the first model are not limited, as long as the corresponding functions can be implemented.

[0038] Figure 2 It is a flowchart of an embodiment of the model acquisition method described in the present disclosure. As Figure 2 shown, it includes the following specific implementation manners.

[0039] In step 201, a first training data set and a second training data set are obtained. The first training data set is a face liveness recognition binary classification data set, which includes real person training data as positive samples and attack training data as negative samples. The second training data set is a forged face discrimination binary classification data set, which includes real person training data as positive samples and forged training data as negative samples. Each training data is image data.

[0040] In step 202, the first training data set and the second training data set are used to train the first model through M training stages. M is a positive integer. Different training stages correspond to different training methods. The first model is used to output the face liveness recognition result and the forged face discrimination result corresponding to the image to be processed.

[0041] Adopting the solution described in the above method embodiment, the two tasks share one model, that is, a multi-task training method is adopted, so that the two tasks can influence and promote each other during the training process, thereby improving the training effect, and correspondingly improving the accuracy of the face liveness recognition result and the forged face discrimination result. Moreover, in actual use, only a single model needs to be run to obtain the face liveness recognition result and the forged face discrimination result at the same time, thereby reducing the time consumption and improving the processing efficiency, etc.

[0042] The first training dataset can be a two-class dataset for face liveness recognition, which includes real-person training data as positive samples and attack training data as negative samples. The second training dataset can be a two-class dataset for forged face discrimination, which includes real-person training data as positive samples and forged training data as negative samples. Each training data is image data. There is no restriction on how to obtain the first training dataset and the second training dataset. For example, as a possible implementation, the real-person training data in the first training dataset can be photographed or collected, and the attack training data in the first training dataset can be generated by means of print attacks, screen attacks, mask attacks, etc. Similarly, the real-person training data in the second training dataset can be photographed or collected, and the forged training data in the second training dataset, that is, forged face training data, can be generated by means of face swapping, expression driving, etc.

[0043] Preferably, the ratio of positive samples to negative samples in both the first training dataset and the second training dataset can be 1:1. Additionally, the labels corresponding to the positive samples and negative samples can be 1 and 0 respectively.

[0044] Using the first training dataset and the second training dataset, the first model can be trained through M training phases. The specific value of M can be determined according to actual needs. For example, it can be one, or greater than one.

[0045] Preferably, when the value of M is one, the training phase can include: the first training phase, which is a joint training phase for face liveness recognition tasks and forged face discrimination tasks. When the value of M is greater than one, the training phase can include: the first training phase, the second training phase, and the third training phase. Among them, the second training phase is a training phase mainly for face liveness recognition tasks and supplemented by forged face discrimination tasks, and the third training phase is a training phase mainly for forged face discrimination tasks and supplemented by face liveness recognition tasks. Additionally, both the second training phase and the third training phase are executed after the first training phase, and the second training phase is executed before the third training phase, or the second training phase is executed after the third training phase.

[0046] Through the joint training in the first training phase, the first model can output the face liveness recognition result and the forged face discrimination result corresponding to the image to be processed simultaneously. Through the training in the second training phase and the third training phase, the performance of the first model can be further optimized, thereby further improving the accuracy of the face liveness recognition result and the forged face discrimination result, etc.

[0047] The specific implementation of each training phase is described below.

[0048] 1) The first training phase

[0049] Preferably, for the training of any batch in the first training stage, that is, any round of training, the following first processing can be performed respectively: obtain a first quantity of training data from the first training dataset, and obtain a second quantity of training data from the second training dataset, where the first quantity is equal to the second quantity, and the sum of the first quantity and the second quantity is equal to a first predetermined value, and the first predetermined value represents the total amount of training data to be input into the first model for this batch; for the training data obtained from the first training dataset, obtain the corresponding first face liveness recognition loss, and for the training data obtained from the second training dataset, obtain the corresponding first forged face discrimination loss; combine the first face liveness recognition loss and the first forged face discrimination loss to determine the first overall loss, and update the first model according to the first overall loss; in response to the training reaching convergence of the first model, determine the end of the first training stage, otherwise, for the next batch, repeat the execution of the first processing.

[0050] The specific value of the first predetermined value can be determined according to actual needs. For example, assuming the value of the first predetermined value is 20 (the number is only for illustration), then 10 (the first quantity) pieces of training data can be obtained from the first training dataset, and 10 (the second quantity) pieces of training data can be obtained from the second training dataset, that is, the ratio of the first quantity to the second quantity can be 1:1.

[0051] For the 10 pieces of training data obtained from the first training dataset, the corresponding first face liveness recognition loss can be obtained, and for the 10 pieces of training data obtained from the second training dataset, the corresponding first forged face discrimination loss can be obtained.

[0052] There is also no restriction on how to obtain the first face liveness recognition loss and the first forged face discrimination loss. For example, they can both be cross-entropy losses. Additionally, taking the first face liveness recognition loss as an example, face features can be extracted from the training data, and face liveness recognition classification (whether it is a live face) can be performed based on the extracted face features. Furthermore, the loss can be determined by combining the classification result and the label corresponding to the training data, etc.

[0053] Furthermore, the first overall loss can be determined by combining the first face liveness recognition loss and the first forged face discrimination loss. Preferably, the product of the first face liveness recognition loss and the first coefficient can be obtained, and the product of the first forged face discrimination loss and the second coefficient can be obtained. Then, the sum of the two products can be used as the first overall loss. The first coefficient is the ratio of the first quantity to the first predetermined value, and the second coefficient is the ratio of the second quantity to the first predetermined value.

[0054] That is, there can be:

[0055] L z = 0.5×L1 + 0.5×L2; (1)

[0056] Among them, L z represents the first overall loss, L1 represents the first face liveness recognition loss, L2 represents the first forged face discrimination loss, and the two 0.5 respectively represent the first coefficient and the second coefficient.

[0057] Furthermore, the first model can be updated according to the first overall loss, that is, the model parameters are updated. After that, if it is determined that convergence is reached, it can be determined that the first training stage ends; otherwise, for the next batch, the above-mentioned first process is repeatedly executed.

[0058] Through the above processing, joint training for the face liveness recognition task and the forged face discrimination task can be achieved, so that the two tasks can influence and promote each other during the training process, thereby improving the training effect of the model, that is, improving the model performance.

[0059] In addition, preferably, the ratio of positive samples to negative samples in the training data obtained from the first training dataset each time can be 1:1, and the ratio of positive samples to negative samples in the training data obtained from the second training dataset each time can be 1:1. By balancing the number of positive and negative samples, the training effect of the model can be further improved.

[0060] 2) The second training stage

[0061] Preferably, for any batch of training in the second training stage, the following second process can be respectively performed: obtain the third quantity of training data from the first training dataset, and obtain the fourth quantity of training data from the second training dataset, where the third quantity is greater than the fourth quantity, and the sum of the third quantity and the fourth quantity is equal to the second predetermined value, and the second predetermined value represents the total amount of training data that needs to be input into the first model for this batch; for the training data obtained from the first training dataset, obtain the corresponding second face liveness classification loss, and for the training data obtained from the second training dataset, obtain the corresponding second forged face discrimination loss; combine the second face liveness classification loss and the second forged face discrimination loss to determine the second overall loss, and update the first model according to the second overall loss; in response to the training reaching convergence of the first model, determine that the second training stage ends; otherwise, for the next batch, repeat the above-mentioned second process.

[0062] The specific value of the second predetermined value can be determined according to actual needs. For example, assuming that the value of the second predetermined value is 20, then 18 (the third quantity) pieces of training data can be obtained from the first training dataset, and 2 (the fourth quantity) pieces of training data can be obtained from the second training dataset, that is, the ratio of the third quantity to the fourth quantity can be 9:1.

[0063] For the 18 training data obtained from the first training dataset, the corresponding second face liveness recognition loss can be obtained. For the 2 training data obtained from the second training dataset, the corresponding second forged face discrimination loss can be obtained.

[0064] Furthermore, the second overall loss can be determined by combining the second face liveness recognition loss and the second forged face discrimination loss. Preferably, the product of the second face liveness classification loss and the third coefficient can be obtained, and the product of the second forged face discrimination loss and the fourth coefficient can be obtained. Then, the sum of the two products can be used as the second overall loss. The third coefficient is the ratio of the third quantity to the second predetermined value, and the fourth coefficient is the ratio of the fourth quantity to the second predetermined value.

[0065] That is:

[0066] L z ' = 0.9×L1' + 0.1×L2'; (2)

[0067] Where, L z ' represents the second overall loss, L1' represents the second face liveness recognition loss, L2' represents the second forged face discrimination loss, and 0.9 and 0.1 represent the third coefficient and the fourth coefficient respectively.

[0068] Furthermore, the first model can be updated according to the second overall loss. After that, if it is determined that convergence is reached, the second training stage can be determined to end. Otherwise, for the next batch, the above-mentioned second processing can be repeatedly executed.

[0069] Through the above processing, targeted optimization for the face liveness recognition task can be achieved, thereby further improving the performance of the model in the face liveness recognition task, and further improving the accuracy of the face liveness recognition result, etc.

[0070] In addition, preferably, the ratio of positive samples to negative samples in the training data obtained from the first training dataset each time can be 1:1, and the ratio of positive samples to negative samples in the training data obtained from the second training dataset each time can be 1:1. By balancing the number of positive and negative samples, the training effect of the model can be further improved.

[0071] 3) The third training stage

[0072] Preferably, for the training of any batch in the third training stage, the following third processing can be performed respectively: obtain a fifth quantity of training data from the first training dataset and a sixth quantity of training data from the second training dataset, where the fifth quantity is less than the sixth quantity, and the sum of the fifth quantity and the sixth quantity is equal to a third predetermined value, and the third predetermined value represents the total amount of training data to be input into the first model for this batch; for the training data obtained from the first training dataset, obtain the corresponding third face liveness classification loss, and for the training data obtained from the second training dataset, obtain the corresponding third forged face discrimination loss; combine the third face liveness classification loss and the third forged face discrimination loss to determine the third overall loss, and update the first model according to the third overall loss; in response to the training reaching convergence of the first model, determine that the third training stage ends, otherwise, for the next batch, repeat the execution of the third processing.

[0073] The specific value of the third predetermined value can be determined according to actual needs. For example, assuming the value of the third predetermined value is 20, then 2 (the fifth quantity) training data can be obtained from the first training dataset, and 18 (the sixth quantity) training data can be obtained from the second training dataset, that is, the ratio of the fifth quantity to the sixth quantity can be 1:9.

[0074] For the 2 training data obtained from the first training dataset, the corresponding third face liveness recognition loss can be obtained, and for the 18 training data obtained from the second training dataset, the corresponding third forged face discrimination loss can be obtained.

[0075] Furthermore, the third overall loss can be determined by combining the third face liveness classification loss and the third forged face discrimination loss. Preferably, the product of the third face liveness classification loss and the fifth coefficient can be obtained, and the product of the third forged face discrimination loss and the sixth coefficient can be obtained, and then the sum of the two products can be used as the third overall loss. The fifth coefficient is the ratio of the fifth quantity to the third predetermined value, and the sixth coefficient is the ratio of the sixth quantity to the third predetermined value.

[0076] That is, there can be:

[0077] L z ” = 0.1×L1” + 0.9×L2”; (3)

[0078] Among them, L z ” represents the third overall loss, L1” represents the third face liveness recognition loss, L2” represents the third forged face discrimination loss, and 0.1 and 0.9 respectively represent the fifth coefficient and the sixth coefficient.

[0079] Further, the first model can be updated according to the third overall loss. After that, if it is determined that convergence is achieved, it can be determined that the third training stage ends; otherwise, for the next batch, the above-mentioned third process can be repeatedly executed.

[0080] Through the above processing, targeted optimization for the forged face identification task can be achieved, thereby further improving the performance of the model in the forged face identification task, and further improving the accuracy of the forged face identification result, etc.

[0081] In addition, preferably, the ratio of positive samples to negative samples in the training data obtained from the first training dataset each time can be 1:1, and the ratio of positive samples to negative samples in the training data obtained from the second training dataset each time can be 1:1. By balancing the number of positive and negative samples, the training effect of the model can be further improved.

[0082] After completing the training of the above three training stages, the final required first model can be obtained. Subsequently, the first model can be used to process the image to be processed, and output the face liveness recognition result and the forged face identification result corresponding to the image to be processed.

[0083] It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present disclosure is not limited by the described action sequence, because according to the present disclosure, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the present disclosure. In addition, for the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions in other embodiments.

[0084] In summary, by adopting the solution described in the method embodiment of the present disclosure, the accuracy of the face liveness recognition result and the forged face identification result can be improved, and the time consumption can be reduced and the processing efficiency can be improved, etc.

[0085] The above is the introduction of the method embodiment. The following further illustrates the solution of the present disclosure through the device embodiment.

[0086] Figure 3 It is a schematic structural diagram of the composition of the image processing device embodiment 300 of the present disclosure. As Figure 3 shown, it includes: an image acquisition module 301 and an image processing module 302.

[0087] The image acquisition module 301 is used to acquire the image to be processed.

[0088] The image processing module 302 is configured to use the image to be processed as the input of the first model, and obtain the output face liveness recognition result and the forged face discrimination result. The first model is trained through M training phases according to the first training dataset and the second training dataset. M is a positive integer, and different training phases correspond to different training methods. The first training dataset is a binary classification dataset for face liveness recognition, which includes real-person training data as positive samples and attack training data as negative samples. The second training dataset is a binary classification dataset for forged face discrimination, which includes real-person training data as positive samples and forged training data as negative samples. Each training data is image data.

[0089] Adopting the solution described in the above device embodiment, the two tasks share one model, that is, the multi-task training method is adopted, so that the two tasks can influence and promote each other during the training process, thereby improving the training effect, and correspondingly improving the accuracy of the face liveness recognition result and the forged face discrimination result. Moreover, in actual use, only a single model needs to be run to obtain the face liveness recognition result and the forged face discrimination result at the same time, thereby reducing the time consumption and improving the processing efficiency, etc.

[0090] Figure 4 It is a schematic structural diagram of the composition of the model acquisition device embodiment 400 described in the present disclosure. As Figure 4 shown, it includes: a data acquisition module 401 and a model training module 402.

[0091] The data acquisition module 401 is configured to acquire the first training dataset and the second training dataset. The first training dataset is a binary classification dataset for face liveness recognition, which includes real-person training data as positive samples and attack training data as negative samples. The second training dataset is a binary classification dataset for forged face discrimination, which includes real-person training data as positive samples and forged training data as negative samples. Each training data is image data.

[0092] The model training module 402 is configured to use the first training dataset and the second training dataset to train the first model through M training phases. M is a positive integer, and different training phases correspond to different training methods. The first model is used to output the face liveness recognition result and the forged face discrimination result corresponding to the image to be processed.

[0093] Adopting the solution described in the above device embodiment, two tasks share one model, that is, a multi-task training method is adopted, so that the two tasks can influence and promote each other during the training process, thereby improving the training effect, and correspondingly improving the accuracy of the face liveness recognition result and the forged face discrimination result. Moreover, in actual use, only one model needs to be run to obtain the face liveness recognition result and the forged face discrimination result at the same time, thereby reducing the time consumption and improving the processing efficiency, etc.

[0094] Using the first training data set and the second training data set, the first model can be trained through M training phases. The specific value of M can be determined according to actual needs. For example, it can be one or greater than one.

[0095] Preferably, when the value of M is one, the training phase may include: a first training phase, and the first training phase is a joint training phase for the face liveness recognition task and the forged face discrimination task. When the value of M is greater than one, the training phase may include: a first training phase, a second training phase, and a third training phase. Among them, the second training phase is a training phase mainly for the face liveness recognition task and supplemented by the forged face discrimination task, and the third training phase is a training phase mainly for the forged face discrimination task and supplemented by the face liveness recognition task. In addition, both the second training phase and the third training phase are executed after the first training phase, and the second training phase is executed before the third training phase, or the second training phase is executed after the third training phase.

[0096] Preferably, for any batch of training in the first training phase, the model training module 402 can respectively perform the following first processing: obtain a first quantity of training data from the first training data set and a second quantity of training data from the second training data set. The first quantity is equal to the second quantity, and the sum of the first quantity and the second quantity is equal to a first predetermined value, and the first predetermined value represents the total amount of training data that needs to be input into the first model for this batch; for the training data obtained from the first training data set, obtain the corresponding first face liveness recognition loss, and for the training data obtained from the second training data set, obtain the corresponding first forged face discrimination loss; combine the first face liveness recognition loss and the first forged face discrimination loss to determine a first overall loss, and update the first model according to the first overall loss; in response to the training reaching the convergence of the first model, determine that the first training phase ends, otherwise, for the next batch, repeat the execution of the first processing.

[0097] Additionally, preferably, the model training module 402 can obtain the product of the first face liveness recognition loss and the first coefficient, and can obtain the product of the first forged face discrimination loss and the second coefficient. Furthermore, the sum of the two products can be used as the first overall loss. The first coefficient is the ratio of the first quantity to the first predetermined value, and the second coefficient is the ratio of the second quantity to the first predetermined value.

[0098] Preferably, for any batch of training in the second training stage, the model training module 402 can respectively perform the following second processing: obtain a third quantity of training data from the first training dataset and a fourth quantity of training data from the second training dataset. The third quantity is greater than the fourth quantity, and the sum of the third quantity and the fourth quantity is equal to the second predetermined value, where the second predetermined value represents the total amount of training data to be input into the first model for this batch; for the training data obtained from the first training dataset, obtain the corresponding second face liveness classification loss, and for the training data obtained from the second training dataset, obtain the corresponding second forged face discrimination loss; determine the second overall loss by combining the second face liveness classification loss and the second forged face discrimination loss, and update the first model according to the second overall loss; in response to the training reaching convergence of the first model, determine the end of the second training stage, otherwise, for the next batch, repeat the execution of the second processing. For example, the ratio of the third quantity to the fourth quantity can be 9:1.

[0099] Additionally, preferably, the model training module 402 can obtain the product of the second face liveness classification loss and the third coefficient, and can obtain the product of the second forged face discrimination loss and the fourth coefficient. Furthermore, the sum of the two products can be used as the second overall loss. The third coefficient is the ratio of the third quantity to the second predetermined value, and the fourth coefficient is the ratio of the fourth quantity to the second predetermined value.

[0100] Preferably, for any batch of training in the third training stage, the model training module 402 can respectively perform the following third processing: obtain a fifth quantity of training data from the first training dataset and a sixth quantity of training data from the second training dataset. The fifth quantity is less than the sixth quantity, and the sum of the fifth quantity and the sixth quantity is equal to the third predetermined value, where the third predetermined value represents the total amount of training data to be input into the first model for this batch; for the training data obtained from the first training dataset, obtain the corresponding third face liveness classification loss, and for the training data obtained from the second training dataset, obtain the corresponding third forged face discrimination loss; determine the third overall loss by combining the third face liveness classification loss and the third forged face discrimination loss, and update the first model according to the third overall loss; in response to the training reaching convergence of the first model, determine the end of the third training stage, otherwise, for the next batch, repeat the execution of the third processing. For example, the ratio of the fifth quantity to the sixth quantity can be 1:9.

[0101] Additionally, preferably, the model training module 402 may obtain the product of the third face liveness classification loss and the fifth coefficient, and may obtain the product of the third forged face discrimination loss and the sixth coefficient. Furthermore, the sum of the two products may be used as the third overall loss. The fifth coefficient is the ratio of the fifth quantity to the third predetermined value, and the sixth coefficient is the ratio of the sixth quantity to the third predetermined value.

[0102] Furthermore, preferably, the ratio of positive samples to negative samples in the training data obtained from the first training dataset each time is 1:1, and the ratio of positive samples to negative samples in the training data obtained from the second training dataset each time is 1:1.

[0103] Figure 3 and Figure 4 The specific working process of the device embodiment shown may refer to the relevant descriptions in the foregoing method embodiment and will not be elaborated herein.

[0104] In summary, by adopting the solution described in the device embodiment of the present disclosure, the accuracy of face liveness recognition results and forged face discrimination results can be improved, and the time consumption can be reduced and the processing efficiency can be enhanced, etc.

[0105] The solution described in the present disclosure can be applied to the field of artificial intelligence, particularly in the fields of computer vision, augmented reality, virtual reality, and deep learning. Artificial intelligence is a discipline that studies how to make a computer simulate certain thinking processes and intelligent behaviors of humans (such as learning, reasoning, thinking, planning, etc.). It has both hardware-level technologies and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing. Artificial intelligence software technologies mainly include several major directions such as computer vision technology, speech recognition technology, natural language processing technology, and machine learning / deep learning, big data processing technology, and knowledge graph technology.

[0106] The images and the like in the embodiments described in the present disclosure are not targeted at a specific user and do not reflect the personal information of a specific user. In the technical solution of the present disclosure, the processing of the collection, storage, use, processing, transmission, provision, and disclosure of the user's personal information involved all comply with the provisions of relevant laws and regulations and do not violate public order and good customs.

[0107] According to the embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0108] Figure 5FIG. 0 shows a schematic block diagram of an electronic device 500 that can be used to implement embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as, for example, laptop computers, desktop computers, workstations, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as, for example, personal digital assistants, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are only examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0109] As Figure 5 shown, the device 500 includes a computing unit 501 that can perform various appropriate actions and processes in accordance with a computer program stored in a read only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the device 500 can also be stored. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0110] A plurality of components in the device 500 are connected to the I / O interface 505, including: an input unit 506, such as, for example, a keyboard, a mouse, etc.; an output unit 507, such as, for example, various types of displays, speakers, etc.; a storage unit 508, such as, for example, a magnetic disk, an optical disk, etc.; and a communication unit 509, such as, for example, a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the device 500 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0111] The computing unit 501 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 executes the various methods and processes described above, such as the methods described in this disclosure. For example, in some embodiments, the methods described in this disclosure can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 508. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the methods described in this disclosure can be executed. Alternatively, in other embodiments, the computing unit 501 can be configured to execute the methods described in this disclosure by any other suitable means (e.g., by means of firmware).

[0112] Various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuitry, integrated circuit systems, field-programmable gate arrays (FPGA), application-specific integrated circuits (ASIC), application-specific standard products (ASSP), systems-on-a-chip (SOC), complex programmable logic devices (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special or general-purpose programmable processor that receives data and instructions from a storage system, at least one input device, and at least one output device, and transmits the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0113] The program code for implementing the methods of this disclosure can be written in any combination of one or more programming languages. These program codes can be provided to the processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program codes can be executed entirely on the machine, partially on the machine, as an independent software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0114] In the context of this disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0115] To provide for interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can also be used to provide for interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic, speech, or tactile input).

[0116] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or in a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0117] A computer system can include a client and a server. The client and the server are generally remote from each other and typically interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, a server of a distributed system, or a server incorporating a blockchain.

[0118] It should be understood that the various forms of the processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution disclosed in the present disclosure can be achieved, and no limitation is imposed herein.

[0119] The above specific embodiments do not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub - combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present disclosure shall be included within the protection scope of the present disclosure.

Claims

1. An image processing method, comprising: Obtaining an image to be processed; Using the image to be processed as the input of a first model to obtain an output face liveness recognition result and a forged face discrimination result, wherein the first model is trained through M training phases according to a first training data set and a second training data set, M being a positive integer, different training phases corresponding to different training methods, the first training data set being a face liveness recognition binary data set, which includes real person training data as positive samples and attack training data as negative samples, the second training data set being a forged face discrimination binary data set, which includes real person training data as positive samples and forged training data as negative samples, and each training data being image data; wherein the M training phases include: a first training phase and one or all of the following: a second training phase, a third training phase, the first training phase being a joint training phase for the face liveness recognition task and the forged face discrimination task, the second training phase being a training phase mainly for the face liveness recognition task and supplemented by the forged face discrimination task, the third training phase being a training phase mainly for the forged face discrimination task and supplemented by the face liveness recognition task, and the second training phase and the third training phase are both executed after the first training phase.

2. A model acquisition method, comprising: Obtaining a first training data set and a second training data set, the first training data set being a face liveness recognition binary data set, which includes real person training data as positive samples and attack training data as negative samples, the second training data set being a forged face discrimination binary data set, which includes real person training data as positive samples and forged training data as negative samples, and each training data being image data; Using the first training data set and the second training data set to train a first model through M training phases, M being a positive integer, different training phases corresponding to different training methods, the first model being used to output a face liveness recognition result and a forged face discrimination result corresponding to the image to be processed; wherein the M training phases include: a first training phase and one or all of the following: a second training phase, a third training phase, the first training phase being a joint training phase for the face liveness recognition task and the forged face discrimination task, the second training phase being a training phase mainly for the face liveness recognition task and supplemented by the forged face discrimination task, the third training phase being a training phase mainly for the forged face discrimination task and supplemented by the face liveness recognition task, and the second training phase and the third training phase are both executed after the first training phase.

3. The method according to claim 2, wherein The training method corresponding to the first training phase includes: For any batch of training in the first training phase, respectively perform the following first processing: Obtain a first quantity of training data from the first training dataset and a second quantity of training data from the second training dataset, where the first quantity is equal to the second quantity, and the sum of the first quantity and the second quantity is equal to a first predetermined value, and the first predetermined value represents the total amount of training data to be input into the first model for this batch; For the training data obtained from the first training dataset, obtain the corresponding first face liveness recognition loss, and for the training data obtained from the second training dataset, obtain the corresponding first forged face discrimination loss; Combine the first face liveness recognition loss and the first forged face discrimination loss to determine a first overall loss, and update the first model according to the first overall loss; In response to the training reaching convergence of the first model, determine that the first training stage ends; otherwise, for the next batch, repeat the first process.

4. The method according to claim 3, wherein, The combining the first face liveness recognition loss and the first forged face discrimination loss to determine a first overall loss includes: Obtain the product of the first face liveness recognition loss and a first coefficient, and obtain the product of the first forged face discrimination loss and a second coefficient, and take the sum of the two products as the first overall loss, where the first coefficient is the ratio of the first quantity to the first predetermined value, and the second coefficient is the ratio of the second quantity to the first predetermined value.

5. The method according to claim 2, wherein, The training method corresponding to the second training stage includes: For the training of any batch in the second training stage, perform the following second process respectively: Obtain a third quantity of training data from the first training dataset and a fourth quantity of training data from the second training dataset, where the third quantity is greater than the fourth quantity, and the sum of the third quantity and the fourth quantity is equal to a second predetermined value, and the second predetermined value represents the total amount of training data to be input into the first model for this batch; For the training data obtained from the first training dataset, obtain the corresponding second face liveness classification loss, and for the training data obtained from the second training dataset, obtain the corresponding second forged face discrimination loss; Combine the second face liveness classification loss and the second forged face discrimination loss to determine a second overall loss, and update the first model according to the second overall loss; In response to the training reaching convergence of the first model, determine that the second training stage ends; otherwise, for the next batch, repeat the second process.

6. The method according to claim 5, wherein, The combining the second face liveness classification loss and the second forged face discrimination loss to determine a second overall loss includes: Obtain the product of the second face liveness classification loss and the third coefficient, and obtain the product of the second forged face discrimination loss and the fourth coefficient. Take the sum of the two products as the second overall loss. The third coefficient is the ratio of the third quantity to the second predetermined value, and the fourth coefficient is the ratio of the fourth quantity to the second predetermined value.

7. The method according to claim 2, wherein the training method corresponding to the third training stage includes: For any batch of training in the third training stage, perform the following third processing respectively: Obtain a fifth quantity of training data from the first training dataset, and obtain a sixth quantity of training data from the second training dataset. The fifth quantity is less than the sixth quantity, and the sum of the fifth quantity and the sixth quantity is equal to a third predetermined value, where the third predetermined value represents the total amount of training data that needs to be input into the first model for this batch; For the training data obtained from the first training dataset, obtain the corresponding third face liveness classification loss, and for the training data obtained from the second training dataset, obtain the corresponding third forged face discrimination loss; Determine a third overall loss by combining the third face liveness classification loss and the third forged face discrimination loss, and update the first model according to the third overall loss; In response to the training reaching convergence of the first model, determine that the third training stage ends. Otherwise, for the next batch, repeat the third processing.

8. The method according to claim 7, wherein the determining of the third overall loss by combining the third face liveness classification loss and the third forged face discrimination loss includes: Obtain the product of the third face liveness classification loss and the fifth coefficient, and obtain the product of the third forged face discrimination loss and the sixth coefficient. Take the sum of the two products as the third overall loss. The fifth coefficient is the ratio of the fifth quantity to the third predetermined value, and the sixth coefficient is the ratio of the sixth quantity to the third predetermined value.

9. The method according to claim 3, 5 or 7, wherein the ratio of the positive samples to the negative samples in the training data obtained from the first training dataset each time is 1:1, and the ratio of the positive samples to the negative samples in the training data obtained from the second training dataset each time is 1:

1.

10. An image processing apparatus, comprising: An image acquisition module and an image processing module; The image acquisition module is used to acquire an image to be processed; The image processing module is configured to use the image to be processed as the input of a first model, and obtain the output face liveness recognition result and fake face discrimination result. The first model is trained through M training phases based on a first training data set and a second training data set, where M is a positive integer, and different training phases correspond to different training methods. The first training data set is a binary classification data set for face liveness recognition, which includes real person training data as positive samples and attack training data as negative samples. The second training data set is a binary classification data set for fake face discrimination, which includes real person training data as positive samples and fake training data as negative samples. Each training data is image data. Among them, the M training phases include: a first training phase and one or both of the following: a second training phase, a third training phase. The first training phase is a joint training phase for face liveness recognition tasks and fake face discrimination tasks. The second training phase is a training phase that focuses on face liveness recognition tasks and supplements with fake face discrimination tasks. The third training phase is a training phase that supplements face liveness recognition tasks and focuses on fake face discrimination tasks. The second training phase and the third training phase are both executed after the first training phase.

11. A model acquisition device, comprising: A data acquisition module and a model training module; The data acquisition module is configured to acquire a first training data set and a second training data set. The first training data set is a binary classification data set for face liveness recognition, which includes real person training data as positive samples and attack training data as negative samples. The second training data set is a binary classification data set for fake face discrimination, which includes real person training data as positive samples and fake training data as negative samples. Each training data is image data; The model training module is configured to use the first training data set and the second training data set to train a first model through M training phases, where M is a positive integer, and different training phases correspond to different training methods. The first model is used to output the face liveness recognition result and fake face discrimination result corresponding to the image to be processed. Among them, the M training phases include: a first training phase and one or both of the following: a second training phase, a third training phase. The first training phase is a joint training phase for face liveness recognition tasks and fake face discrimination tasks. The second training phase is a training phase that focuses on face liveness recognition tasks and supplements with fake face discrimination tasks. The third training phase is a training phase that supplements face liveness recognition tasks and focuses on fake face discrimination tasks. The second training phase and the third training phase are both executed after the first training phase.

12. According to the apparatus of claim 11, wherein, For any batch of training in the first training stage, the model training module respectively performs the following first processing: obtaining a first quantity of training data from the first training dataset and obtaining a second quantity of training data from the second training dataset, where the first quantity is equal to the second quantity, and the sum of the first quantity and the second quantity is equal to a first predetermined value, and the first predetermined value represents the total amount of training data required for the batch to be input into the first model; For the training data obtained from the first training dataset, obtaining a corresponding first face liveness recognition loss, and for the training data obtained from the second training dataset, obtaining a corresponding first forged face discrimination loss; determining a first overall loss by combining the first face liveness recognition loss and the first forged face discrimination loss, and updating the first model according to the first overall loss; in response to the training reaching convergence of the first model, determining the end of the first training stage, otherwise, for the next batch, repeating the first processing.

13. The apparatus according to claim 12, wherein, The model training module obtains the product of the first face liveness recognition loss and a first coefficient, and obtains the product of the first forged face discrimination loss and a second coefficient, and takes the sum of the two products as the first overall loss, where the first coefficient is the ratio of the first quantity to the first predetermined value, and the second coefficient is the ratio of the second quantity to the first predetermined value.

14. The apparatus according to claim 11, wherein, For any batch of training in the second training stage, the model training module respectively performs the following second processing: obtaining a third quantity of training data from the first training dataset and obtaining a fourth quantity of training data from the second training dataset, where the third quantity is greater than the fourth quantity, and the sum of the third quantity and the fourth quantity is equal to a second predetermined value, and the second predetermined value represents the total amount of training data required for the batch to be input into the first model; For the training data obtained from the first training dataset, obtaining a corresponding second face liveness classification loss, and for the training data obtained from the second training dataset, obtaining a corresponding second forged face discrimination loss; determining a second overall loss by combining the second face liveness classification loss and the second forged face discrimination loss, and updating the first model according to the second overall loss; in response to the training reaching convergence of the first model, determining the end of the second training stage, otherwise, for the next batch, repeating the second processing.

15. The apparatus according to claim 14, wherein, The model training module obtains the product of the second face liveness classification loss and a third coefficient, and obtains the product of the second forged face discrimination loss and a fourth coefficient, and takes the sum of the two products as the second overall loss, where the third coefficient is the ratio of the third quantity to the second predetermined value, and the fourth coefficient is the ratio of the fourth quantity to the second predetermined value.

16. The device according to claim 11, wherein, for the training of any batch in the third training stage, the model training module respectively performs the following third processing: obtaining a fifth quantity of training data from the first training dataset and obtaining a sixth quantity of training data from the second training dataset, the fifth quantity being less than the sixth quantity, and the sum of the fifth quantity and the sixth quantity being equal to a third predetermined value, the third predetermined value representing the total amount of training data that needs to be input into the first model for the batch; for the training data obtained from the first training dataset, obtaining a corresponding third face liveness classification loss, and for the training data obtained from the second training dataset, obtaining a corresponding third forged face discrimination loss; determining a third overall loss by combining the third face liveness classification loss and the third forged face discrimination loss, and updating the first model according to the third overall loss; in response to the training reaching convergence of the first model, determining the end of the third training stage, otherwise, for the next batch, repeating the execution of the third processing.

17. The device according to claim 16, wherein, the model training module obtains the product of the third face liveness classification loss and a fifth coefficient, and obtains the product of the third forged face discrimination loss and a sixth coefficient, and takes the sum of the two products as the third overall loss, the fifth coefficient being the ratio of the fifth quantity to the third predetermined value, and the sixth coefficient being the ratio of the sixth quantity to the third predetermined value.

18. The device according to claim 12, 14 or 16, wherein, the ratio of the positive samples to the negative samples in the training data obtained from the first training dataset each time is 1:1, and the ratio of the positive samples to the negative samples in the training data obtained from the second training dataset each time is 1:

1.

19. An electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to execute the method according to any one of claims 1-9.

20. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause a computer to execute the method according to any one of claims 1-9.

21. A computer program product, comprising computer programs / instructions, and when the computer programs / instructions are executed by a processor, the method according to any one of claims 1-9 is implemented.

Images