Image processing method, device, electronic device and readable storage medium
By cyclically generating adversarial networks to convert fake face images across domains, the problem of incomplete training data is solved, and false face images with fusion features is generated, which improves the detection accuracy and stability of the face live detection model.
Patent Information
- Application Number
- CN202110208227.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-02-24
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2041-02-24
AI Technical Summary
The existing facial data used to train the live facial detection model is not comprehensive enough, resulting in poor detection accuracy.
A circular generation adversarial network is adopted to obtain false face images of different image domains, perform cross-domain conversion and reverse conversion, generate false face images of fused features, expand the data set, and improve the detection accuracy and stability of the model.
By generating false face images with more dispersed and richer distributions in statistical features, the training data is expanded and the detection accuracy and stability of the face live detection model are improved.
Smart Images

Figure CN114973345B_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of image processing technology, and specifically relates to an image processing method, device, electronic device and readable storage medium. Background Art
[0002] With the development and widespread application of face recognition technology, the security of applications that rely on face recognition technology has gradually received attention. At present, face liveness detection models are usually used to determine whether the currently detected face is from a real person or a fake face obtained by using paper-printed photos, videos, or real people wearing masks. However, the face data used to train face liveness detection models, whether from public face liveness detection datasets or self-collected face datasets, have certain limitations and are difficult to include all attack scenarios and fake types, resulting in incomplete training data, which in turn leads to poor detection accuracy of the trained face liveness detection model. Summary of the invention
[0003] The purpose of the embodiments of the present application is to provide an image processing method, device, electronic device and readable storage medium to solve the problem of incomplete face data currently used to train face liveness detection models.
[0004] In order to solve the above technical problems, this application is implemented as follows:
[0005] In a first aspect, an embodiment of the present application provides an image processing method, the method comprising:
[0006] Acquire a plurality of first false face images in a first image domain and a plurality of second false face images in a second image domain; wherein the first false face images have a first feature, the second false face images have a second feature, and the first feature is different from the second feature;
[0007] Input the first fake face image into a first generator of a recurrent generative adversarial network to obtain a first output image, and input the first output image and the first fake face image into a first discriminator of the recurrent generative adversarial network to obtain a first adversarial loss value of the first output image and the first fake face image; and input the second fake face image into a second generator of the recurrent generative adversarial network to obtain a second output image, and input the second output image and the second fake face image into a second discriminator of the recurrent generative adversarial network to obtain a second adversarial loss value of the second output image and the second fake face image; wherein the first generator is used to convert the input first fake face image from the first image domain to the second image domain; and the second generator is used to convert the input second fake face image from the second image domain to the first image domain;
[0008] Inputting the second output image into the first generator, obtaining a third output image that restores the second feature, and calculating a first cycle loss value between the second output image and the third output image; and, inputting the first output image into the second generator, obtaining a fourth output image that restores the first feature, and calculating a second cycle loss value between the first output image and the fourth output image;
[0009] Iteratively training the cyclic generative adversarial network according to the first adversarial loss value, the second adversarial loss value, the first cycle loss value, and the second cycle loss value until the cyclic generative adversarial network converges;
[0010] The converged recurrent generative adversarial network is used to process the false face images in the first image domain and the false face images in the second image domain to generate false face images with a third feature; the third feature is a feature that combines the first feature and the second feature.
[0011] In a second aspect, an embodiment of the present application provides an image processing device, including:
[0012] A first acquisition module is used to acquire a plurality of first false face images in a first image domain and a plurality of second false face images in a second image domain; wherein the first false face images have a first feature, the second false face images have a second feature, and the first feature is different from the second feature;
[0013] A first processing module, used for inputting the first false face image into a first generator of a recurrent generative adversarial network to obtain a first output image, and inputting the first output image and the first false face image into a first discriminator of the recurrent generative adversarial network to obtain a first adversarial loss value of the first output image and the first false face image; and inputting the second false face image into a second generator of the recurrent generative adversarial network to obtain a second output image, and inputting the second output image and the second false face image into a second discriminator of the recurrent generative adversarial network to obtain a second adversarial loss value of the second output image and the second false face image; wherein the first generator is used for converting the input first false face image from the first image domain to the second image domain; and the second generator is used for converting the input second false face image from the second image domain to the first image domain;
[0014] a second processing module, configured to input the second output image into the first generator, obtain a third output image that restores the second feature, and calculate a first cycle loss value between the second output image and the third output image; and, input the first output image into the second generator, obtain a fourth output image that restores the first feature, and calculate a second cycle loss value between the first output image and the fourth output image;
[0015] A training module, configured to iteratively train the cyclic generative adversarial network according to the first adversarial loss value, the second adversarial loss value, the first cycle loss value, and the second cycle loss value until the cyclic generative adversarial network converges;
[0016] A generation module is used to use a converged cyclic generative adversarial network to process the false face image in the first image domain and the false face image in the second image domain to generate a false face image with a third feature; the third feature is a feature that integrates the first feature and the second feature.
[0017] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method described in the first aspect.
[0018] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.
[0019] In an embodiment of the present application, domain-based adaptive false face image enhancement can be achieved through the trained recurrent generative adversarial network, so that the recurrent generative adversarial network can be used to generate false face images with more dispersed and richer statistical feature distribution, and false face images that integrate different attack types or different scenes, environmental characteristics, etc. are obtained, thereby expanding the false face images, making the face data used to train the face liveness detection model comprehensive and rich, and improving the detection accuracy and stability of the trained face liveness detection model. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] Figure 1 is a flow chart of an image processing method provided by an embodiment of the present application;
[0021] Figure 2 Schematic diagram of a recurrent adversarial generative network in an embodiment of the present application;
[0022] Figure 3 is a schematic diagram of a face liveness detection process in a specific embodiment of the present application;
[0023] Figure 4 is a structural schematic diagram of an image processing device provided in an embodiment of the present application;
[0024] Figure 5 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0025] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0026] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.
[0027] In order to solve the problem of incomplete face data currently used to train face liveness detection models, the embodiment of the present application provides a data processing method, namely a domain-based adaptive false face image enhancement method, which uses a recurrent adversarial generative network to generate false face images with more dispersed and richer statistical feature distribution, and obtains false face images that integrate different attack types or different scenes and environmental characteristics, so that the face data used to train face liveness detection models is comprehensive and rich. Furthermore, the newly generated false face images and the original face images can be used to train the face liveness detection model, so as to obtain a face liveness detection model that is robust to new attack types and different scene effects, and improve the accuracy and stability of face liveness detection.
[0028] The image processing method and model training process provided by the embodiments of the present application are described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.
[0029] See also Figure 1 , Figure 1 is a flowchart of an image processing method provided in an embodiment of the present application, the method is applied to electronic devices such as Figure 1 As shown, the method comprises the following steps:
[0030] Step 11: Acquire a plurality of first false face images in the first image domain and a plurality of second false face images in the second image domain.
[0031] In this embodiment, the first image domain and the second image domain are different domains. A domain refers to a collection of images of different attack / false types, or a collection of images of the same attack / false method but collected in different environmental scenes or different devices. Different domains have different image features. Among them, the first false face image has a first feature, and the second false face image has a second feature. The first feature is different from the second feature. It should be pointed out that the first feature and the second feature may be different in lighting, background, attack type, scene style or collection device, etc., and this embodiment does not limit this.
[0032] In one implementation, the first image domain can be used as a source domain, and the second image domain can be used as a target domain. Accordingly, the first fake face image is a source domain image, the first feature is a source domain feature, the second fake face image is a target domain image, and the second feature is a target domain feature. In addition, the first image domain can also be used as a target domain, and the second image domain can be used as a source domain.
[0033] In the embodiment of the present application, in order to obtain false face images of different domains, different scenes can be set, such as different lighting, different backgrounds, different attack types and / or different data acquisition devices, etc., to obtain image data, or randomly extract image frames from discretized video data, and if the obtained image has a face, the face image is cropped out and adjusted to a uniform size, such as 256*256. If the obtained false face images belong to multiple different domains, two domains can be randomly selected, one of which is used as the source domain and the other as the target domain.
[0034] Step 12: Input the first false face image into the first generator of the cyclic generative adversarial network to obtain a first output image, and input the first output image and the first false face image into the first discriminator of the cyclic generative adversarial network to obtain a first adversarial loss value for the first output image and the first false face image; and, input the second false face image into the second generator of the cyclic generative adversarial network to obtain a second output image, and input the second output image and the second false face image into the second discriminator of the cyclic generative adversarial network to obtain a second adversarial loss value for the second output image and the second false face image.
[0035] The first generator is used to convert the input first false face image from the first image domain to the second image domain. The second generator is used to convert the input second false face image from the second image domain to the first image domain.
[0036] Understandable, such as Figure 2 As shown, the cyclic generative adversarial network in the embodiment of the present application is an improvement on the generative adversarial network, including two generators and two discriminators, such as a first generator, a first discriminator, a second generator and a second discriminator. Among them, the first generator (such as generator 1) and the first discriminator (such as discriminator 1) are mainly aimed at the first image domain (such as the source domain), and the second generator (such as generator 2) and the second discriminator (such as discriminator 2) are mainly aimed at the second image domain (such as the target domain). The function of the generator is to generate samples that are very close to the target sample (that is, samples that fuse the first feature and the second feature), and the function of the discriminator is to determine whether a picture is the original picture or the one generated by the corresponding generator. The generator and the discriminator compete with each other and are trained at the same time, and the final model can generate new samples with a high degree of similarity to the target sample.
[0037] Step 13: Input the second output image into the first generator, obtain the third output image that restores the second feature, and calculate the first cycle loss value between the second output image and the third output image; and, input the first output image into the second generator, obtain the fourth output image that restores the first feature, and calculate the second cycle loss value between the first output image and the fourth output image.
[0038] Understandable, such as Figure 2 As shown, the cyclic generative adversarial network in the embodiment of the present application adds a reverse generation process on the basis of the generative adversarial network, that is, the second output image generated by the second generator is input into the first generator to obtain the third output image that restores the second feature, and the first cycle loss value between the second output image and the third output image is calculated. At the same time, the first output image generated by the first generator is input into the second generator to obtain the fourth output image that restores the first feature, and the second cycle loss value between the first output image and the fourth output image is calculated, so as to combine the first cycle loss value and the second cycle loss value to train the cyclic generative adversarial network.
[0039] In one embodiment, the above-mentioned first generator and second generator include convolutional layers, pooling layers, etc. of a convolutional neural network, and the corresponding first discriminator and second discriminator are fully connected layers with different parameters.
[0040] Step 14: Iteratively train the cyclic generative adversarial network according to the first adversarial loss value, the second adversarial loss value, the first cycle loss value, and the second cycle loss value until the cyclic generative adversarial network converges.
[0041] It should be pointed out that the training purpose of the recurrent generative adversarial network in the present application is to generate an image between the first false face image and the second false face image, the characteristics of which are a fusion of the first and second characteristics, but the image does not belong to the first image domain, nor to the second image domain, but to a new image domain, and the characteristics of the false face images in the new image domain are a fusion of the first and second characteristics.
[0042] Step 15: Using the converged cyclic generative adversarial network, the false face image in the first image domain and the false face image in the second image domain are processed to generate a false face image with a third feature; the third feature is a feature that integrates the first feature and the second feature.
[0043] It should be pointed out that if there are multiple image domains, two image domains can be randomly selected each time as the source domain and the target domain, and the above steps can be repeated to generate a new fake face image that fuses different source domains and target domains.
[0044] In an embodiment of the present application, domain-based adaptive false face image enhancement can be achieved through the trained recurrent generative adversarial network, so that the recurrent generative adversarial network can be used to generate false face images with more dispersed and richer statistical feature distribution, and false face images that integrate different attack types or different scenes, environmental characteristics, etc. are obtained, thereby expanding the false face images, making the face data used to train the face liveness detection model more comprehensive and rich, and improving the detection accuracy and stability of the trained face liveness detection model.
[0045] In the embodiment of the present application, the loss function of the above-mentioned cyclic adversarial generative network can be:
[0046]
[0047] in,
[0048]
[0049] in, represents the adversarial loss function corresponding to the first adversarial loss value, that is, the adversarial loss function between the first output image and the first fake face image. s G represents the first false face image, that is, the false face image in the first image domain. ST (x s ,z) represents the first output image, i.e., the output image of the first generator converted from the first image domain to the second image domain. S (x s ) represents the discrimination result for the first false face image, that is, the discrimination result for the false face image from the first image domain, for example, a probability value. D S (G ST (x s ,z)) represents the discrimination result for the first output image, that is, the discrimination result for the output image of the first discriminator, such as a probability value. S Represents the distribution of fake face images with the first feature. z represents a variable, which can be input into the recurrent adversarial generative network together with the first fake face image and the second fake face image, and is used to adjust the proportion of the first feature and the second feature in the third feature, that is, to control the relationship between the newly generated fake face image and the first fake face image and the second fake face image. It can be understood that during the training of the recurrent adversarial generative network, the variable z can be uniformly sampled between [0,1]. However, in the process of training a recurrent adversarial generative network, it is necessary to fix z, and then alternately train and update the generator and the discriminator until convergence.
[0050] represents the adversarial loss function corresponding to the second adversarial loss value, that is, the adversarial loss function between the second output image and the second fake face image.t G represents the second false face image, that is, the false face image in the first image domain. TS (x t ,z) represents the second output image, i.e., the output image of the second generator converted from the second image domain to the first image domain. T (x t ) represents the discrimination result for the second false face image, that is, the discrimination result for the false face image from the second image domain, for example, a probability value. T (G TS (x t ,z)) represents the discrimination result for the second output image, that is, the discrimination result for the output image of the second discriminator, such as a probability value. T Represents the distribution of fake face images with the second feature.
[0051] L cyc1 L represents the loop loss function corresponding to the first loop loss value, that is, the loop loss function corresponding to the reverse loop process corresponding to the first generator. cyc2 G represents the loop loss function corresponding to the second loop loss value, that is, the loop loss function corresponding to the reverse loop process corresponding to the first generator. TS (G ST (x s ,z),z) represents the fourth output image, that is, the output image obtained by using the first generator to restore the second feature of the second output image of the second generator. G ST (G TS (x t ,z),z) represents the third output image, i.e., the output image obtained by restoring the first feature of the first output image of the first generator using the second generator. λ represents a coefficient, which is used to balance the adversarial loss and the cycle loss during training. λ is a hyperparameter.
[0052] In an embodiment of the present application, after a new false face image is generated by fusion, it can be combined with the original face image to train a face liveness detection model to obtain a face liveness detection model that is robust to new attack types and different scene effects, thereby improving the accuracy and stability of face liveness detection.
[0053] Optionally, the face liveness detection model in the present application can be a classification neural network, for example, including five convolutional pooling layers, and three fully connected layers, etc. The classification loss function of the face liveness detection model can be selected as a cross entropy loss function, but the embodiment of the present application does not limit the specific form of this cross entropy loss function, which can be selected based on actual needs.
[0054] In order to improve the liveness detection performance of the face liveness detection model, in the face liveness detection model of the present application, clustering of false face images can be used to make the model pay attention to more subtle features in the false face, so that the face liveness detection model can better distinguish between false face images and real face images, and enhance the liveness detection performance. For example, the feature vector extracted by the face liveness detection model is clustered using the KMenas clustering method to obtain pseudo labels of the false face image data, and then combined with the real label information of the false face image data, the standard mutual information of the clustering is calculated to design the clustering loss function.
[0055] Optionally, the image processing method in the present application may further include:
[0056] Acquire a first image set and a second image set; wherein the first image set includes a plurality of third false face images in the first image domain and a plurality of fourth false face images in the second image domain; and the second image set includes a plurality of real face images;
[0057] Inputting the plurality of third false face images and the plurality of fourth false face images into the converged recurrent generative adversarial network to generate a plurality of fifth false face images having the third feature;
[0058] The face liveness detection model is trained by using the false face images in the first image set, the real face images in the second image set, and a plurality of fifth false face images.
[0059] It is understandable that when training the face liveness detection model, the loss value of the loss function of the face liveness detection model can be continuously minimized to converge to obtain the face liveness detection model. In this way, by combining the original face image and the new fake face image generated by fusion to train the face liveness detection model, a face liveness detection model that is robust to new attack types and different scene effects can be obtained, thereby improving the accuracy and stability of face liveness detection.
[0060] In one implementation, the acquisition process of the first image set and the second image set may be: first, collect a number of real-person images and videos, as well as attack images and videos, to form a data set; second, for each video, discretize the video and extract the image frames; then, pre-process all real-person image data and false image data, such as performing face detection using the MTCNN algorithm, and cropping the detected faces; finally, uniformly adjust the cropped images to a size of 256*256 to form the first image set and the second image set.
[0061] Optionally, the loss function of the face liveness detection model in this application can be:
[0062]
[0063] in,
[0064]
[0065]
[0066] in, Represents the classification loss function of the face liveness detection model. For example, the classification loss function can be selected as an existing cross entropy loss function without restriction.
[0067] in, represents the clustering loss function of the face liveness detection model; α represents The coefficient of Ω represents the clustering estimation result of the features of the false face image extracted by the face liveness detection model, and C represents the clustering true result of the features of the corresponding false face image (such as the true label information); I(Ω; C) represents the mutual information value between the clustering estimation result and the clustering true result, H(Ω) represents the information entropy of the clustering estimation result, H(C) represents the information entropy of the clustering true result, and NMI(Ω, C) represents the standardized mutual information value between the clustering estimation result and the clustering true result. NMI(Ω, C) can measure the clustering result. The larger the standardized mutual information value, the better the clustering result.
[0068] For example, the features of the fake face image extracted by the face liveness detection model can be feature vectors extracted by a neural network feature generator. For example, the face liveness detection model can be a ResNet18 network, and three fully connected layers are set after the last average pooling of the ResNet18 network, the first fully connected layer is used as a feature generator, the next two fully connected layers are used as classifiers, and the features extracted by the first fully connected layer are used for KMeans clustering.
[0069] Furthermore, after the face liveness detection model is trained, a face image to be detected can be obtained, and the face liveness detection model can be used to detect whether the face image to be detected is a false face image, that is, to detect whether the face image to be detected is a real face image or a false face image.
[0070] Next, combine Figure 3 The face liveness detection process in the specific example of this application is explained.
[0071] like Figure 3 As shown, the face liveness detection process in the specific example of the present application may include:
[0072] S1: Obtaining pre-processed face image data, where the pre-processing includes, for example, adjusting the face images to a uniform size and dividing them into real face images and false face images.
[0073] S2: For fake face images, they are divided into different image domains according to attack types or scenarios, and image domains with different image features are randomly selected as source domains and target domains to train the recurrent adversarial generative network. The training process can be found in the above content and will not be repeated here.
[0074] S3: Use the trained recurrent adversarial generative network to generate new fake face images, such as fake faces with new attack methods or fake faces in new environments or scene styles.
[0075] S4: Combine the real face image and the fake face image in S1, and the new fake face image generated in S3 to train the face liveness detection model. The training process can be found in the above content and will not be repeated here.
[0076] S5: Input the test face image (i.e., the face image to be detected) into the trained face liveness detection model, obtain a test result of whether the test face image is a false face image or not, and output the test result.
[0077] See also Figure 4 , Figure 4 is a schematic diagram of the structure of an image processing device provided in an embodiment of the present application, and the device is applied to electronic devices such as Figure 4 As shown, the image processing device 40 includes:
[0078] A first acquisition module 41 is used to acquire a plurality of first false face images in a first image domain and a plurality of second false face images in a second image domain; wherein the first false face images have a first feature, the second false face images have a second feature, and the first feature is different from the second feature;
[0079] The first processing module 42 is used to input the first false face image into the first generator of the recurrent generative adversarial network to obtain a first output image, and input the first output image and the first false face image into the first discriminator of the recurrent generative adversarial network to obtain a first adversarial loss value of the first output image and the first false face image; and input the second false face image into the second generator of the recurrent generative adversarial network to obtain a second output image, and input the second output image and the second false face image into the second discriminator of the recurrent generative adversarial network to obtain a second adversarial loss value of the second output image and the second false face image; wherein the first generator is used to convert the input first false face image from the first image domain to the second image domain; and the second generator is used to convert the input second false face image from the second image domain to the first image domain;
[0080] A second processing module 43 is used to input the second output image into the first generator, obtain a third output image that restores the second feature, and calculate a first cycle loss value between the second output image and the third output image; and input the first output image into the second generator, obtain a fourth output image that restores the first feature, and calculate a second cycle loss value between the first output image and the fourth output image;
[0081] A training module 44, configured to iteratively train the cyclic generative adversarial network according to the first adversarial loss value, the second adversarial loss value, the first cycle loss value, and the second cycle loss value until the cyclic generative adversarial network converges;
[0082] The generation module 45 is used to use the converged cyclic generative adversarial network to process the false face image in the first image domain and the false face image in the second image domain to generate a false face image with a third feature; the third feature is a feature that integrates the first feature and the second feature.
[0083] Optionally, the loss function of the cyclic generative adversarial network is:
[0084]
[0085] in,
[0086]
[0087]
[0088] in, represents the adversarial loss function corresponding to the first adversarial loss value, x s represents the first false face image, G ST (x s , z) represents the first output image, D S (x s ) represents the discrimination result for the first false face image, D S (G ST (x s , z)) represents the discrimination result for the first output image; P S represents the distribution of false face images having the first feature; z represents a variable used to adjust the proportion of the first feature and the second feature in the third feature;
[0089] represents the adversarial loss function corresponding to the second adversarial loss value, x t represents the second false face image, GTS (x t , z) represents the second output image, D T (x t ) represents the discrimination result for the second false face image, D T (G TS (x t , z)) represents the discrimination result for the second output image; P T represents the distribution of false face images having the second feature;
[0090] L cyc1 represents the loop loss function corresponding to the first loop loss value, L cyc2 represents the loop loss function corresponding to the second loop loss value, G TS (G ST (x s ,z),z) represents the fourth output image, G ST (G TS (x t ,z),z) represents the third output image; λ represents the coefficient.
[0091] Optionally, the first acquisition module 41 is further used to acquire a first image set and a second image set; wherein the first image set includes a plurality of third false face images in the first image domain and a plurality of fourth false face images in the second image domain; and the second image set includes a plurality of real face images;
[0092] The generating module 45 is further configured to input the plurality of third false face images and the plurality of fourth false face images into the converged recurrent generative adversarial network to generate a plurality of fifth false face images having the third feature;
[0093] The training module 44 is further configured to train a live face detection model using the false face images in the first image set, the real face images in the second image set, and the plurality of fifth false face images.
[0094] Optionally, the loss function of the face liveness detection model is:
[0095]
[0096] in,
[0097]
[0098]
[0099] in, Represents the classification loss function of the face liveness detection model;
[0100] represents the clustering loss function of the face liveness detection model; α represents coefficient; Ω represents the clustering estimation result of the features of the false face image extracted by the face liveness detection model, and C represents the clustering true result of the features of the false face image; I(Ω; C) represents the mutual information value of the clustering estimation result and the clustering true result, H(Ω) represents the information entropy of the clustering estimation result, H(C) represents the information entropy of the clustering true result, and NMI(Ω,C) represents the standardized mutual information value of the clustering estimation result and the clustering true result.
[0101] Optionally, the device further comprises:
[0102] The second acquisition module is used to acquire the face image to be detected;
[0103] The detection module is used to use the face liveness detection model to detect whether the face image to be detected is a false face image.
[0104] It is understandable that the image processing device 40 of the embodiment of the present application can implement the above Figure 1 The various processes of the method embodiment shown can achieve the same technical effect, and will not be described again here to avoid repetition.
[0105] It should be noted that the division of units in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional unit in each embodiment of the present application may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0106] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a processor-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.
[0107] In addition, an embodiment of the present application also provides an electronic device, which includes a processor 51, a memory 52, and a program or instruction stored in the memory 52 and executable on the processor 51. When the program or instruction is executed by the processor 51, each process of the above-mentioned image processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0108] The embodiment of the present application also provides a readable storage medium on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned image processing method embodiment can be implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0109] Computer readable media include permanent and non-permanent, removable and non-removable media, and can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0110] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.
[0111] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.
[0112] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a service classification device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.
[0113] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.
Claims
1. An image processing method, characterized in that: include: Acquire a plurality of first false face images in a first image domain and a plurality of second false face images in a second image domain; wherein the first false face images have a first feature, the second false face images have a second feature, and the first feature is different from the second feature; the first image domain is a set of false face images having the first feature, and the second image domain is a set of false face images having the second feature; Input the first fake face image into a first generator of a recurrent generative adversarial network to obtain a first output image, and input the first output image and the first fake face image into a first discriminator of the recurrent generative adversarial network to obtain a first adversarial loss value of the first output image and the first fake face image; and input the second fake face image into a second generator of the recurrent generative adversarial network to obtain a second output image, and input the second output image and the second fake face image into a second discriminator of the recurrent generative adversarial network to obtain a second adversarial loss value of the second output image and the second fake face image; wherein the first generator is used to convert the input first fake face image from the first image domain to the second image domain; and the second generator is used to convert the input second fake face image from the second image domain to the first image domain; Inputting the second output image into the first generator, obtaining a third output image that restores the second feature, and calculating a first cycle loss value between the second output image and the third output image; and, inputting the first output image into the second generator, obtaining a fourth output image that restores the first feature, and calculating a second cycle loss value between the first output image and the fourth output image; Iteratively training the cyclic generative adversarial network according to the first adversarial loss value, the second adversarial loss value, the first cycle loss value, and the second cycle loss value until the cyclic generative adversarial network converges; Using the converged recurrent generative adversarial network, the false face image in the first image domain and the false face image in the second image domain are processed to generate a false face image with a third feature; the third feature is a feature that combines the first feature and the second feature; Wherein, the method further comprises: Acquire a first image set and a second image set; wherein the first image set includes a plurality of third false face images in the first image domain and a plurality of fourth false face images in the second image domain; and the second image set includes a plurality of real face images; The method of using the converged recurrent generative adversarial network to process the false face images in the first image domain and the false face images in the second image domain to generate false face images with a third feature comprises: inputting the plurality of third false face images and the plurality of fourth false face images into the converged recurrent generative adversarial network to generate a plurality of fifth false face images with the third feature; After generating a plurality of fifth false face images, the method further includes: Using the false face images in the first image set, the real face images in the second image set, and the plurality of fifth false face images, a face liveness detection model is trained; Among them, the loss function of the face liveness detection model is: in, in, Represents the classification loss function of the face liveness detection model; represents the clustering loss function of the face liveness detection model; α represents coefficient; Ω represents the clustering estimation result of the features of the false face image extracted by the face liveness detection model, and C represents the clustering true result of the features of the false face image; I(Ω; C) represents the mutual information value of the clustering estimation result and the clustering true result, H(Ω) represents the information entropy of the clustering estimation result, H(C) represents the information entropy of the clustering true result, and NMI(Ω,C) represents the standardized mutual information value of the clustering estimation result and the clustering true result.
2. The method according to claim 1, characterized in that The loss function of the cyclic generative adversarial network is: in, in, represents the adversarial loss function corresponding to the first adversarial loss value, x s represents the first false face image, G ST (x s , z) represents the first output image, D S (x s ) represents the discrimination result for the first false face image, D S (H ST (x s , z)) represents the discrimination result for the first output image; P S represents the distribution of false face images having the first feature; z represents a variable used to adjust the proportion of the first feature and the second feature in the third feature; represents the adversarial loss function corresponding to the second adversarial loss value, x t represents the second false face image, G TS (x t , z) represents the second output image, D T (x t ) represents the discrimination result for the second false face image, D T (G TS (x t , z)) represents the discrimination result for the second output image; P T represents the distribution of false face images having the second feature; L cyc1 represents the loop loss function corresponding to the first loop loss value, L cyc2 represents the loop loss function corresponding to the second loop loss value, G TS (G ST (x s ,z),z) represents the fourth output image, G ST (G TS (x t ,z),z) represents the third output image; λ represents the coefficient.
3. The method according to claim 1, characterized in that The method further comprises: Get the face image to be detected; The face liveness detection model is used to detect whether the face image to be detected is a false face image.
4. An image processing device, characterized in that: include: A first acquisition module is used to acquire a plurality of first false face images in a first image domain and a plurality of second false face images in a second image domain; wherein the first false face images have a first feature, the second false face images have a second feature, and the first feature is different from the second feature; the first image domain is a set of false face images having the first feature, and the second image domain is a set of false face images having the second feature; A first processing module, used for inputting the first false face image into a first generator of a recurrent generative adversarial network to obtain a first output image, and inputting the first output image and the first false face image into a first discriminator of the recurrent generative adversarial network to obtain a first adversarial loss value of the first output image and the first false face image; and inputting the second false face image into a second generator of the recurrent generative adversarial network to obtain a second output image, and inputting the second output image and the second false face image into a second discriminator of the recurrent generative adversarial network to obtain a second adversarial loss value of the second output image and the second false face image; wherein the first generator is used for converting the input first false face image from the first image domain to the second image domain; and the second generator is used for converting the input second false face image from the second image domain to the first image domain; a second processing module, configured to input the second output image into the first generator, obtain a third output image that restores the second feature, and calculate a first cycle loss value between the second output image and the third output image; and, input the first output image into the second generator, obtain a fourth output image that restores the first feature, and calculate a second cycle loss value between the first output image and the fourth output image; A training module, configured to iteratively train the cyclic generative adversarial network according to the first adversarial loss value, the second adversarial loss value, the first cycle loss value, and the second cycle loss value until the cyclic generative adversarial network converges; A generation module, configured to process the false face image in the first image domain and the false face image in the second image domain by using a converged cyclic generative adversarial network to generate a false face image having a third feature; the third feature is a feature that combines the first feature and the second feature; Wherein, the first acquisition module is further used to acquire a first image set and a second image set; the first image set includes a plurality of third false face images in the first image domain and a plurality of fourth false face images in the second image domain; the second image set includes a plurality of real face images; The generating module is further used to input the plurality of third false face images and the plurality of fourth false face images into the converged recurrent generative adversarial network to generate a plurality of fifth false face images having the third feature; The training module is further used to train a face liveness detection model using the false face images in the first image set, the real face images in the second image set, and the plurality of fifth false face images; Among them, the loss function of the face liveness detection model is: in, Represents the classification loss function of the face liveness detection model; represents the clustering loss function of the face liveness detection model; α represents coefficient; Ω represents the clustering estimation result of the features of the false face image extracted by the face liveness detection model, and C represents the clustering true result of the features of the false face image; I(Ω; C) represents the mutual information value of the clustering estimation result and the clustering true result, H(Ω) represents the information entropy of the clustering estimation result, H(C) represents the information entropy of the clustering true result, and NMI(Ω,C) represents the standardized mutual information value of the clustering estimation result and the clustering true result.
5. The device according to claim 4, characterized in that The loss function of the cyclic generative adversarial network is: in, in, represents the adversarial loss function corresponding to the first adversarial loss value, x s represents the first false face image, G ST (x s , z) represents the first output image, D S (x s ) represents the discrimination result for the first false face image, D S (G ST (x s , z)) represents the discrimination result for the first output image; P S represents the distribution of false face images having the first feature; z represents a variable used to adjust the proportion of the first feature and the second feature in the third feature; represents the adversarial loss function corresponding to the second adversarial loss value, x t represents the second false face image, G TS (x t , z) represents the second output image, D T (x t ) represents the discrimination result for the second false face image, D T (G TS (x t , z)) represents the discrimination result for the second output image; P T represents the distribution of false face images having the second feature; L cyc1 represents the loop loss function corresponding to the first loop loss value, L cyc2 represents the loop loss function corresponding to the second loop loss value, G TS (G ST (x s ,z),z) represents the fourth output image, G ST (G TS (x t ,z),z) represents the third output image; λ represents the coefficient.
6. The device according to claim 4, characterized in that The device also includes: The second acquisition module is used to acquire the face image to be detected; The detection module is used to use the face liveness detection model to detect whether the face image to be detected is a false face image.
7. An electronic device, characterized in that: The method comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the image processing method as described in any one of claims 1 to 3.
8. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the image processing method according to any one of claims 1 to 3 are implemented.
