Face image generation method and apparatus, computer-readable medium, and electronic device

Abstract

The present disclosure provides a face image generation method and device, a computer readable medium and an electronic device, and relates to the technical field of artificial intelligence. The method comprises: obtaining an original face image containing real identity information, and inputting the original face image into a pre-trained face anonymization model to generate a target face image, the target face image not containing real identity information, and the similarity between the target face image and the original face image being greater than or equal to a similarity threshold. The present disclosure directly generates an anonymized target face image through the face anonymization model, the image quality of the obtained target face image is higher, the image details are clearer, and the face anonymization model does not need to decouple features, and the computational complexity is smaller.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of artificial intelligence, in particular to a face image generation method, a face image generation device, a computer readable medium and an electronic device. BACKGROUND

[0002] With the continuous development of science and technology, face recognition technology has been more and more widely used. Face recognition technology refers to a biometric technology based on face feature information for identity recognition. However, with the popularization of face recognition technology, the risk of leakage of personal privacy information contained in face images has also been increasingly valued by people.

[0003] At present, in related face image anonymization schemes, face regions in original face images are generally replaced to achieve anonymization. However, the face replacement method will have obvious boundaries, poor image quality, and may cause problems such as unrealistic face images after anonymization, dissimilarity with original face images, and poor user experience. SUMMARY

[0004] The purpose of the present disclosure is to provide a face image generation method, a face image generation device, a computer readable medium and an electronic device, thereby at least to some extent improving the image quality of the target face image obtained after anonymization.

[0005] According to a first aspect of the present disclosure, a face image generation method is provided, comprising:

[0006] obtaining an original face image containing real identity information;

[0007] inputting the original face image into a pre-trained face anonymization model to generate a target face image;

[0008] wherein the target face image does not contain the real identity information, and the similarity between the target face image and the original face image is greater than a similarity threshold.

[0009] According to a second aspect of the present disclosure, a face image generation device is provided, comprising:

[0010] an original face image obtaining module configured to obtain an original face image containing real identity information;

[0011] a target face image generating module configured to input the original face image into a pre-trained face anonymization model to generate a target face image;

[0012] wherein the target face image does not contain the real identity information, and the similarity between the target face image and the original face image is greater than a similarity threshold.

[0013] According to a third aspect of the present disclosure, a computer readable medium is provided, having stored thereon a computer program which, when executed by a processor, implements the method described above.

[0014] According to a fourth aspect of the present disclosure, an electronic device is provided, characterized by comprising:

[0015] a processor; and

[0016] a memory for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to implement the method described above.

[0017] The face image generation method provided by an embodiment of the present disclosure can input an original face image containing real identity information into a pre-trained face anonymization model to directly generate a target face image that does not contain real identity information and has a similarity greater than a similarity threshold with the original face image. On the one hand, the target face image is directly generated by the face anonymization model, avoiding the problem of unrealistic face images and poor image quality caused by replacing face regions, and effectively improving the image quality of the target face image. On the other hand, while hiding the real identity information contained in the original face image, the generated target face image is ensured to be as similar as possible to the original face image, without losing the face display features in the original face image, and improving the user experience.

[0018] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0019] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the specification, serve to explain the principles of the present disclosure. It is obvious that the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative labor. In the drawings:

[0020] Figure 1 a schematic diagram showing an exemplary system architecture to which embodiments of the present disclosure can be applied;

[0021] Figure 2 a flowchart schematically showing a face image generation method in an exemplary embodiment of the present disclosure;

[0022] Figure 3 a flowchart schematically showing a model training process of a face anonymization model in an exemplary embodiment of the present disclosure;

[0023] Figure 4A schematic diagram of a principle of a face anonymization model in an example embodiment of the present disclosure is shown.

[0024] Figure 5 A schematic diagram of a process of constructing a distance loss function in an example embodiment of the present disclosure is shown.

[0025] Figure 6 A schematic diagram of a process of training a face anonymization model in an example embodiment of the present disclosure is shown.

[0026] Figure 7 A schematic diagram of another process of generating a face image in an example embodiment of the present disclosure is shown.

[0027] Figure 8 A schematic diagram of a face image generation apparatus in an example embodiment of the present disclosure is shown.

[0028] Figure 9 A schematic diagram of an electronic device to which an embodiment of the present disclosure can be applied is shown. DETAILED DESCRIPTION

[0029] Example implementations will now be described more fully with reference to the accompanying drawings. Example implementations may, however, be implemented in many different forms and should not be construed as limited to the implementations set forth herein; rather, these implementations are provided so that this disclosure will be thorough and complete, and will fully convey the scope of example implementations to those skilled in the art. Features described in one implementation can be combined with features described in another implementation.

[0030] In addition, the accompanying drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments of the present disclosure and serve to explain the principles of the present disclosure. Like reference numerals refer to like elements throughout.

[0031] Figure 1 A schematic diagram of a system architecture of an example application environment of a face image generation method and apparatus to which an embodiment of the present disclosure can be applied is shown.

[0032] As Figure 1As shown, the system architecture 100 may include one or more terminal devices 101, 102, 103, a network 104, a server 105, and a server cluster 106. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105 or the server cluster 106. The network 104 may include various connection types, such as wired, wireless communication links or fiber optic cables, etc. The terminal devices 101, 102, 103 may be various electronic devices with artificial intelligence (AI) processors, including but not limited to edge computing devices 101, smart phones 102, and intelligent service robots 103, etc. It should be understood that Figure 1 The number of terminal devices, networks and servers in the figure is merely illustrative, and any number of terminal devices, networks and servers or server clusters may be provided according to implementation requirements.

[0033] The facial image generation method provided in the embodiments of the present disclosure is generally executed by the server 105 or the server cluster 106, and accordingly, the facial image generation device is generally disposed in the server 105 or the server cluster 106. However, it will be readily understood by those skilled in the art that the facial image generation method provided in the embodiments of the present disclosure may also be executed by the terminal devices 101, 102, and 103, and accordingly, the facial image generation device may also be disposed in the terminal devices 101, 102, and 103, and this is not particularly limited in this exemplary embodiment.

[0034] For example, in an exemplary embodiment, a user may upload an original facial image containing real identity information to server 105 or server cluster 106 through terminal devices 101, 102, and 103. Server 105 or server cluster 106 anonymizes the original facial image using the facial image generation method provided in the embodiment of the present disclosure, and then transmits the generated target facial image to terminal devices 101, 102, 103, etc.

[0035] A related technical solution uses a generative adversarial network structure, trained by establishing objective functions for anonymizing facial regions and preserving scene content regions. During the model inference phase, facial anonymity is achieved by replacing the facial regions in the image with synthetic faces. This approach is more efficient and visually friendly than mosaic occlusion. However, this technical solution, which achieves facial anonymization by replacing the facial regions in the original image, may result in the replaced facial image not being realistic enough, and a clear boundary between the replaced facial region and the original image may be visible, resulting in poor image quality for the generated facial image.

[0036] Another related technical solution achieves face anonymization by decoupling identity representation information from other attribute information. Specifically, a codec neural network model with identity decoupling capabilities is first trained. This model is then used to decouple existing facial features to obtain identity information and other attribute information. While keeping the attribute information unchanged, the identity information is manipulated to achieve the purpose of anonymizing the facial image. This anonymized facial image, while maintaining a high degree of similarity to the original image, cannot be detected as the original identity by existing facial recognition systems, thereby effectively protecting the privacy of the original facial image holder. However, this technical solution uses feature decoupling to achieve anonymization, requiring at least two subnetworks: an encoding network for feature extraction and decoupling, and a decoding network for generating faces from the encoding. The facial images generated in this way are relatively blurry and lack many details compared to the original facial images, resulting in poor image quality and a poor user experience.

[0037] Based on one or more problems in the related art, the present disclosure first provides a facial image generation method and facial image generation device. The following uses a server or server cluster as an example to specifically describe the facial image generation method and facial image generation device in an exemplary embodiment of the present disclosure.

[0038] Figure 2 The flow of a facial image generation method in this exemplary embodiment is shown, which may include step S210 and step S220:

[0039] In step S210, an original face image containing real identity information is obtained.

[0040] In an exemplary embodiment, the original facial image is a facial image that has not been processed in any way. For example, the original facial image can be a facial image collected by a community access control or surveillance camera, or a facial image collected by facial payment software. The original facial image can be collected in real time or non-real time. This exemplary embodiment does not specifically limit this.

[0041] Real identity information refers to unique identification information corresponding to facial features in the original facial image. For example, the real identity information can be the ID number or name corresponding to the face in the original facial image, or it can be a special code corresponding to the face in the original facial image. This example embodiment does not specifically limit this. The real identity information can be used to uniquely identify a person in real life.

[0042] In step S220, the original face image is input into a pre-trained face anonymization model to generate a target face image.

[0043] In one exemplary embodiment, a face anonymization model refers to a pre-trained neural network model that can modify facial features in an original face image. For example, the face anonymization model can be built based on convolutional neural networks (CNNs). Specifically, the face anonymization model in this embodiment can include at least a face recognition network (FaceRecog), a face manipulation network (ManipulationCNN), and an adversarial discriminant network (D).

[0044] The target facial image refers to a facial image obtained by modifying the facial features in the original facial image. The target facial image does not contain the real identity information in the original facial image, that is, when facial recognition is performed on the target facial image, the output facial recognition result is a null value, or virtual identity information is output, where the virtual identity information refers to the unique identification information corresponding to the facial features in the target facial image, but the virtual identity information cannot be used to identify a person in real life.

[0045] At the same time, the similarity between the target facial image and the original facial image is greater than the similarity threshold. For example, the similarity between the target facial image and the original facial image can be determined by histogram matching, or by image similarity calculation based on feature points. This example embodiment does not specifically limit the method of determining the similarity between the target facial image and the original facial image.

[0046] The similarity threshold refers to a numerical value used to assess whether the target facial image and the original facial image are visually similar. For example, the similarity threshold can be 80%. Then, if the similarity between the target facial image and the original facial image is 90%, it can be considered that the target facial image and the original facial image are visually similar. Of course, the similarity threshold can also be 90%, and the specific setting needs to be customized according to the actual usage scenario. This example embodiment does not specifically limit this.

[0047] In an example application scenario, when a user object undergoes facial recognition at a community access control, the community access control face recognition device collects the user object's face image and uploads the user object's face image to the server. The server inputs the user object's face image into the trained face anonymization model to generate a virtual face image with virtual identity information, and returns the virtual face image to the community access control face recognition device, which displays the virtual face image corresponding to the user object on the display interface. This not only effectively protects the user object's identity privacy information and prevents others from taking photos for face recognition in other scenarios, but also the displayed virtual face image is similar to the user object's real face image, without affecting the user's experience.

[0048] Next, the contents in steps S210 and S220 are expanded.

[0049] In an exemplary embodiment, before converting the original face image into the target face image by the pre-trained face anonymization model, the initial face anonymization model can be pre-trained to obtain the trained face anonymization model.

[0050] Specifically, a training sample batch can be obtained first, and the training sample batch can include at least two sample original face images corresponding to each real identity information. For example, assuming that there are real identity information 001 and real identity information 002, for the training sample batch, at least two sample original face images of real identity information 001 and two sample original face images of real identity information 002 are required. Of course, the number of sample original face images under each real identity information can be set according to the computing capability of the terminal or the server. The more sample original face images under each real identity information, the stronger the robustness of the final face anonymization model. This example embodiment does not make special limitations.

[0051] The existing real face training set can be obtained, and then sampling is performed according to the real identity information in the real face training set to realize the construction of the training sample batch. In the sampling process, it is ensured that each real identity information corresponds to at least two sample original face images in each training sample batch.

[0052] The initial face anonymization model can be trained by the sample original face images in the training sample batch until the sample virtual face image output by the initial face anonymization model does not contain the real identity information corresponding to the sample original face image, and the similarity between the sample virtual face image and the sample original face image is greater than or equal to the similarity threshold. It can be considered that the model training of the face anonymization model is completed, and the trained face anonymization model is obtained.

[0053] In an exemplary embodiment, a face anonymization model may include at least a face manipulation network, a face recognition network, and an adversarial discriminant network; wherein the face manipulation network may be used to modify and adjust the facial features of a sample original face image or an original face image to generate a sample virtual face image or a target face image; the face recognition network may discriminate the distance between each sample target face image or a target face image, and is mainly used to supervise the face manipulation network. The face recognition network ensures through a distance loss function that the distance between different samples within the same virtual identity information generated by the face manipulation network is as small as possible, the sample distance between different virtual identity information is as large as possible, and the distance between the generated virtual identity information and other real identity information is as large as possible to avoid editing one real identity information into another real identity information; the adversarial discriminant network may supervise the face manipulation network to make the sample virtual face image or the target face image generated by the face manipulation network as similar as possible to the sample original face image or the original face image.

[0054] Optionally, you can pass Figure 3 The steps in the implementation of the initial face anonymization model training. Figure 3 Specifically, it may include:

[0055] Step S310, inputting the sample original face image into the face manipulation network to obtain a sample virtual face image with virtual identity information;

[0056] Step S320, determining a distance loss function using the sample original face image, the sample virtual face image, and the face recognition network;

[0057] Step S330, determining an adversarial loss function using the sample original face image, the sample virtual face image, and the adversarial discriminant network;

[0058] Step S340: Adjust the network weights of the face manipulation network based on the distance loss function and the adversarial loss function to implement model training of the face manipulation network.

[0059] Among them, the face manipulation network can be used to modify and adjust the facial features in the sample original face image to obtain the neural network structure of the sample virtual face image. For example, the face manipulation network may include but is not limited to a fully convolutional network (FCN). Through the face manipulation network based on the fully convolutional network structure, the details and background in the input face image can be better retained, and the image quality of the generated face image can be effectively improved.

[0060] A face recognition network refers to a neural network structure used to extract features from sample original face images or sample virtual face images, and to determine the distance between each sample target face image or target face images through a constructed distance loss function. For example, a face recognition network may include but is not limited to a FaceRecogKNN network.

[0061] The image features in the sample original face image or the sample virtual face image can be extracted through the face recognition network to obtain a feature matrix, and the distance between the sample original face image and the generated sample virtual face image, and between the sample virtual face image and other sample virtual face images can be measured through the distance loss function, so as to ensure that the distance between different samples in the same virtual identity information generated by the face manipulation network is as small as possible, the sample distance between different virtual identity information is as large as possible, and the distance between the generated virtual identity information and other real identity information is ensured to be as large as possible, so as to avoid editing one real identity information into another real identity information.

[0062] An adversarial loss function can be constructed using an adversarial discriminant network. Specifically, the sample original face image and the generated sample virtual face image can be input into the adversarial discriminant network separately. The network weights are trained using the classification loss function (e.g., classifying the sample original face image into category 0 and the sample virtual face image into category 1). The generated image is then input alone to construct the adversarial loss function (i.e., classifying the sample virtual face image into category 0).

[0063] The distance loss obtained by the distance loss function constructed by the face recognition network and the adversarial loss obtained by the adversarial loss function constructed by the adversarial discriminant network can be used to jointly train and adjust the network weights of the face manipulation network. The training goal is to ensure that the sample virtual face images generated by the face manipulation network do not contain the real identity information in the sample original face images, and the similarity between the sample virtual face images and the sample original face images is greater than or equal to the similarity threshold, thereby ultimately achieving model training for the face manipulation network.

[0064] Since the face anonymization model does not need to decouple the identity information and other attributes of the face image, the learning difficulty of the face anonymization model is reduced and the training speed is improved; and the face manipulation network ManipulationCNN in the face anonymization model can be a fully convolutional network, which directly edits the real identity information to the virtual identity information in the dimension with higher spatial resolution, and can better retain the details and background in the input original face image, and the generated target face image has higher image quality.

[0065] Figure 4 The following schematically illustrates the principle of a face anonymization model in an exemplary embodiment of the present disclosure.

[0066] refer to Figure 4 As shown, the face anonymization model 401 may include at least a face manipulation network 402 (ManipulationCNN), a face recognition network 403, and an adversarial discrimination network 404. It should be noted that, Figure 4 Although only one face manipulation network 402 is shown in the embodiment, of course, K face manipulation networks can also be used (where K is a positive integer greater than 1). Each face manipulation network can adopt a different network structure or different random initialization parameters to be trained together, thereby constructing a face dataset of K groups of virtual identity information, and keeping each virtual identity information different.

[0067] The original face image 405 can be obtained first, and the original face image 405 is input into the face anonymization model 401. Through the image conversion processing of the face manipulation network 402, a virtual face image 406 with virtual identity information is obtained; then the original face image 405 and the virtual face image 406 are input into the face recognition network 403, and the feature matrices corresponding to the original face image 405 and the virtual face image 406 are extracted. A distance loss function (such as relational formula (1)) is constructed based on the feature matrix to measure the sample distance within the same virtual identity information, the sample distance between different virtual identity information, and the real identity information. The distance between the original face image 405 and the virtual identity information is obtained, and the network weight parameters of the face manipulation network are trained through the distance loss obtained by the distance loss function; the original face image 405 and the virtual face image 406 are continued to be input into the adversarial discriminant network 404 respectively, and the network weights of the adversarial discriminant network 404 are trained through the classification loss function (such as dividing the original face image 405 into category 0 and the virtual face image 406 into category 1), and then the virtual face image 406 is input alone to construct the adversarial loss function (that is, the virtual face image 406 is divided into category 0), and the network weight parameters of the face manipulation network are trained through the adversarial loss supervision obtained by the adversarial loss function.

[0068] The distance loss function and the adversarial loss function are used to assist in the training of the face manipulation network, so that the virtual face image 406 generated by the face manipulation network has virtual identity information (i.e., does not contain real identity information), and the facial similarity between the virtual face image 406 and the original face image 405 is greater than or equal to the similarity threshold. If the newly generated virtual face image 406 meets the conditions, the training of the face anonymization model 401 can be considered complete, and the virtual face image 406 that meets the conditions is output as the final output result target face image 407.

[0069] In an exemplary embodiment, the distance loss function can be determined by the face recognition network through the steps in 5, referring to Figure 5As shown, specifically can include:

[0070] Step S510, input the first original face image, the second original face image, the first virtual face image and the second virtual face image into the face recognition network respectively, to obtain a first feature matrix, a second feature matrix, a third feature matrix and a fourth feature matrix;

[0071] Step S520, determine the distance loss function by the first feature matrix, the second feature matrix, the third feature matrix and the fourth feature matrix.

[0072] Wherein, the sample original face image can include a first original face image corresponding to a first real identity information and a second original face image corresponding to a second real identity information, and the sample virtual face image converted by the face manipulation network can include a first virtual face image corresponding to a first virtual identity information and a second virtual face image corresponding to a second virtual identity information.

[0073] The first feature matrix, the second feature matrix, the third feature matrix and the fourth feature matrix can be respectively a feature matrix extracted by the face recognition network from the first original face image corresponding to the first real identity information, a feature matrix extracted by the face recognition network from the second original face image corresponding to the second real identity information, a feature matrix extracted by the face recognition network from the first virtual face image corresponding to the first virtual identity information, and a feature matrix extracted by the face recognition network from the second virtual face image corresponding to the second virtual identity information.

[0074] By constructing a distance loss function that makes the distance between different samples within the same virtual identity information small and the distance between samples of different virtual identity information categories large, it is ensured that the edited virtual identity information also has separability and will not collapse into the same virtual identity information; The distance loss function also contains the distance measurement between virtual identity information and other real identity information, which can also avoid editing one real identity information into another real identity information, and ensure the accuracy and robustness of the output results of the face anonymization model.

[0075] For example, assuming that the first original face image corresponding to the first real identity information can be represented as I o 1 , then the first feature matrix obtained by feature extraction of the face image I o 1 through the face recognition network can be represented as M o 1 ; the second original face image corresponding to the second real identity information can be represented as I o 2Then the first feature matrix obtained by performing feature extraction on the face image I o 1 The first feature matrix obtained by performing feature extraction on the face image I o 2 The first virtual face image corresponding to the first virtual identity information can be represented as I m 1 Then the first feature matrix obtained by performing feature extraction on the face image I m 1 The first feature matrix obtained by performing feature extraction on the face image I m 1 The second virtual face image corresponding to the second virtual identity information can be represented as I m 2 Then the first feature matrix obtained by performing feature extraction on the face image I m 2 The first feature matrix obtained by performing feature extraction on the face image I m 2 ;

[0076] Taking the first real identity information as an example, the distance loss function can be represented as relationship (1):

[0077] L1=Minimize(M o 1 ×M m 1T )+Minimize(M m 2 ×M m 1T )+Minimize(M o 2 ×M m 1T )-Maximize(M m 1 ×M m 1T ) (1)

[0078] Wherein, L1 can represent the distance loss function corresponding to the first real identity information, M m 1T can represent the transpose matrix of the first feature matrix M m 1 Similarly, the distance loss function corresponding to the second real identity information can be obtained, which is not described here.

[0079] It should be noted that the "first", "second", "third" and "fourth" in the "first feature matrix", "second feature matrix", "third feature matrix" and "fourth feature matrix" in the example embodiment are only used to distinguish the feature matrices corresponding to different face images, and do not have any special meaning, and should not cause any special limitation to the example embodiment.

[0080] In an example embodiment, after converting the original face image into the target face image, the target face image can also be input into the face image reconstruction model to recover the original face image containing the real identity information.

[0081] The face image reconstruction model can include a reconstruction loss function, which is a mean square error loss function constructed by the face feature maps of the original face image and the target face image.

[0082] For example, the original face image and the target face image can be input into the face detection network to obtain the bottom-level face feature maps F o and F m Then, the reconstruction loss function can be represented as relationship (2):

[0083] L r =||F o -F m ||2 (2)

[0084] Wherein, L r represents the reconstruction loss function, F o represents the face feature map corresponding to the original face image, and F m represents the face feature map corresponding to the target face image.

[0085] Figure 6 An example flowchart of training a face anonymization model in an example embodiment of the present disclosure is schematically shown.

[0086] Referring to Figure 6 S610, a batch of training samples is obtained, which can include at least two sample original face images corresponding to each real identity information, for example, the batch of training samples can include a first original face image 601 corresponding to a first real identity information and a second original face image 602 corresponding to a second real identity information; then the first original face image 601 and the second original face image 602 are input into the face manipulation network 603 to obtain a first virtual face image 604 corresponding to a first virtual identity information and a second virtual face image 605 corresponding to a second virtual identity information, respectively.

[0087] Determine whether the first virtual identity information or the second virtual identity information belongs to the first real identity information or the second real identity information, and determine whether the similarity between the first original facial image 601 or the second original facial image 602 and the first virtual facial image 604 or the second virtual facial image 605 is greater than or equal to a similarity threshold. If the conditions are met, use the first virtual facial image 604 or the second virtual facial image 605 as the output result of the first original facial image 601 or the second original facial image 602, determine that the face anonymization model training is completed, and terminate the current process; if not, continue to step S620;

[0088] Step S620: Input the first original face image 601, the second original face image 602, the first virtual face image 604, and the second virtual face image 605 into the face recognition network 606 to obtain feature matrices 607, 608, 609, and 610, respectively. Based on the obtained feature matrices, a distance loss function 611 is constructed to supervise the face manipulation network 603. The network weight parameters of the face manipulation network 603 are adjusted through the distance loss feedback of the distance loss function 611.

[0089] In step S630, the first original face image 601, the second original face image 602, the first virtual face image 604, and the second virtual face image 605 are input into the adversarial discriminant network 612. The network weight parameters of the adversarial discriminant network 612 are trained by using the classification loss function with the goal of classifying the first original face image 601 and the second original face image 602 into category 0 (or category True, this example embodiment is not limited to this), and classifying the first virtual face image 604 and the second virtual face image 605 into category 1 (or category False, this example embodiment is not limited to this). Then, the first virtual face image 604 and the second virtual face image 605 are input separately with the goal of classifying the first virtual face image 604 and the second virtual face image 605 into category 0, and an adversarial loss function 613 is constructed to supervise the face manipulation network 603. The network weight parameters of the face manipulation network 603 are adjusted through the adversarial loss feedback obtained by the adversarial loss function 613.

[0090] In another exemplary embodiment, the Figure 7 The steps in the implementation of the initial face anonymization model training. Figure 7 Specifically, it may include:

[0091] Step S710, determining the true identity feature vector corresponding to the original face image of the sample in the training sample set;

[0092] Step S720, statistics is performed on the real identity feature vector to obtain a real identity feature distribution;

[0093] Step S730, a virtual identity feature vector is randomly sampled from the real identity feature distribution;

[0094] Step S740, the virtual identity feature vector is taken as label data of the sample original face image, and regression training is performed on the initial face anonymization model to obtain a trained face anonymization model.

[0095] The real identity feature vector refers to a feature vector corresponding to real identity information of a sample original face image. The embodiment provides a new generation manner of virtual identity information, that is, first, statistics is performed on real identity feature vectors in a training sample set to obtain a general distribution, then a virtual identity feature vector is obtained in a manner of random sampling from the distribution, and during training, the virtual virtual identity information is directly regressed to obtain a trained face anonymization model.

[0096] The embodiments in the disclosure do not need to decouple the identity information and other attributes of the face image, reduce the learning difficulty of the face anonymization model, and improve the training speed; and the face manipulation network ManipulationCNN in the face anonymization model can be a full convolutional network, therefore, details and backgrounds in the input original face image can be better preserved, and the image quality of the generated target face image is high; meanwhile, in the embodiments in the disclosure, the same real identity information is edited into the same virtual identity information, and therefore the generated data set can still be used to train the face recognition network.

[0097] The embodiments in the disclosure directly edit the real identity information into the virtual identity information in a dimension with a higher spatial resolution, the image quality of the generated target face image is high, and details are rich; by constructing a distance loss function that makes different samples in the same virtual identity information close and makes samples of different virtual identity information categories far, it is ensured that the edited virtual identity information also has separability and will not collapse into the same virtual identity information; the distance loss function also contains distance measurement between the virtual identity information and other real identity information, and can also avoid editing one real identity information into another real identity information.

[0098] In summary, in this exemplary embodiment, an original facial image containing real identity information can be input into a pre-trained face anonymization model to directly generate a target facial image that does not contain real identity information and whose similarity to the original facial image is greater than a similarity threshold. On the one hand, directly generating the target facial image through the face anonymization model avoids the problem of replacing facial regions, which results in an unrealistic facial image and poor image quality, and effectively improves the image quality of the target facial image. On the other hand, while hiding the real identity information contained in the original facial image, the generated target facial image is ensured to be as similar as possible to the original facial image, without losing the facial display features in the original facial image, thereby improving the user experience.

[0099] It should be noted that the above figures are merely illustrative of the processes included in the methods according to exemplary embodiments of the present disclosure and are not intended to be limiting. It is readily understood that the processes illustrated in the above figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0100] For further reference, Figure 8 As shown, in the embodiment of this example, a facial image generation device 800 is further provided, comprising an original facial image acquisition module 810 and a target facial image generation module 810. In which:

[0101] The original face image acquisition module 810 is used to obtain the original face image containing the real identity information;

[0102] The target face image generation module 820 is used to input the original face image into the pre-trained face anonymization model to generate a target face image;

[0103] The target facial image does not contain the real identity information, and the similarity between the target facial image and the original facial image is greater than a similarity threshold.

[0104] In an exemplary embodiment, the facial image generation apparatus 800 may include a facial anonymization model training unit, which may be configured to:

[0105] Obtaining a training sample batch, wherein the training sample batch includes at least one sample original face image corresponding to each real identity information;

[0106] The initial face anonymization model is trained using the sample original face image until the output sample virtual face image does not contain the real identity information and the similarity between the sample virtual face image and the sample original face image is greater than or equal to a similarity threshold, thereby obtaining a trained face anonymization model.

[0107] In an example embodiment, the face anonymization model can include a face manipulation network, a face recognition network, and an adversarial discriminant network; the face anonymization model training unit can be configured to:

[0108] input the sample original face image into the face manipulation network to obtain a sample virtual face image with virtual identity information;

[0109] determine a distance loss function through the sample original face image, the sample virtual face image, and the face recognition network;

[0110] determine an adversarial loss function through the sample original face image, the sample virtual face image, and the adversarial discriminant network;

[0111] adjust network weights of the face manipulation network based on the distance loss function and the adversarial loss function to realize model training of the face manipulation network.

[0112] In an example embodiment, the sample original face image can include a first original face image corresponding to first real identity information and a second original face image corresponding to second real identity information, and the sample virtual face image can include a first virtual face image corresponding to first virtual identity information and a second virtual face image corresponding to second virtual identity information; the face anonymization model training unit can be further configured to:

[0113] input the first original face image, the second original face image, the first virtual face image, and the second virtual face image into the face recognition network respectively to obtain a first feature matrix, a second feature matrix, a third feature matrix, and a fourth feature matrix;

[0114] determine the distance loss function through the first feature matrix, the second feature matrix, the third feature matrix, and the fourth feature matrix.

[0115] In an example embodiment, the face image generation apparatus 800 can further include a target face image recovery unit, which can be configured to:

[0116] input the target face image into a face image reconstruction model to obtain an original face image containing the real identity information;

[0117] wherein the face image reconstruction model includes a reconstruction loss function, and the reconstruction loss function is a mean square error loss function constructed by face feature maps of the original face image and the target face image.

[0118] In an example embodiment, the face anonymization model training unit can be further configured to:

[0119] determining a real identity feature vector corresponding to a sample original face image in the training sample set;

[0120] performing statistics on the real identity feature vector to obtain a real identity feature distribution;

[0121] randomly sampling a virtual identity feature vector from the real identity feature distribution;

[0122] regression training of the initial face anonymization model by taking the virtual identity feature vector as label data of the sample original face image, to obtain a trained face anonymization model.

[0123] The specific details of each module in the above apparatus have been described in detail in the method part embodiments, and the undisclosed details can be referred to the method part embodiments, and thus will not be described again.

[0124] Those skilled in the art can understand that each aspect of the present disclosure can be implemented as a system, a method or a program product. Therefore, each aspect of the present disclosure can be embodied as a whole hardware embodiment, a whole software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects, which can be collectively referred to as "circuitry", "module" or "system".

[0125] The exemplary embodiments of the present disclosure provide an electronic device for implementing the face image generation method, which can be a terminal device 101, 102, 103, a server 105 or a server cluster 106 in Figure 1 . The electronic device at least includes a processor and a memory, the memory is used to store executable instructions of the processor, and the processor is configured to execute the face image generation method by executing the executable instructions.

[0126] The following will take the electronic device 900 in Figure 9 as an example to exemplarily illustrate the structure of the electronic device in the present disclosure. Figure 9 The electronic device 900 shown is only an example, and should not bring any limitation to the function and use range of the embodiments of the present disclosure.

[0127] As shown in Figure 9 , the electronic device 900 is in the form of a general computing device. The components of the electronic device 900 can include but are not limited to: at least one processing unit 910, at least one storage unit 920, a bus 930 connecting different system components (including the storage unit 920 and the processing unit 910), a display unit 940.

[0128] The storage unit 920 stores program codes which can be executed by the processing unit 910, so that the processing unit 910 performs the face image generation method in the present specification.

[0129] The storage unit 920 can include a readable medium in the form of a volatile storage unit, such as a random access memory (RAM) 921 and / or a cache memory 922, and can further include a read-only memory (ROM) 923.

[0130] The storage unit 920 can further include program / utility 924 having a set of program modules 925 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which can give the electronic device 900 its functionality, as well as implementing the network environment in some combination.

[0131] The bus 930 can represent one or more of several types of bus structures, including a storage unit bus or bus controller, a peripheral bus, a graphics acceleration port, a processing unit bus, or a local bus using any of a variety of bus architectures.

[0132] The electronic device 900 can also communicate with one or more external devices 970 such as a sensor device, a Bluetooth device, or the like, and / or communicate with one or more devices that enable input to the electronic device 900, and / or enable output from the electronic device 900, including to one or more devices that enable shared communication between the electronic device 900 and other computing devices. Such communication can occur via an input / output (I / O) interface 950. Still yet, the electronic device 900 can communicate with one or more networks, such as one or more local area networks (LANs), wide area networks (WANs), and / or the Internet, through a network adapter 960. As depicted, the network adapter 960 can communicate with the other components of the electronic device 900 via the bus 930. It should be appreciated that although the network adapter 960 is depicted as a single component in FIG. 9, the network adapter 960 can comprise two or more components that work together to facilitate communications with one or more networks. Moreover, it is noted that the electronic device 900 can be implemented using a plurality of devices, which can each comprise one or more components as described herein. For example, the electronic device 900 can comprise a plurality of devices that communicate with one another over a network, such as a local area network (LAN) or a wide area network (WAN), such as the Internet.

[0133] Those skilled in the art can easily understand, through the above description of the embodiments, that the example embodiments described herein can be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product. The software product can be stored in a non-volatile storage medium (which can be a CD-ROM, a U disk, a mobile hard disk, etc.) or on a network, and includes a number of instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0134] The example embodiments of the present disclosure also provide a computer-readable storage medium having stored thereon a program product capable of implementing the method described above. In some possible embodiments, various aspects of the present disclosure can also be implemented in the form of a program product, which includes program codes for causing an end device to execute the steps according to various example embodiments of the present disclosure described in the "example method" section of the present specification when the program product is run on the end device.

[0135] It should be noted that the computer-readable medium shown in the present disclosure can be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0136] In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination of the foregoing.

[0137] In addition, the program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, and the like, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a stand-alone software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0138] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow from the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the claims.

[0139] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for generating a face image, characterized in that: include: Obtain the original face image containing real identity information; Inputting the original face image into a pre-trained face anonymization model to generate a target face image; the face anonymization model includes a face manipulation network, a face recognition network, and an adversarial discriminant network; The target facial image does not contain the real identity information, and the similarity between the target facial image and the original facial image is greater than or equal to a similarity threshold; The model training of the face manipulation network includes: Inputting the original face image into the face manipulation network to obtain a sample virtual face image with virtual identity information; Determining a distance loss function using the original face image, the sample virtual face image, and the face recognition network; Determining an adversarial loss function using the original face image, the sample virtual face image, and the adversarial discriminant network; Adjusting the network weights of the face manipulation network based on the distance loss function and the adversarial loss function to implement model training of the face manipulation network; The model training of the face anonymization model includes: Determine the true identity feature vector corresponding to the original face image in the training sample set; Performing statistics on the real identity feature vector to obtain a real identity feature distribution; Randomly sampling from the real identity feature distribution to obtain a virtual identity feature vector; The virtual identity feature vector is used as label data of the original face image, and regression training is performed on the initial face anonymization model to obtain a trained face anonymization model.

2. The method according to claim 1, characterized in that The method further comprises: Obtaining a training sample batch, wherein the training sample batch includes at least one original face image corresponding to each real identity information; The initial face anonymization model is trained using the original face image until the output sample virtual face image does not contain the real identity information and the similarity between the sample virtual face image and the original face image is greater than or equal to a similarity threshold, thereby obtaining a trained face anonymization model.

3. The method according to claim 1, characterized in that The original facial image includes a first original facial image corresponding to the first real identity information and a second original facial image corresponding to the second real identity information, and the sample virtual facial image includes a first virtual facial image corresponding to the first virtual identity information and a second virtual facial image corresponding to the second virtual identity information; The determining of a distance loss function using the original face image, the sample virtual face image, and the face recognition network includes: Inputting the first original face image, the second original face image, the first virtual face image, and the second virtual face image into the face recognition network respectively to obtain a first feature matrix, a second feature matrix, a third feature matrix, and a fourth feature matrix; The distance loss function is determined by using the first feature matrix, the second feature matrix, the third feature matrix, and the fourth feature matrix.

4. The method according to claim 1, wherein The method further comprises: Inputting the target face image into a face image reconstruction model to obtain an original face image containing the real identity information; The facial image reconstruction model includes a reconstruction loss function, which is a mean square error loss function constructed by the facial feature map of the original facial image and the target facial image.

5. The method according to claim 1, wherein The face manipulation network is a fully convolutional network.

6. A facial image generation device, characterized in that: include: The original face image acquisition module is used to obtain the original face image containing the real identity information; A target face image generation module, configured to input the original face image into a pre-trained face anonymization model to generate a target face image; the face anonymization model includes a face manipulation network, a face recognition network, and an adversarial discriminant network; The target facial image does not contain the real identity information, and the similarity between the target facial image and the original facial image is greater than a similarity threshold; The model training of the face manipulation network includes: Inputting the original face image into the face manipulation network to obtain a sample virtual face image with virtual identity information; determining a distance loss function using the original face image, the sample virtual face image, and the face recognition network; determining an adversarial loss function using the original face image, the sample virtual face image, and the adversarial discriminant network; adjusting the network weights of the face manipulation network based on the distance loss function and the adversarial loss function to implement model training of the face manipulation network; The model training of the face anonymization model includes: Determine the real identity feature vector corresponding to the original face image in the training sample set; perform statistics on the real identity feature vector to obtain a real identity feature distribution; randomly sample from the real identity feature distribution to obtain a virtual identity feature vector; use the virtual identity feature vector as label data of the original face image to perform regression training on the initial face anonymization model to obtain a trained face anonymization model.

7. A computer-readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

8. An electronic device, characterized in that: include: processor; as well as a memory for storing executable instructions of the processor; The processor is configured to perform the method according to any one of claims 1 to 5 by executing the executable instructions.

Images