A method for generating and recognizing face anonymized images based on maintaining identity relationships

Abstract

The present invention discloses a method for generating and recognizing face anonymized images based on maintaining identity relationships, including: obtaining the ciphertext of the identity relationship of the original face image; using a trained visual appearance editor to perform face appearance editing on the original face image to obtain the edited anonymized image; inputting the edited anonymized image into a trained visual enhancer to add image details and textures; using a trained ciphertext embedder to embed the ciphertext of the identity relationship into the visually enhanced anonymized image; inputting the anonymized face image maintaining the identity relationship into a trained anonymized recognizer to obtain the recognized ciphertext of the identity relationship; comparing the recognized ciphertext of the identity relationship with the saved ciphertext of the identity relationship, and selecting the closest one as the final ciphertext of the identity relationship and the face image. The present invention can improve the recognition rate of anonymized faces and complete the face anonymization task with relatively high qualitative and quantitative quality.

Description

Technical Field

[0001] The present invention belongs to the technical field of face anonymization, and in particular relates to a method for generating and recognizing anonymous face images based on identity relationship preservation. Background Art

[0002] With the widespread application of big data technology, we are exposed to more and more video surveillance. In order to prevent serious social problems caused by the leakage of face data, face anonymization has become an important method to protect face privacy. Face anonymization manipulates visual appearance and identity information simultaneously in the generation of anonymized face images, while keeping identity-independent visual information such as posture and facial expressions as much as possible. With the widespread application of deep learning, face recognition has become an indispensable technology for identity authentication, and is used in scenarios such as mobile payment and security monitoring. As we are exposed to more and more cameras, many companies have collected a large number of faces, which leads to many security risks in the privacy of these identities. In order to protect face privacy, face anonymization has recently become an important field.

[0003] Face anonymization technology can be applied to many scenarios to protect facial privacy. For example: (1) It can protect the privacy of interviewees in news interviews, such as people in news case reports, people interviewed on the street, anti-drug police, and others who need to protect their identity and privacy; (2) In medical consultation and case sharing, it can protect the face privacy of patients; (3) When sharing pictures on social networking sites, it can protect the privacy of uploaders; (4) In order to protect the privacy of faces in public datasets, it is sometimes necessary to perform privacy protection preprocessing on datasets containing faces, otherwise they may not be able to continue sharing them. Both the DUKE MTMC dataset and the MS-Celeb-1M dataset were terminated due to leaking the face identity privacy in the datasets. The famous ImageNet dataset also blurs the faces in the dataset to protect privacy.

[0004] Existing face image anonymization methods can be divided into two different types: traditional face anonymization methods and deep learning-based face anonymization methods. The former uses traditional techniques such as image blurring and mosaics to make facial images visually unrecognizable. The latter uses deep network models to modify facial and identity information in images while keeping visual features (e.g., pose, facial expression) unchanged to reconstruct anonymous faces.

[0005] Traditional face anonymization methods: In the early stage of dealing with face anonymization problems, a large number of operations were performed on face images, such as image blurring, mosaics, and downsampling, to hide identity information. For example, Newton et al. proposed a K-Same anonymization method, the core idea of which is to fuse K face images that are most similar to the original face to obtain an anonymized face. However, the weakness of this method is that the visual quality of the anonymized face is not good enough, and the anonymized face images are often unclear and accompanied by artificial forgery traces.

[0006] Deep learning-based anonymization methods: Thanks to the remarkable progress of deep learning technology, the images generated by the GAN (Generative Adversarial Network) model have reached a level where it is difficult for the human eye to distinguish between genuine and fake. Deep learning-based face anonymization methods generally perform better than traditional face anonymization methods in terms of image resolution and realism, and thus have become a research hotspot. Face anonymization methods using deep learning technology can be divided into face restoration, face replacement, and feature disentanglement-based methods. For example, Li et al. proposed a simple and effective image restoration method DeepBlur, which first uses a pre-trained model to blur the input image, and then generates a realistic face based on the blurred face. Compared with existing image blurring techniques, the visual quality of the images generated by this method has reached a higher level.

[0007] Reversible face anonymization methods: In some cases, it is desirable to anonymize face images to protect face privacy and restore the anonymized face images when needed. For example, when sharing pictures on social networks, the image owner hopes to use anonymization tools to protect their facial images from being accessed by unknown people. On the other hand, they hope that their relatives can restore the anonymized images. Gu et al. proposed a password-based anonymization method. Given a password and a facial image to obtain an anonymized image, the original facial image can be restored using the previously used password and the anonymized face. Cao et al. proposed a method that uses the decoupling operation of identity and attribute features, retaining the attribute details such as facial expressions, poses, and lighting of the anonymized face image, and achieving the identity consistency between the de-identified image and the original image.

[0008] However, existing face anonymization methods all focus on how to modify the visual identity information of the original image, and have not considered whether the anonymized face can still be applied to related scenarios such as face recognition. Recently, several reversible anonymization methods have emerged, which can reconstruct the original face from anonymized images. Although the reconstructed face can be used for face recognition tasks, the meaning of face anonymization is lost after reconstruction, and privacy issues are brought back to the reconstructed face. Summary of the Invention

[0009] To solve the above problems existing in the prior art, the present invention provides a method for generating and recognizing face anonymized images based on maintaining identity relationships. The technical problems to be solved by the present invention are realized through the following technical solutions:

[0010] The present invention provides a method for generating and recognizing face anonymized images based on maintaining identity relationships, including:

[0011] S1: Obtain the identity relationship ciphertext of the original face image, where the identity relationship ciphertext is a binary encrypted text corresponding to the identity information of the face;

[0012] S2: Use a trained visual appearance editor to perform face appearance editing on the original face image to obtain an edited anonymized image;

[0013] S3: Input the edited anonymized image into a trained visual enhancer to add image details and textures to obtain a visually enhanced anonymized image;

[0014] S4: Use a trained ciphertext embedder to embed the identity relationship ciphertext into the visually enhanced anonymized image to obtain an anonymized face image maintaining the identity relationship;

[0015] S5: Input the anonymized face image maintaining the identity relationship into a trained anonymized recognizer to obtain the recognized identity relationship ciphertext;

[0016] S6: Compare the recognized identity relationship ciphertext with the saved identity relationship ciphertext, and select the closest one as the final identity relationship ciphertext and face image.

[0017] In an embodiment of the present invention, the binary encrypted text is a random binary code or generated using the MD5 or SHA-256 encryption algorithm.

[0018] In an embodiment of the present invention, S2 further includes:

[0019] Train the visual appearance editor, forgery discriminator, and identity extractor together to obtain a trained visual appearance editor. Among them, the visual appearance editor inputs the face image and the corresponding identity relationship ciphertext to obtain a manipulated visual appearance image. Subsequently, input the manipulated visual appearance image and the original face image into the forgery discriminator to judge its authenticity. The training loss function is:

[0020]

[0021]

[0022] Among them, D represents the forgery discriminator, V represents the visual appearance editor, x represents the input original face image, D(x) represents the output of the forgery discriminator, and V(x) represents the output of the visual appearance editor. represents the loss function of the forgery discriminator. represents the loss function of the visual appearance editor. represents the cross-entropy loss.

[0023] In one embodiment of the present invention, the S2 further includes:

[0024] During the training process, the face image output by the visual appearance editor is input into the identity extractor to compare the face image after appearance editing with the original face image, so as to improve the appearance editing performance of the image generated by the visual appearance editor. During the training process, and the loss function guides the visual appearance editor:

[0025]

[0026]

[0027] Among them, V represents the visual appearance editor, F represents the identity extractor, c1 and c1 are two different identity relationship ciphertexts, and y represents the identity relationship ciphertext of the original face image. represents the identity relationship ciphertext extracted from the image x' after appearance editing i in, F(x) represents the output of the identity extractor, and F emb represents the function for extracting face embedding features.

[0028] In one embodiment of the present invention, the S2 further includes:

[0029] After the training of the visual appearance editor model is completed, the ciphertext embedder and the anonymous recognizer are trained together so that the anonymous recognizer can accurately extract the identity relationship ciphertext embedded in the anonymous face image. The training loss function is:

[0030]

[0031]

[0032] x' = V(x)

[0033] Among them, R is the anonymous recognizer, E is the ciphertext embedder, I is the visual enhancer, V is the visual appearance editor, and c k represents the k-th bit of the identity relationship ciphertext, and n is the length of the identity relationship ciphertext. It represents the encrypted text of the recognized identity relationship, and E(I(x′)) represents the encrypted text of the identity relationship embedded in the image after visual appearance editing.

[0034] In one embodiment of the present invention, the S2 further includes:

[0035] Training the ciphertext editor and the visual enhancer together to reduce the impact of the embedded encrypted text of the identity relationship on the visual quality of the anonymized image, and the training loss function is:

[0036]

[0037] where E is the ciphertext embedder and I is the visual enhancer, represents calculating the L2 norm of two vectors.

[0038] Another aspect of the present invention provides a storage medium, in which a computer program is stored, and the computer program is used to execute the steps of the method for generating and recognizing face anonymized images based on identity relationship preservation described in any one of the above embodiments.

[0039] Another aspect of the present invention provides an electronic device, including a memory and a processor. A computer program is stored in the memory, and when the processor calls the computer program in the memory, it implements the steps of the method for generating and recognizing face anonymized images based on identity relationship preservation described in any one of the above embodiments.

[0040] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0041] The method for generating and recognizing face anonymized images based on identity relationship preservation in the present invention encodes the identity relationship of face images into binary ciphertext, and designs a generative adversarial network to embed relationship clues in anonymized face images to perform face anonymization. A face forgery discriminator is designed to enhance the realism of anonymized face images, and anonymized face recognition is performed by a well-designed identity relationship recognition network. The proposed anonymization recognizer provides a new perspective for the recognition and application of anonymized face images. Experiments on the Megaface dataset show that the method of the present invention can improve the recognition rate of anonymized faces by 100%, and complete the face anonymization task with high qualitative and quantitative quality, while being robust to various real-world image perturbations.

[0042] The following will further describe the present invention in detail with reference to the accompanying drawings and embodiments. Description of the Drawings

[0043] Figure 1 It is a flowchart of a method for generating and recognizing face anonymized images based on identity relationship preservation provided by an embodiment of the present invention;

[0044] Figure 2 It is a schematic diagram of the processing process of a method for generating and recognizing face anonymous images based on identity relationship preservation provided by an embodiment of the present invention;

[0045] Figure 3 It is a schematic diagram of the process of generating a face anonymous image based on identity relationship preservation provided by an embodiment of the present invention;

[0046] Figure 4 It is a schematic diagram of the processing process of a ciphertext embedder provided by an embodiment of the present invention;

[0047] Figure 5 It is a schematic diagram of the processing process of an anonymous recognizer provided by an embodiment of the present invention;

[0048] Figure 6 They are anonymous images generated by using existing methods and the method of the embodiment of the present invention respectively. Detailed implementation manners

[0049] In order to further elaborate on the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following, in combination with the accompanying drawings and specific implementation manners, details a method for generating and recognizing face anonymous images based on identity relationship preservation proposed according to the present invention.

[0050] Regarding the foregoing and other technical contents, features and effects of the present invention, they can be clearly presented in the following detailed description in conjunction with the accompanying drawings. Through the description of the specific implementation manners, a more in-depth and specific understanding of the technical means and effects adopted by the present invention to achieve the predetermined purpose can be obtained. However, the accompanying drawings are only for reference and illustration, and are not used to limit the technical solution of the present invention.

[0051] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant is intended to cover non-exclusive inclusion, so that an article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed. Without more limitations, an element defined by the statement "including one..." does not exclude the existence of another identical element in the article or device including the said element.

[0052] Embodiment 1

[0053] Please refer to Figure 1 and Figure 2 , the method for generating and recognizing face anonymous images based on identity relationship preservation in this embodiment includes the following steps:

[0054] S1: Obtain the ciphertext of the identity relationship of the original face image, where the ciphertext of the identity relationship is a binary encrypted text corresponding to the identity information of the face.

[0055] Introducing the ciphertext of the identity relationship is one of the cores in the face anonymization and recognition framework of this embodiment. Specifically, the identity relationship between face images is encoded as a binary encrypted text, and a generative adversarial network is designed to embed the ciphertext of the identity relationship into the anonymized face image during the process of generating the anonymized face image. This embodiment uses 100-bit binary encoding to represent the identity relationship between face images. There are two requirements for the ciphertext of the identity relationship: (1) The face images of different people should have different relationship ciphertexts; (2) The face images from the same person should keep their relationship ciphertexts the same. Following the above two principles, the ciphertext of the identity relationship can be generated by applying random binary encoding or using MD5 or SHA-256 encryption algorithms, where the privacy information of the face (such as its identity, name, gender, etc.) is encrypted.

[0056] S2: Use the trained visual appearance editor to perform face appearance editing on the original face image to obtain the edited anonymized image.

[0057] Specifically, as Figure 3 shown, input the original face image to be anonymized into the visual appearance editor, and the visual appearance editor can complete the face appearance editing to hide the identity information of the original face image and obtain the edited anonymized image. The visual appearance editor of this embodiment is modified based on CycleGAN (Cycle Generative Adversarial Network), replacing the small-stride convolutional layer in it with a resized convolutional layer to reduce artifacts in the image. In other embodiments, this visual appearance editor can also use other suitable networks capable of performing appearance editing.

[0058] S3: Input the edited anonymized image into the trained visual enhancer to add image details and textures to obtain the visually enhanced anonymized image.

[0059] Specifically, a visual enhancer can be used to improve the resolution of an image, add image details and textures. The visual enhancer in this embodiment uses the GPEN (GAN Prior Embedded Network) model, which generally follows the design idea of the U-shaped model. A mapping network is used to map the latent code z to a deconstructed space w∈W, which will be input into each module of each GAN network. Since the pre-trained GAN network needs to be fine-tuned after being embedded in the U-shaped deep neural network, space needs to be reserved for the feature maps in the generation part. For this purpose, an additional noise input is added. The GPEN model can input a low-resolution picture, map the picture to the latent code z through multiple convolutional layers and a fully connected layer, then map it to the feature vector w of the structure, and finally send it into the fine-tuned pre-trained GAN model to generate the restored high-resolution picture, thereby enhancing the details and textures of the picture.

[0060] S4: Use the trained ciphertext embedder to embed the identity relationship ciphertext into the visually enhanced anonymous image to obtain an anonymous face image that preserves the identity relationship.

[0061] In order to embed the identity relationship ciphertext into the anonymized face image, the ciphertext embedder uses the high-quality face editing image processed by the visual enhancer and the corresponding identity relationship ciphertext as inputs to obtain an anonymous face image that preserves the identity relationship.

[0062] Specifically, please refer to Figure 4 , Figure 4 FIG. is a schematic diagram of the processing process of a ciphertext embedder provided by an embodiment of the present invention. The ciphertext embedder in this embodiment is based on the StegaStamp neural network and is generally a U-shaped network structure. There will be splicing from the downsampling stage in the upsampling stage of the U-shaped network to retain the detail information of the image. In the input stage, the identity relationship ciphertext passes through a fully connected layer and multiple upsampling layers, and is adjusted to a tensor with the same shape as the input image, and then spliced with the image to form a six-channel tensor and input into the U-shaped network. The identity relationship ciphertext is embedded into the anonymous image while minimizing the pixel difference between the input image and the anonymous image. The binary identity relationship ciphertext vector first passes through a fully connected layer, and then is reshaped into a tensor with the same spatial dimension as the covering image in one channel dimension. Subsequently, the identity relationship ciphertext tensor and the image along the channel dimension are connected as the input of the U-shaped architecture. The output of the ciphertext embedder, that is, the anonymous image with the identity relationship ciphertext added, has the same size as the input image.

[0063] As Figure 3As shown in the figure, the process of generating an anonymous face image in this embodiment includes an image data preprocessing stage and a face anonymization stage. In the image data preprocessing stage, anonymization preparation is performed on the face image to be processed, and the ciphertext of the identity relationship corresponding to the face image is obtained. In the face anonymization stage, the original face image x i and the corresponding ciphertext of the identity relationship c i are input into a module composed of a visual appearance editor, a visual enhancer, and a ciphertext embedder, thereby generating an anonymized face image.

[0064] S5: Input the anonymized face image maintaining the identity relationship into the trained anonymous recognizer to obtain the recognized ciphertext of the identity relationship.

[0065] The method for generating and recognizing an anonymous face image in this embodiment can also extract the ciphertext of the identity relationship from the anonymous image containing the ciphertext of the identity relationship to meet the face recognition requirements in the anonymous environment. In this embodiment, the ciphertext of the identity relationship is extracted through the anonymous recognizer, and face recognition is completed by comparing different ciphertexts of the identity relationship. The anonymous recognizer in this embodiment consists of a series of convolutional layers with a kernel size of 3x3 and a stride ≥ 1, a dense layer, and a sigmoid output activation to produce an output with the same length as the binary ciphertext of the identity relationship. Please refer to Figure 5 , Figure 5 which is a schematic diagram of the processing process of an anonymous recognizer provided by an embodiment of the present invention. Input the anonymous image embedded with the ciphertext of the identity relationship into this anonymous recognizer. First, downsample it by a factor of two, and then pass through the convolutional layer. After repeating it 2 times, the number of channels is adjusted to 64. Then, through two consecutive downsamplings, and finally through two fully connected layers and an activation layer, the recognized ciphertext of the identity relationship is obtained.

[0066] S6: Compare the recognized ciphertext of the identity relationship with the pre-saved ciphertext of the identity relationship, and select the closest one as the final ciphertext of the identity relationship and the face image.

[0067] Specifically, in the process of obtaining the anonymous image embedded with the ciphertext of the identity relationship using the above anonymous process, the original image and its corresponding ciphertext of the identity relationship can be saved in text form to form a relationship ciphertext library, which can be used in the following recognition stage.

[0068] Subsequently, in the recognition stage, the face image is already anonymous. We can use the anonymous recognizer to extract the ciphertext of the identity relationship from the anonymous image. Then, compare the extracted ciphertext of the identity relationship with the ciphertext in the relationship ciphertext library, and sort the ciphertext with the closest L1 distance as the recognition result, and the corresponding picture is the image before anonymization.

[0069] In summary, the processing procedure of the face anonymization image generation method based on identity relationship maintenance in this embodiment is as follows: obtaining the ciphertext of the identity relationship of the original face picture, using a visual appearance editor to perform preliminary face attribute editing on the original face image to obtain a high-quality visually edited image, then inputting it into a visual enhancer to enhance the visual effect, and then inputting it into a ciphertext embedder to embed the ciphertext of the identity relationship in the high-quality visually edited image to obtain an anonymized image containing the ciphertext of the identity relationship.

[0070] It should be noted that before the actual image anonymization and recognition process, it is necessary to train each processing network module. To obtain the face anonymization model of this embodiment, first, it is necessary to train the visual appearance editor, the forgery discriminator, and the identity extractor together. The forgery discriminator and the identity extractor are designed to improve the authenticity and diversity of the face image after appearance editing. Specifically, the visual appearance editor inputs a training data set, and the training data set includes a large number of face images x i and the corresponding ciphertext of the identity relationship c i , and obtains the visually edited image x' of each face image after being edited i . Then, the forgery discriminator inputs x' i and the original face image x i and determines their authenticity. This embodiment uses the LSGAN loss function as the training loss function for this process:

[0071]

[0072]

[0073] where D represents the forgery discriminator, V represents the visual appearance editor, x represents the input original face image, D(x) represents the output of the forgery discriminator, V(x) represents the output of the visual appearance editor, represents the loss function of the forgery discriminator, represents the loss function of the visual appearance editor, represents the cross-entropy loss.

[0074] The identity extractor used in this embodiment is a pre-trained SphereFace network. This SphereFace network is a multi-output network. Inputting the image x, it outputs F(x) and Femb(x). Among them, F(x) extracts the ciphertext of the identity relationship (decimal number), and Femb(x) extracts the high-dimensional face feature vector. The forgery discriminator used in this embodiment is modified based on ProGAN. It uses two discriminators with the same structure but inputting images of different scales. They assist each other to determine whether the input image is an original image or an artificially generated image. In the adversarial training with the visual appearance editor, it improves the photo-realism of the output of the visual appearance editor.

[0075] Furthermore, in order to improve the face editing performance of the visual appearance editor for the generated image x′ i during the training process, the face image output by the visual appearance editor is input into the identity extractor to compare the face image after face editing with the original face image, so as to improve the face editing performance of the image generated by the visual appearance editor. During the training process, and the loss function is used to guide the visual appearance editor:

[0076]

[0077]

[0078] where V represents the visual appearance editor, F represents the identity extractor, c1 and c1 are two different identity relationship ciphertexts, y represents the identity relationship ciphertext of the original face image, represents the identity relationship ciphertext extracted from the image x′ i after face editing, F(x) represents the output of the identity extractor, and F emb represents the function for extracting face embedding features. and both aim to ensure that the identity recognized from the image after visual appearance editing is different from that of the original image. It is expected that the high-dimensional features of the recognized faces are different, and it is expected that the recognized identity relationship ciphertexts are different.

[0079] In addition, when the visual appearance editor manipulates the external information of the input face image, it is expected to retain as much background information in the face image as possible. Therefore, this embodiment also uses the loss function, as follows:

[0080]

[0081] where V is the visual appearance editor, and ‖*‖1 represents the L1 distance between two vectors.

[0082] After the training of the visual appearance editor model is completed, the ciphertext embedder and the anonymous recognizer are trained together so that the anonymous recognizer can accurately extract the identity relationship ciphertext embedded in the anonymous face image. The training loss function is:

[0083]

[0084]

[0085] x′ = V(x)

[0086] Among them, R is an anonymous identifier, E is a ciphertext embedder, I is a visual enhancer, V is a visual appearance editor, and c k represents the k-th bit of the ciphertext of the identity relationship, and n is the length of the ciphertext of the identity relationship. represents the recognized ciphertext of the identity relationship, and E(I(x′)) represents the ciphertext of the identity relationship embedded in the image after visual appearance editing.

[0087] Further, in order to reduce the impact of the embedded ciphertext of the identity relationship on the visual quality of the anonymous image, the ciphertext editor is trained together with the visual enhancer, and the loss function is applied here, and its definition is as follows:

[0088]

[0089] Among them, E is a ciphertext embedder, and I is a visual enhancer. represents calculating the L2 norm of two vectors.

[0090] In order to evaluate the performance of the face anonymous image generation and recognition method based on identity relationship preservation, this embodiment applies objective image quality evaluation indicators and subjective user studies to the anonymous faces.

[0091] The following describes the effects of the face anonymous image generation and recognition method of the embodiment of the present invention from two aspects of anonymous face image quality and face recognition respectively.

[0092] Please refer to Figure 6 , Figure 6 which are anonymous images generated by using existing deep privacy-based and password-based face anonymization methods and the method of this embodiment of the present invention respectively. Among them, the first row shows the original images from the CelebA dataset. From the visual comparison, it can be observed that all three methods produce face anonymization results, which look very different from the original images and all complete the protection of facial privacy. However, if Figure 6 is magnified, it can be found that the anonymous images generated by the existing deep privacy-based face anonymization method are blurred and of low quality. The results of the password-based face anonymization method are similar to those of the method of this embodiment of the present invention. However, the faces in the anonymous images obtained by the method of this embodiment are clearer, have better visual quality, and significantly improve the quality of the anonymous faces.

[0093] Furthermore, for anonymous face recognition evaluation, this embodiment uses rank-1 face recognition accuracy to evaluate the performance of anonymous face recognition. For the Megaface dataset, this embodiment follows the protocol of Challenge 2 for evaluation. For the CelebA dataset, this embodiment refers to the evaluation method of Megaface challenge 2, takes 50 identities in the CelebA dataset as the probe set, and takes the remaining 10127 identities as the gallery set to evaluate the rank-1 face recognition rate. The following gives the quantitative results of comparing the method of this embodiment of the invention with the existing methods. Table 1 shows the rank-1 accuracy of the anonymized images generated by various face anonymization methods on the CelebA dataset according to the above evaluation protocol. It can be seen that the anonymous recognizer of this embodiment of the invention achieves 100% accuracy, far superior to other methods. In addition, to illustrate the ability of the method of this embodiment in large-scale anonymous face recognition tasks, experiments were further carried out on the Megaface dataset and again achieved 100% rank-1 accuracy, which proves the effectiveness of the anonymous recognizer in large-scale face recognition tasks in an anonymous environment.

[0094] Table 1 Comparison of the face anonymous recognition accuracy between the proposed method and existing methods

[0095] Password-based Anonymous Method Deep Privacy-based Method This Method 0.2％ 1％ 100％

[0096] In summary, the method for generating and recognizing anonymous face images based on identity relationship preservation in this embodiment first introduces the topic of joint face anonymization and recognition, which can protect the identity information in face images, while the anonymized images can still be applied to face recognition tasks.

[0097] The method for generating and recognizing anonymous face images in this embodiment of the invention encodes the identity relationship of face images into binary ciphertext and designs a generative adversarial network to embed relationship clues in anonymous face images to perform face anonymization. A face forgery discriminator is designed to enhance the realism of anonymous face images, and anonymous face recognition is performed by a well-designed identity relationship recognition network. The proposed anonymous recognizer provides a new perspective for the recognition and application of anonymous face images. Experiments on the Megaface dataset show that the method of the invention can improve the recognition rate of anonymous faces by 100%, and complete the face anonymization task with high qualitative and quantitative quality, while being robust to various real-world image perturbations.

[0098] The above content is a further detailed description of the present invention in combination with specific preferred embodiments. It cannot be determined that the specific implementation of the present invention is only limited to these descriptions. For those of ordinary skill in the technical field to which the present invention pertains, without departing from the concept of the present invention, several simple deductions or substitutions can still be made, and all should be regarded as belonging to the protection scope of the present invention.

Claims

1. A method for generating and recognizing face anonymous images based on identity relationship maintenance, characterized in that Including: S1: Obtain the ciphertext of the identity relationship of the original face image, where the ciphertext of the identity relationship is a binary encrypted text corresponding to the identity information of the face; S2: Use the trained visual appearance editor to perform face appearance editing on the original face image to obtain an edited anonymous image; S3: Input the edited anonymous image into the trained visual enhancer to add image details and textures to obtain a visually enhanced anonymous image; S4: Use the trained ciphertext embedder to embed the ciphertext of the identity relationship into the visually enhanced anonymous image to obtain an anonymous face image that maintains the identity relationship; S5: Input the anonymous face image that maintains the identity relationship into the trained anonymous recognizer to obtain the recognized ciphertext of the identity relationship; S6: Compare the recognized ciphertext of the identity relationship with the saved ciphertext of the identity relationship, and select the closest one as the final ciphertext of the identity relationship and the face image; The S2 further includes: Train the visual appearance editor, the forgery discriminator, and the identity extractor together to obtain the trained visual appearance editor. The visual appearance editor inputs the face image and the corresponding ciphertext of the identity relationship to obtain the manipulated visual appearance image. Subsequently, input the manipulated visual appearance image and the original face image into the forgery discriminator to judge its authenticity. The training loss function is: Among them, D represents the forgery discriminator, V represents the visual appearance editor, x represents the input original face image, D(x) represents the output of the forgery discriminator, and V(x) represents the output of the visual appearance editor. represents the loss function of the forgery discriminator, represents the loss function of the visual appearance editor; The S2 further includes: After the training of the visual appearance editor model is completed, train the ciphertext embedder and the anonymous recognizer together so that the anonymous recognizer can accurately extract the ciphertext of the identity relationship embedded in the anonymous face image. The training loss function is: x ′ = V(x) Among them, R is an anonymous identifier, E is a ciphertext embedder, I is a visual enhancer, V is a visual appearance editor, and c k represents the k-th bit of the ciphertext of the identity relationship, and n is the length of the ciphertext of the identity relationship. represents the recognized ciphertext of the identity relationship, and E(I(x ′ )) represents the ciphertext of the identity relationship embedded in the image after visual appearance editing; The S6 includes: In the recognition stage, the face image is already anonymous. Use the anonymous recognizer to extract the ciphertext of the identity relationship from the anonymous image. Subsequently, compare the extracted ciphertext of the identity relationship with the ciphertext of the identity relationship in the relationship ciphertext library, and sort the ciphertext of the identity relationship with the closest L1 distance as the recognition result. The corresponding picture is the image before anonymization. The saved ciphertext of the identity relationship is formed by saving the original image and its corresponding ciphertext of the identity relationship in text during the process of obtaining the anonymous image with the embedded ciphertext of the identity relationship through the anonymization process, forming a relationship ciphertext library.

2. The method for generating and recognizing face anonymized images based on maintaining identity relationships according to claim 1, wherein, The binary encrypted text is a random binary code or is generated using the MD5 or SHA-256 encryption algorithm.

3. The method for generating and recognizing face anonymized images based on maintaining identity relationships according to claim 1, characterized in that, The S2 further includes: During the training process, the face images output by the visual appearance editor are input into an identity extractor to compare the face images after appearance editing with the original face images, so as to improve the appearance editing performance of the images generated by the visual appearance editor. During the training process, and the loss function is used to guide the visual appearance editor: Among them, V represents the visual appearance editor, F represents the identity extractor, c1 and c1 are two different identity relationship ciphertexts, and y represents the identity relationship ciphertext of the original face image. denotes the identity relationship ciphertext extracted from the image x after appearance editing i ′ , F(x) represents the output of the identity extractor, and F emb represents the function for extracting face embedding features.

4. The method for generating and recognizing face anonymous images based on identity relationship preservation according to claim 1, wherein, The S2 further includes: Train the ciphertext embedder and the visual enhancer together to reduce the impact of the embedded ciphertext of the identity relationship on the visual quality of the anonymous image. The training loss function is: Among them, E is the ciphertext embedder, and I is the visual enhancer. Denotes the L2 norm of two vectors.

5. A storage medium, characterized in that, The storage medium stores a computer program, and the computer program is used to execute the steps of the method for generating and recognizing an anonymous face image while maintaining the identity relationship according to any one of claims 1 to 4.

6. An electronic device, characterized in that, Including a memory and a processor. A computer program is stored in the memory, and when the processor calls the computer program in the memory, the steps of the method for generating and recognizing an anonymous face image while maintaining the identity relationship according to any one of claims 1 to 4 are implemented.

Images