A face privacy protection method based on thumbnail keeping

Abstract

The application discloses a face privacy protection method based on thumbnail preservation, which comprises the following steps: firstly, obtaining an original face image dataset and a face image dataset encrypted by thumbnail preservation; training a generative adversarial network model with a disturbance generation module by using the two datasets; generating a face picture with details erased but thumbnail preserved by using the network model, and the generated picture can be used to attack a face recognition model; setting a thumbnail consistency loss and a feature adversarial loss, and iteratively training to optimize the network model. The generated protection picture can resist the attack of the face recognition model, and the image owner with prior knowledge can identify the content of the image through the thumbnail of the generated picture, but the enemy without prior knowledge cannot obtain the information in the image.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, and in particular to a face privacy protection method based on thumbnail preservation. BACKGROUND

[0002] With the continuous development of technology, the popularity of smartphones with photography functions is increasing. People can use them to record the moments of life at any time and anywhere. However, these photos may contain personal sensitive information, such as the user's interpersonal relationships, and some file materials containing personal information. At the same time, the rapid development of cloud storage makes users gradually accustomed to uploading information in smartphones, including images taken or saved, to some cloud servers, such as Baidu Cloud. This storage solution can well avoid information loss caused by damage or loss of user hardware devices, but at the same time increases the probability of user privacy leakage when uploading to the cloud. In 2015, a user information leakage event occurred in Meizu cloud service, and the user's address book, SMS, note information, etc. were disclosed. Images uploaded to the cloud have also been leaked, such as the 2014 Hollywood scandal event that caused a sensation. In this major privacy leakage accident, hundreds of Hollywood actors' private photos uploaded to iCloud cloud service were stolen by hackers. In 2017, the Hollywood scandal event was exposed again, and this time the event was also due to the leakage of images in iCloud. Recently, iCloud announced that they have the right to scan images uploaded by users, and the reason is to identify child pornography content in images.

[0003] Thanks to the development of deep convolutional neural networks, research related to face recognition has made great progress in recent years. The most advanced face recognition system can achieve a recognition accuracy of up to 99.8%, which is even better than humans. Currently, face recognition systems are widely used in device authentication, payment verification, access control systems, etc. However, face recognition systems are a double-edged sword. Due to their continuous updating and iteration and wide use, people have begun to worry that illegal attackers will use this tool to gain private benefits. At present, a large number of pictures will be uploaded by users to social platforms, and these social platforms or third-party attackers may use face recognition systems to obtain private information from images uploaded by users, such as the user's interpersonal relationships and family relationships. For example, Meta collected face information from images uploaded by users without their consent, and was therefore sued by users. The development of face recognition has brought convenience to users, but also increased their concerns about information leakage of photos uploaded to the cloud.

[0004] A common method to protect images from face recognition systems is to add adversarial perturbations to the images. This method is based on the vulnerability of neural networks, that is, a model can be fooled by adversarial samples with tiny perturbations and output incorrect results. These perturbations are often imperceptible to the human eye, but machines can correspond the identity of the person in the perturbed picture to another person.

[0005] However, in addition to the above face recognition systems, a curious human adversary without prior knowledge can also try to obtain valuable information from the pictures. For example, employees of cloud storage services can directly access pictures uploaded to the cloud and identify them using the naked eye. The scheme of adding imperceptible perturbations to images to the naked eye cannot resist such attacks.

[0006] For this type of adversary, traditional image encryption can effectively protect face image privacy. For example, traditional encryption schemes encrypt pictures into snowflake pictures, completely erasing the privacy information of the image. However, such schemes protect images while also causing many problems, such as users being unable to identify images if they do not decrypt the images, which reduces the usability of encrypted images; at the same time, using traditional number theory-based image encryption schemes to encrypt images is large, which is undoubtedly inconvenient for users with limited storage space devices. Some image degradation schemes, such as blurring and mosaicking, can also be used to protect face image privacy. However, such schemes are usually irreversible, that is, users cannot recover the original image from the converted image. Such schemes can be used to protect the privacy of images on social networks, but their irreversibility is undoubtedly a major drawback for images that need to be uploaded to a network disk.

[0007] In order to protect image privacy while maintaining the usability of encrypted images, a thumbnail-preserving encryption scheme is proposed. This scheme erases the detailed information of the image while keeping the thumbnail of the encrypted image the same as the thumbnail of the original image, thus preserving the usability of the picture to some extent. On this basis, the owner of the picture with prior knowledge can identify the content of the image through the thumbnail of the encrypted image, while the adversary without prior knowledge cannot identify the content of the image. However, in existing thumbnail-preserving encryption schemes, when the thumbnail block is small, existing advanced face recognition systems can still recognize the face in the image. Therefore, there is a need for a novel image protection scheme that maintains the usability of images while resisting recognition by humans and machines. SUMMARY

[0008] The technical problem to be solved by the present application is to provide a thumbnail-preserving face privacy protection method to solve the defects involved in the background art.

[0009] The present application adopts the following technical solutions to solve the above technical problems:

[0010] A face privacy protection method based on thumbnail preservation, comprising the following steps:

[0011] Step 1), randomly select several images from the preset face image dataset CelebA as a normal face training set trainA, and perform thumbnail preservation encryption on the images in the training set trainA to generate a training set trainB, the images in trainA belong to domain A, and the images in trainB belong to domain B;

[0012] Step 2), establish a generation model G A , a generation model G B , a discrimination model D A , a discrimination model D B , a perturbation generation model H, wherein the generation model G A is used to convert the images in the training set trainA into images fake_B that have consistent thumbnails and original images and have erased detailed information; the perturbation model H is used to generate a small perturbation adv and superimpose it on fake_B to obtain a protected image adv_fake_B; the generation model G B is used to restore the generated image adv_fake_B and obtain a restored original image rec_A; the discrimination model D A is used to discriminate whether the generated image is within the A domain; the discrimination model D B is used to discriminate whether the generated image is within the B domain;

[0013] Step 3), train the generation model G A , the generation model G B , the discrimination model D A , the discrimination model D B , and the perturbation generation model H using a loss function;

[0014] The loss function includes an adversarial loss L GAN_A , an identity consistency loss L Idt_A , a thumbnail Figure One consistency loss L TPE , an adversarial loss L GAN_B , an identity consistency loss L IdT_B , a cycle consistency loss L Cycle , an adversarial loss L D_A , an adversarial loss L D_B , a thumbnail Figure One consistency loss L TPE_adv , an adversarial loss L Adv , and a perturbation loss L per , wherein:

[0015] The adversarial loss L GAN_A is a loss function used to constrain the generation model G A , using the discriminator D B to judge whether the generated image belongs to the domain B, thereby constraining the generation model G A ; A The generated image belongs to the domain B;

[0016] The identity consistency loss L Idt_A is used to constrain the image in the B domain to be input into G A , and then constrain G A to output an image as close as possible to the input image;

[0017] The non-disturbance thumbnail consistency loss L Figure One is used to constrain the generated image fake_B to be as close as possible to the thumbnail of the original image real_A accepted by G TPE ; A The output image; A The output image;

[0018] The adversarial loss L GAN_B is a loss function used to constrain the generation model G B , using the discriminator D A to judge whether the generated image belongs to the domain A, thereby constraining the generation model G B ; B The generated image belongs to the domain A;

[0019] The identity consistency loss L Idt_B is used to constrain the image in the A domain to be input into G B , and then constrain G B to output an image as close as possible to the input image;

[0020] The cycle consistency loss L Cycle is used to constrain the generation model G A and G B , and is used to constrain the original image real_A input into G A and the image output by G A to be as close as possible to the image rec_A output by G B after G B ;

[0021] The adversarial loss L D_A is a loss function used to constrain the discriminator D A , that is, the discriminator D A judges the correctness of the domain of the picture real_A and rec_A in trainA.

[0022] The adversarial loss L D_B is a loss function for constraining the discrimination model D B , that is, the discriminator D B corresponding to the loss of judging the correctness of the domain of the picture real_B and fake_B in trainB;

[0023] The perturbation has a thumbnail Figure One The consistency loss L TPE_adv is a loss function for constraining the perturbation generation model H, which is used to constrain the thumbnail of the protected image adv_fake_B superimposed with the perturbation generated by H to be as same as the original image real_A as possible;

[0024] The adversarial loss L Adv , that is, the loss for constraining the face features of the image adv_fake_B superimposed with the perturbation to be as different as possible from the face features of the original image real_A;

[0025] The perturbation loss L per is used to constrain the size of the perturbation generated by H;

[0026] Step 4), using the generation model G A and the perturbation generation model H to transform the image to be protected, generate a new image that retains the thumbnail and is resistant to attacks on the face recognition model; using the generation model G B restore the new image to obtain a restored image that is basically consistent with the original image.

[0027] As a further optimization scheme of the thumbnail retention-based face privacy protection method of the present application, the detailed steps of the thumbnail retention encryption of the image in step 1) are as follows:

[0028] Step 1.1), divide the image into (H / b)×(W / b)×3 thumbnail blocks, each of which has b×b pixels, wherein H is the height of the image, W is the width of the image, 3 is the channel number of the image, and b×b is the size of the image generated by the thumbnail retention encryption;

[0029] Step 1.2), regard the pixels in each thumbnail block as a b×b two-dimensional array, and reduce its dimension to a one-dimensional array arr with length b×b;

[0030] Step 1.3), use a shuffling algorithm to randomly shuffle all one-dimensional arrays arr;

[0031] Step 1.4), convert the one-dimensional array arr after random shuffling back to a b×b two-dimensional array, and put the new array elements back into the thumbnail block.

[0032] As a further optimization scheme of the thumbnail-based face privacy protection method of the present application, the L GAN_A , GAN_B , Cycle , Idt_A , Idt_B , TPE The calculation formulas of L

[0033] L GAN_A = MSELoss(D B (fake_B), true)

[0034] L GAN_B = MSELoss(D A (rec_A), true)

[0035] L Cycle = L1Loss(rec_A, real_A)

[0036] L Idt_A = L1Loss(real_B, G A (real_B))

[0037] L Idt_B = L1Loss(real_A, G B (real_A))

[0038] L TPE = L1Loss(Thumbnail(fake_B), Thumbnail(real_A))

[0039] In the formula, D B (x) is the result of sending x into discriminator D B ; D A (x) is the result of sending x into discriminator D A ; MSELoss(x, y) refers to the mean square error of x and y; L1Loss(x, y) refers to the L1 distance of x and y; G A (real_B) is the result of sending real_B into G A ; G B (real_A) is the result of sending real_A into G B ; Thumbnail(x) refers to the thumbnail of x.

[0040] The loss functions of the generation models G A , G B are as follows:

[0041] As a further optimization scheme of the thumbnail-based face privacy protection method, the calculation formulas of L D_A , D_B in step 3) are as follows:

[0042] L D_A = MSELoss(D A (real_B), true) + MSELoss(D A (fake_B), false)

[0043] L D_B = MSELoss(D B (real_A), true) + MSELoss(D B (rec_A), false)

[0044] The loss function L A of the generation model D B is L D = L D_A + L D_B .

[0045] As a further optimization scheme of the thumbnail-based face privacy protection method, the calculation formulas of L TPE_adv , L Adv , and L per in step 3) are as follows:

[0046] L TPE_adv = L1Loss(Thumbnail(adv_fake_B), Thumbnail(real_A))

[0047] L Adv = Cosine_distance(FR(adv_fake_B), FR(real_A))

[0048] L per = L2Loss(per)

[0049] In the formula, FR(x) is the face feature of x, Cosine_distance(x, y) is the cosine distance of x and y, and L2Loss(x) is the L2 norm of x.

[0050] The loss function L H of the generation model H is L TPE_adv = L Adv + L per .

[0051] Compared with the prior art, the above technical scheme has the following technical effects:

[0052] The present application combines visual psychology research and proposes a face privacy protection method based on thumbnail preservation. An adversarial model is used to reversibly convert images. A loss function is used to constrain the generation model so that the thumbnail of the original image and the converted image are consistent. At the same time, the converted image is mapped to the desired domain through adversarial learning, and a disturbance generation module is used to generate disturbance to prevent machine recognition. Users can set different resolution thumbnails to balance image availability and privacy. When the thumbnail block size is larger, more detailed information is erased, and the picture retains less availability; when the thumbnail block size is smaller, less detailed information is erased, and the picture retains more availability. It is worth noting that the converted image has the ability to resist machine recognition, regardless of the set thumbnail block size. BRIEF DESCRIPTION OF DRAWINGS

[0053] Figure 1 is the loss function constraint diagram of the generation model G A of the present application B ;

[0054] Figure 2 is the loss function constraint diagram of the discriminator D A , D B of the present application

[0055] Figure 3 is the loss function constraint diagram of the generation model H of the present application

[0056] Figure 4 is the image conversion flowchart of the present application DETAILED DESCRIPTION

[0057] The technical solutions of the present application will be further described in detail below in combination with the drawings:

[0058] The present application can be implemented in many different forms, and should not be considered limited to the embodiments described herein. On the contrary, these embodiments are provided to make the present disclosure thorough and complete, and to fully convey the scope of the present application to those skilled in the art. In the drawings, the components are enlarged for clarity.

[0059] The process of constructing the training set in the present application is as follows:

[0060] Randomly select several images from the pre-set face image dataset CelebA as normal face training set trainA, and generate training set trainB by thumbnail preservation encryption of the images in training set trainA. The images in trainA belong to domain A, and the images in trainB belong to domain B.

[0061] In this embodiment, the number of pictures in trainA and trainB is 10000, and the picture size is 256*256*3;

[0062] In this embodiment, the process of keeping the thumbnail encrypted is as follows:

[0063] Step 1.1), the image is divided into 64*64*3 thumbnail blocks, and each thumbnail block has 4*4 pixels;

[0064] Step 1.2), the pixels in each thumbnail block are taken as a 4*4 two-dimensional array, and are reduced to a one-dimensional array arr with a length of 16;

[0065] Step 1.3), all one-dimensional arrays arr are randomly shuffled using a shuffle algorithm;

[0066] Step 1.4), the one-dimensional array arr after random shuffling is converted into a 4*4 two-dimensional array again, and the new array elements are put back into the thumbnail block.

[0067] In this embodiment, the process of training the model is as follows:

[0068] Establishing a generation model G A , a generation model G B , a discrimination model D A , a discrimination model D B , a disturbance generation model H, wherein the generation model G A is used to convert the image real_A in the training set trainA into an image fake_B which is consistent with the original thumbnail and has the detail information removed; the disturbance model H is used to generate a small disturbance adv and superimpose it on fake_B to obtain a protected image adv_fake_B; the generation model G B is used to restore the generated image adv_fake_B and obtain the restored original image rec_A; the discrimination model D A is used to discriminate whether the generated image is in the A domain; the discrimination model D B is used to discriminate whether the generated image is in the B domain.

[0069] The generation model G A , the generation model G B , the discrimination model D A , the discrimination model D B , and the disturbance generation model H are trained using a loss function.

[0070] The generation model G A , the generation model G B is constrained by a loss function as shown in Figure 1 , including an adversarial loss L GAN_A, identity consistency loss L Idt_A , non-disturbance thumbnail Figure One , identity consistency loss L TPE , adversarial loss L GAN_B , identity consistency loss L Idt_B , cycle consistency loss L Cycle , wherein:

[0071] The adversarial loss L GAN_A is a loss function for constraining the generation model G A , using the discriminant model D B to judge whether the image generated by G A belongs to domain B, so as to constrain the generation model G A to generate images belonging to domain B, and its calculation formula is:

[0072] L GAN_A = MSELoss(D B (fake_B), true),

[0073] In the formula, D B (x) result is true when the discriminator D B thinks that x belongs to the B domain, and MSELoss(x, y) means the mean square error of x and y;

[0074] The identity consistency loss L Idt_A is used to send the image of the B domain into G A , and constrain the output image of G A and the input image to be as same as possible, and its calculation formula is:

[0075] L Idt_A = L1Loss(real_B, G A (real_B)),

[0076] In the formula, L1Loss(x, y) is the L1 distance of x and y, and G A (real_B) is the result obtained by sending real_B into G A ;

[0077] The non-disturbance thumbnail Figure One consistency loss L TPE is used to constrain the generated image fake_B of G A and the thumbnail of the original image real_A accepted by G A to be as same as possible, and its calculation formula is:

[0078] L TPE = L1Loss(Thumbnail(fake_B), Thumbnail(real_A)),

[0079] where Thumbnail(x) denotes a thumbnail of x, and the thumbnail block size is 4x4;

[0080] The adversarial loss L GAN_B is a loss function for constraining the generation model G B , using the discriminator D A to judge whether the image generated by G B belongs to the domain A, so as to constrain the generation model G B to generate an image belonging to the domain A, and the calculation formula is as follows:

[0081] L GaN_B = MSELoss(D A (rec_A), true),

[0082] where D A (x) is true when the discriminator D A considers that x belongs to the domain A;

[0083] The identity consistency loss L Idt_B is used to constrain the image real_A in the domain A to be input into G B , so that the image output by G B is as same as the input image as much as possible, and the calculation formula is as follows:

[0084] L Idt_B = L1Loss(real_A, G B (real_A)),

[0085] where G B (real_A) is the result obtained by inputting real_A into G B ;

[0086] The cycle consistency loss L Cycle constrains the generation model G A and G B at the same time, and is used to constrain the original image real_A input into G A and the image output by G A to be input into G B , so that the image rec_A output by G B is as same as the image input into G Cycle as much as possible, and the calculation formula is as follows:

[0087] L D_A = L1Loss(rec_A, real_A)

[0088] The loss function for constraining the discriminator L D_B is as follows:Figure 2 As shown, this includes counter-loss L D_A , countering losses L D_B ;

[0089] The resistance loss L D_A To identify model D A The loss function for applying constraints, i.e., the discriminator D A The loss for judging the correctness of the domain of images real_A and rec_A in trainA is calculated using the following formula:

[0090] L D_A =MSELoss(D A (real_B),true)+MSELoss(D A (fake_B),false)

[0091] The resistance loss L D_B To identify model D B The loss function for applying constraints, i.e., the discriminator D B The loss for judging the correctness of the domain of images real_B and fake_B in trainB is calculated using the following formula:

[0092] L D_B =MSELoss(D B (real_A),true0+MSELoss(D B (rec_A),false)

[0093] The loss function constrained by the perturbation generation model is as follows: Figure 3 As shown, this includes abbreviated forms with perturbations. Figure One Loss of sexual function L TPE_adv , antagonistic loss L Adv and disturbance loss L per ;

[0094] The perturbation abbreviation Figure One Loss of sexual function L TPE_adv The loss function constrains the perturbation generation model H, and is used to constrain the thumbnail of the protected image adv_fake_B, which is superimposed with the perturbation generated by H, to be as similar as possible to the original image real_A. Its calculation formula is as follows:

[0095] L TPE_adv =L1Loss(Thumbnail(adv_fake_B),Thumbnail(real_A)),

[0096] The adversarial loss L AdvThat is, for the face feature of the image superimposed with the disturbance adv_fake_B and the face feature of the original image real_A, the constraint that they are as different as possible is used, and the calculation formula is:

[0097] L Adv = Cosine_distance (FR (adv_fake_B), FR (real_A)),

[0098] In the formula, Cosine_distance (x, y) is the cosine distance of x and y, and FR (x) is the face feature of x.

[0099] The disturbance loss L per is used to constrain the size of the disturbance generated by H, and the calculation formula is:

[0100] L per = L2Loss (per),

[0101] In the formula, L2Loss (x) is the L2 norm of x.

[0102] The image conversion schematic diagram of the application is shown in Figure 4 , and specifically includes the following steps:

[0103] Image generation: input the original image real_A of the size of 256*256*3 to be protected into the generation model G A , and the convolutional neural network extracts features from the image and converts the features. Then, the converted features are decoded using the deconvolution layer, and finally the image in the generated B domain, that is, the image fake_B which retains the thumbnail but erases the detailed information. Then, the disturbance generator H extracts the features from fake_B and generates a small disturbance adv. The disturbance is superimposed on the fake_B image to obtain the final protected image adv_fake_Loss, that is, the conversion which realizes both the consistency of the thumbnail and the resistance to machine face recognition attack.

[0104] Image recovery: input the image adv_fake_B of the last step into the generation model G B , and the convolutional neural network extracts features from the input image and converts the features to the A domain. Finally, the deconvolution layer restores the original image rec_A using the features.

[0105] As used herein, and unless otherwise indicated, all terms have their ordinary meanings. It should be understood that any definitions are to be used as elucidated herein and should not be taken in any restrictive sense unless expressly defined.

[0106] The above description is further explained with reference to the specific embodiments. The specific embodiments are illustrations of a particular implementation of the present application. Numerous specific details are described in connection with the embodiments. However, in other implementations, different or similar specific details can be used. In the description herein, numerous specific details are set forth in order to provide a thorough understanding of the present application. However, it will be apparent to one skilled in the art that the present application can be practiced without incorporating these specific details. In general, the description and examples are intended to provide an overview for understanding the application innovation and the substance desired to be protected by the patent. They are not intended to serve as limitations of the application innovation's scope as it is defined by the claims.

Claims

1. A face privacy protection method based on thumbnail keeping, characterized in that, The method comprises the following steps: Step 1), randomly selecting a plurality of images from the preset face image dataset CelebA as a normal face training set trainA, and performing thumbnail maintaining encryption on the images in the training set trainA to generate a training set trainB, the images in trainA belong to domain A, and the images in trainB belong to domain B; Step 2), establishing a generation model G A , the generation model G B , the discrimination model D A , the discrimination model D B , the disturbance generation model H, wherein the generation model G A is used for converting the image real_A in the training set trainA into an image fake_B which is consistent with the original image and has the detailed information removed; the disturbance model H is used for generating a slight disturbance adv and superimposing it on fake_B to obtain a protected image adv_fake_B; the generation model G B is used for restoring the generated image adv_fake_B and obtaining a restored original image rec_A; the discrimination model D A is used for discriminating whether the generated image is in the A domain; the discrimination model D B is used for discriminating whether the generated image is in the B domain; Step 3), training the generation model G using a loss function A , the generation model G B , the discrimination model D A , the discrimination model D B , the perturbation generation model H is trained; The loss function comprises an adversarial loss L GAN_A , an identity consistency loss L Idt_A , a non-disturbance thumbnail consistency loss L TPE , an adversarial loss L GAN_B , an identity consistency loss L Idt_B , a cycle consistency loss L Cycle , an adversarial loss L D_A , an adversarial loss L D_B , a disturbance thumbnail consistency loss L TPE_adv , an adversarial loss L Adv , and a disturbance loss L per , wherein: The adversarial loss L GAN_A To constrain the generation model G A The loss function is used to constrain the generation model G B The generated image is judged by the discriminator model D A Whether it belongs to domain B, thereby constraining the generation model G A The generated image belongs to domain B; The identity consistency loss L Idt_A For sending the image of B domain into G A After that, constraint G A The output image and the input image are as same as possible; The disturbance-free thumbnail consistency loss L TPE For constraining G A The generated image fake_B and G A The thumbnail of the accepted original image real_A is as identical as possible; the adversarial loss L GAN_B To constrain the generation model G B Loss function, using the discriminant model D A G B generated images belong to domain A, thereby constraining the generation model G B generated images belong to domain A; The identity consistency loss L Idt_B For sending the image real_A of the A domain into G B After that, constrain G B The output image and the input image are as same as possible; The cycle consistency loss L Cycle The generated model G A And G B Is constrained, which is used to constrain the input to G A The original image real_A and the image output by G A Input to G B The image rec_A output by G B Is as identical as possible; The adversarial loss L D_A For the discriminant model D A The loss function that constrains the discriminator D A The loss corresponding to the correctness of the domain to which the picture real_A and rec_A in trainA belong The adversarial loss L D_B To the discriminant model D B The loss function that is constrained, namely the discriminator D B The loss corresponding to the correctness of the domain to which the pictures real_B and fake_B in trainB belong. Disturbed thumbnail consistency loss L TPE_adv A loss function is used to constrain the model H generating the disturbance, so that the thumbnail of the protected image adv_fake_B superimposed with the disturbance generated by H is as identical as possible to the thumbnail of the original image real_A. The adversarial loss L Adv That is, to constrain the facial features of the image superimposed with the disturbance adv_fake_B to be as different as possible from the facial features of the original image real_A; the perturbation loss L per to constrain the magnitude of the perturbation generated by H L GAN_A , L GAN_B , L Cycle , L Idt_A , L Idt_B , L TPE The calculation formulas of L, L, L, L, L, L, and L are respectively: L GAN_A = MSE Loss(D B (fake_B), true) L GAN_B = MSELoss(D A (rec_A), true) L Cycle = L1Loss(rec_A, real_A) L Idt_A = L1Loss(real_B, G A (real_B)) L Idt_B = L1Loss(real_A, G B (real_A)) L TPE = L1Loss(Thumbnail(fake_B), Thumbnail(real_A)) where D B (x) is the result of sending x into discriminator D B where D A (x) is the result of sending x into discriminator D A where D A (real_B) is the result of sending real_B into G A where D B (real_A) is the result of sending real_A into G B where D For the generator model G A , G B , the loss function L D_A , L D_B is calculated as follows: L D_A = MSELoss(D A (real_B), true) + MSELoss(D A (fake_B), false) L D_B = MSELoss(D B (real_A), true) + MSELoss(D B (rec_A), false) For the generative model D A , the loss function L B is defined as L D = L D_A + L D_B . L TPE_adv , L Adv , L Per The calculation formulas are respectively: L TPE_adv = L1Loss(Thumbnail(adv_fake_B), Thumbnail(real_A)) L Adv = Cosine_distance(FR(adv_fake_B), FR(real_A)) L per = L2Loss(per) In the formula, FR(x) is the face feature of x, cosine_distance(x, y) is the cosine distance of x and y, and L2Loss(x) is the L2 norm of x; Loss function L for the generative model H H = L TPE_adv + L Adv + L per ; Step 4), using the generative model G A and the perturbation generative model H to transform the image to be protected, to generate a new image that retains the thumbnail and is resistant to attacks on the human face recognition model; using the generative model G B restore the new image to obtain a restored image that is substantially consistent with the original image. 2.The thumbnail-based face privacy preserving method of claim 1, wherein, The detailed steps of performing thumbnail maintaining encryption on the images in step 1) are as follows: Step 1.1), dividing the image into (H / b)×(W / b)×3 thumbnail blocks, each of which has b×b pixels, wherein H is the height of the image, W is the width of the image, 3 is the channel number of the image, and b×b is the size of the image generated by performing thumbnail maintaining encryption; Step 1.2), regarding the pixels in each thumbnail block as a b×b two-dimensional array, and reducing the dimension to a one-dimensional array arr with a length of b×b; Step 1.3), using a shuffling algorithm to randomly permute all one-dimensional arrays arr; Step 1.4), converting the one-dimensional array arr after random permutation into a b×b two-dimensional array, and putting the new array elements back into the thumbnail block.

Images