A Differential Privacy-based Human Anonymization Synthesis Method with Consistent Usability

Abstract

The present invention discloses a differential privacy human body anonymization synthesis method with consistent usability. The present invention is characterized in that an anonymous human body dataset is obtained by passing a source human body dataset through a delegated human body dataset and a usability-preserving pose conversion network; specific steps: Step 1: Preprocessing; Step 2: Assigning a delegated identity; Step 3: Constructing a usability-preserving pose conversion network; Step 4: Training constraints; Step 5: Model training and testing. The present invention can generate new anonymous human body images, avoiding the collection and abuse of privacy data, and the anonymous images can maintain the usability of the original data. In addition, the present invention can generate natural images under different privacy levels, and maintain human body privacy and data usability.

Description

Technical Field

[0001] The present invention relates to the field of image privacy protection, and proposes a differential privacy human body anonymization synthesis method with consistent usability. Background Art

[0002] The unprecedented application of image acquisition devices (such as smart phones and surveillance cameras) has led to the generation of a large number of private human body images in private areas as well as public areas, and these images are easily accessible if they are released to the Internet. At the same time, advanced artificial technologies (such as behavior analysis, pedestrian re-identification, face recognition, etc.) can use these data to generate applications with excellent performance and play an important role in today's society. However, these applications may be misused by criminals, resulting in serious privacy leakage problems.

[0003] To solve the above privacy problems, some countries have formulated increasingly strict laws and even taken down public data sets without privacy protection, which may lead to the progress of many important artificial intelligence researches. Therefore, it is very important to develop effective privacy protection technologies to solve privacy problems, and the goal is to make the original images unable to be misused and the generated anonymized images can be reused without considering privacy. The above privacy protection technologies should be able to help manage human body image data in different scenarios. Currently, common privacy protection methods such as blurring and mosaics mainly focus on protection, but cannot guarantee the usability of anonymized images. In fact, balancing data privacy protection and usability is an open and challenging problem.

[0004] In recent years, remarkable progress in the research of generative adversarial networks has made it possible to synthesize real images, which has opened up a new path for human body replacement synthesis in human body privacy protection. Compared with traditional blurring and mosaics, the anonymized images generated by generative adversarial networks are more realistic. A typical example is DeepFake, but simple replacement will damage the usability of the original image and violate the privacy of the replaced person. Therefore, developing specific privacy synthesis technologies is crucial for visual image security. Currently, most researches are based on face anonymization, but this is insufficient for human body anonymization because in addition to the face, the human body includes a large amount of identity information. For example, in a surveillance camera, a human body can be identified even if his face is blocked. In addition, the latest pedestrian re-identification algorithms can also identify human body identities even if their faces are blocked. Summary of the Invention

[0005] The purpose of the present invention is to provide a differential privacy human body anonymization synthesis method with consistent usability in view of the deficiencies of the prior art. The present invention is characterized in that the source human body data set passes through a delegated human body data set and a usability-preserving pose conversion network to obtain an anonymized human body data set

[0006] The technical solution adopted by the present invention to solve the technical problem includes the following steps:

[0007] Step 1: Preprocessing;

[0008] Step 2: Assign a delegated identity;

[0009] Step 3: Construct an availability-preserving posture transition network;

[0010] Step 4: Training constraints;

[0011] Step 5: Model training and testing.

[0012] Further, step 1 preprocessing, the specific steps are as follows:

[0013] 1-1. Delegating human data sets Collect, use public datasets or synthetic datasets as commissioned human datasets

[0014] 1-2. Image labeling, for the human body dataset Mark required non-sensitive attributes;

[0015] 1-3. Parsing the delegated human body dataset using the pose parser The human body image in the image is obtained to obtain the key points of the human body;

[0016] 1-4. Use the instance segmentation model to obtain the human body region mask image.

[0017] Furthermore, step 2 allocates the delegated identity. The specific steps are as follows:

[0018] 2-1. Construct attribute consistent candidate set;

[0019] Use the pre-trained attribute classifier to classify the original human image I x Attribute A x Make predictions; attribute consistent candidate set S x From the delegated human dataset Select attribute and A x A consistent delegated identity is constructed, wherein the selected attributes are all non-sensitive and are selected by maximizing the attribute consistency selection method;

[0020] 2-2. Differential Privacy Identity Delegation

[0021] Delegating identities to anonymize human beings is actually achieved by properly designing a mapping from the original identity to the delegated identity. In order to find a suitable delegated identity, we introduce an image-level differential privacy random identity mapping φ. In order to protect φ from differential attacks that cause attackers to steal the original identity through the delegated human sd Infer I x , we design identity mapping based on ∈-differential privacy. Specifically, we adopt the exponential mechanism to achieve ∈-differential privacy, and the exponential mechanism needs to define an availability function. Since human bodies with the same attributes have smaller feature distances, we define the availability function based on pedestrian features. Human bodies with closer feature distances have higher availability. According to the availability function, candidate identities in the candidate set with consistent attributes have different probabilities of being selected as the delegated identity, and here a uniform distribution is used to select the delegated identity.

[0022] Furthermore, step 3 constructs a pose conversion network for maintaining availability, and the specific steps are as follows:

[0023] For a given original human body image First, use the delegated human body image corresponding to the delegated identity assigned in step 2 Then use the UPT network to replace the human body in the original human body image I x to obtain an anonymous target image The UPT network is as follows:

[0024] 3-1. Construct a generator;

[0025] The generator consists of several encoders and a decoder, including encoders E x , E p , E a , E f and a decoder where E x and E p are composed of 3 convolutional layers, E f is composed of 2 convolutional layers, E a is composed of 7 fully connected layers, and is composed of a progressive pose transfer model;

[0026] 3-2. Construct a discriminator;

[0027] The UPT network uses two discriminators D p and D t to train the generator adversarially; discriminator D p focuses on judging whether the pose of the target image corresponds to the pose of the source human body and is composed of multiple convolutional layers and residual blocks; discriminator D t uses the same 7-layer fully connected layer as encoder E a to discriminate the authenticity and attribute consistency of the generated human body image;

[0028] 3-3. Background restoration module;

[0029] Use the original human body image I xThe background sticker is pasted onto the generated human body image and then input into the discriminator D p and D t In this way, the original background can be well restored.

[0030] Furthermore, the training constraints in step 4 are as follows:

[0031] 4-1. The complete objective function of UPT is defined as:

[0032]

[0033] where L GD is the adversarial loss between the generator and the discriminator, L s is the identity exchange loss, L r is the reconstruction loss, λ 1 、λ 2 and λ 3 are parameters that balance the importance of different losses;

[0034] 4-2. The adversarial loss L GD ;

[0035] The adversarial loss can be expressed as follows:

[0036]

[0037] D p (G(z x , A x , P x ), P x ))] (2)

[0038] where A x and P x are the attribute and pose heatmaps sampled from the real data, M x is the human body region mask in the original human body image I x , is the pose heatmap of the delegated human body image s d ;

[0039] 4-3. The identity exchange loss;

[0040] When the identities of the original human body image I x and the delegated human body image s d come from different humans in the training samples, we define the identity exchange loss:

[0041]

[0042] where and respectively represent the original human body image I x the corresponding source identity and the delegated human body image s d the corresponding delegated identity, H(T x , s d ) is a relaxed identity feature constraint, defined as:

[0043] H(T x , s d ) = max{|f(s d ) - f(T x )| 1 -α, 0} (4)

[0044] where f is a pre-trained human feature extractor; during the pose transformation process, using a margin α > 0 can prevent complete identity transfer;

[0045] 4 - 4. Reconstruction loss;

[0046] When the source identity and the delegated identity come from the same human body in the training samples, the reconstruction loss L r is defined as the l 1 distance at the feature level and the pixel level:

[0047]

[0048] where the feature-level l 1 distance U(T x , I x ) = |f(T x ) - f(I x )| 1 , and the pixel-level l 1 distance V(T x , I x ) = |T x - I x | 1 , and β is a parameter that balances the feature and pixel l 1 distance.

[0049] Furthermore, step 5 is to train the model and test, and the specific steps are as follows:

[0050] 5 - 1. Prepare the dataset (such as the public dataset Market - 1501 and DukeMTMC - reID) and process it according to the requirements of step 1.

[0051] 5 - 2. Combine step 3 and step 4 to train UPT, and anonymize the original dataset using the delegated identity assignment method proposed in step 2.

[0052] 5-4. To verify the effectiveness of the proposed method, it is compared with the current excellent methods, and the anonymity rate, attribute retention rate, pose retention rate, and image quality are calculated.

[0053] The beneficial effects of the present invention are as follows:

[0054] The present invention proposes a novel differential privacy human body anonymization synthesis method with consistent usability. This method can generate new anonymized human body images, avoid the collection and abuse of privacy data, and the anonymized images can maintain the usability of the original data. It has the following three beneficial effects: the anonymized images are relatively natural and have no obvious difference compared with normal images, and are very different compared with the original images; the anonymized images generated by this method can retain good data usability for various visual tasks (such as non-sensitive attributes and poses) and can maintain the human body privacy therein; this method can control the generator to generate various anonymized human body images at different privacy levels. Brief Description of the Drawings

[0055] Figure 1 is the flowchart of this method;

[0056] Table 1 shows the comparison experiment results of this method and other methods on the Market-1501 dataset;

[0057] Table 2 shows the comparison experiment results of this method and other methods on the DukeMTMC-reID dataset. Detailed Embodiment

[0058] The present invention will be further described below with reference to the accompanying drawings.

[0059] For a given source image First, the delegated image corresponding to the delegated identity is assigned in step 2 Then, the human body in I x is replaced using the UPT network to obtain the anonymized target image

[0060] The process of the differential privacy human body anonymization synthesis method with consistent usability is as follows Figure 1 shown. For a given input human body image First, a delegated identity (DPA) is assigned to obtain the delegated human body image through differential privacy random identity mapping. Then, the usability-preserving pose conversion network (UPT) is executed to replace the human body in I x to obtain the anonymized target image The DPA process includes two steps: establishing an attribute-consistent candidate set (ACCS); selecting a differential privacy delegated human body (DPDP). The UPT network takes P x 、A x 、Bx , M x , s d , Input into the generator (including encoder E x , E a , E f , E p , and decoder ), and discriminator (including D t and D p ). P x and respectively represent the pose information of I x and s d in the form of a heat map. A x , B x , M x respectively represent non-sensitive attributes, I x background, I x pedestrian area mask. E x Encodes s d and B x to obtain identity and background information, E a Encodes A x to obtain attribute information, E f Fuses the results of E x and E a , E p Encodes P x and to obtain pose conversion information, Decodes the results of E f and E p and inputs them into the background repair module to obtain T x , D p discriminates the authenticity of T x pose, D t discriminates the authenticity of T x image. The specific implementation steps of the present invention are as follows:

[0061] Step 1: Preprocessing;

[0062] Step 2: Assign delegated identity;

[0063] Step 3: Availability-preserving pose conversion network;

[0064] Step 4: Training constraints;

[0065] Step 5: Model training and testing.

[0066] The specific steps of Step 1 preprocessing are as follows:

[0067] 1-1. Delegated human body dataset For the collection, use a publicly available dataset (Market-1501 or DukeMTMC-reID) or a synthetic dataset (such as synthesized through a generative adversarial network) as the delegated dataset.

[0068] 1-2. Image tagging, tagging the non-sensitive attributes (such as gender, clothing, etc.) required for the human body images in the delegated human body dataset in the human body images.

[0069] 1-3. Use a pose parser (such as OpenPose) to parse the human body image to obtain human body key points.

[0070] 1-4. Use an instance segmentation model (such as Mask RCNN) to segment the human body image to obtain a human body region mask image.

[0071] Step 2 Assign a delegated identity, and the specific steps are as follows:

[0072] 2-1. Construct a candidate set with consistent attributes.

[0073] Use a pre-trained attribute classifier to predict the attributes A of the original image I x in the source human body dataset. x Select human body images with attributes consistent with the non-sensitive attributes A from the delegated human body dataset x to construct a candidate set S x with consistent attributes. Among them, the prediction result of the i-th non-sensitive attribute of the attribute classifier on the original image I x is indicating the corresponding non-sensitive attribute softmax score, and m represents the number of corresponding non-sensitive attributes. Let When ρ(x, d) = m, the delegated human body image s d and the original image I x are a perfect match, where is the i-th attribute label of the delegated human body image s d . When the number n of perfect match delegated human body images is less than the preset threshold t, t - n additional images will be added to the candidate set S x with consistent attributes, and the rule is to select the delegated human body image s that maximizes the attribute consistency from the delegated human body dataset d , and the attribute consistency is expressed as

[0074] 2-2. Differential privacy identity delegation.

[0075] To find a suitable delegated human body image s d, the image-level differential random identity mapping φ is introduced. The goal of introducing differential privacy here is to protect the differential random identity mapping φ. For a random mechanism K, if for any adjacent mappings φ 1 and φ 2 and all possible outputs O, it satisfies:

[0076] Pr[K(φ 1 ) ∈ O] ≤ e ∈ Pr[K(φ 2 ) ∈ O]

[0077] then the random mechanism K is said to satisfy ∈-differential privacy. This method uses the exponential mechanism to achieve ∈-differential privacy, and its definition is as follows:

[0078] Since human body images with similar attributes have smaller feature distances, the usability function is defined as where represents the usability of mapping the x-th image from the source human body dataset to the d-th image in the delegated human body dataset, k represents the index of all possible mapped images of the x-th image in the source human body dataset on the delegated human body dataset, and ξ(k, x) = |f(s k ) - f(I x )| 2 , f is a pre-trained human body feature extractor, and s k is the k-th image in the delegated human body dataset. Finally, a random sampling is performed with a probability of to select the identity delegation. Since Pr[φ|x→d] is proportional to exp(∈u(φ, r) / 2Δu), our identity delegation method satisfies the exponential mechanism and thus satisfies ∈-differential privacy.

[0079] ∈-differential privacy is to prevent differential attacks, where ∈ is used to quantify the privacy budget, and the larger ∈ is, the more likely the original data is to be leaked. In the step of constructing the attribute-consistent candidate set (ACCS), the identity domain of the delegation set being different from the identity domain of the input set can also prevent identity re-identification. Potential privacy miners can only obtain the delegated identity information and non-sensitive attributes of the original images. Since the usability function u(φ, r) is defined based on the feature distance, a larger ∈ will lead to a greater probability of leaking the original identity but can improve the usability of the generated images.

[0080] Step 3: Construct a usability-preserving pose conversion network, and the specific steps are as follows:

[0081] 3-1. Construct the generator.

[0082] The generator has six inputs: the delegated human body image s d(3, 128, 64), I x Key point heat map P of x (18, 128, 64), s d Key point heat map of Non-sensitive attribute A x (m), background B x (3, 128, 64) and human body region mask M x (1, 128, 64). P x and In the stacked input E p (E p Consisting of 3 convolutional layers) outputs pose feature F p , non-sensitive attribute A x Input E a (E a Consisting of 7 fully connected layers) outputs attribute feature F a , B x and s d Then stack the input E x (E x Consisting of 3 convolutional layers) outputs image feature F x , attribute feature F a and image feature F x Stack the input E f (E f Consisting of 2 convolutional layers) outputs fusion feature F f , pose feature F p and fusion feature F f Stack the input (9 pose attention transfer blocks, 2 deconvolutional layers, 1 convolutional layer) to obtain I′ x , the final human body region mask M x , background B x and I′ x Input the background restoration module to obtain the anonymized image T x .

[0083] 3-2. Construct the discriminator.

[0084] Discriminator D t Has 2 inputs: anonymized image T x (3, 128, 64) and non-sensitive attribute A x (m). Discriminator D t First uses 7 fully connected layers to encode A x , then uses 2 convolutional layers to encode T x , and finally stacks the encoding results of A x and T x Input into 2 convolutional layers, 3 residual blocks, and 1 sigmoid layer to output the discrimination score

[0085] Discriminator D p has two inputs: the source image I x keypoint heatmap P x (18, 128, 64) and the anonymized image T x (3, 128, 64). P x and T x are stacked and input into 2 convolutional layers, 3 ResNet residual blocks, and 1 sigmoid layer to output the discrimination score

[0086] Step 4: Training constraints, the specific steps are as follows:

[0087] 4-1. The complete objective function of UPT is defined as:

[0088]

[0089] where L GD is the adversarial loss between the generator and the discriminator, L s is the identity exchange loss, L r is the reconstruction loss, and λ 1 、λ 2 and λ 3 are parameters that balance the importance of different losses.

[0090] 4-2. Adversarial loss L GD .

[0091] The adversarial loss can be expressed as follows:

[0092]

[0093] D p (G(z x , A x , P x ), P x ))] (2)

[0094] where A x and P x are the attribute and pose heatmaps sampled from real data, M x is the human body region mask in the original human body image I x , is the pose heatmap of the delegated human body image s d .

[0095] 4-3. Identity exchange loss.

[0096] When the original human body image Ix Identity and proxy body images d When the identities of the source and proxy are from different bodies in the training samples, we define the identity exchange loss as follows:

[0097]

[0098] where and represent the source identity and the proxy identity respectively, and H(T x , s d ) is the relaxed identity feature constraint, defined as:

[0099] H(T x , s d ) = max{|f(s d ) - f(T x )| 1 - α, 0} (4)

[0100] where f is the pre-trained body feature extractor. During the pose transformation process, using a margin α > 0 can prevent complete identity transfer, which can improve the security of our network.

[0101] 4 - 4. Reconstruction loss.

[0102] When the source identity and the proxy identity are from the same body in the training samples, we define the reconstruction loss L r as the l 1 distance at the feature level and the pixel level:

[0103]

[0104] where the l 1 distance at the feature level U(T x , I x ) = |f(T x ) - f(I x )| 1 , and the l 1 distance at the pixel level V(T x , I x ) = |T x - I x | 1 , and β is the parameter for balancing the feature and pixel l 1 distance.

[0105] Furthermore, λ 1 , λ 2 and λ 3 are the parameters for balancing the importance of different losses. During training, we take λ 1 = 5, λ 2 = 1, λ 3= 1. During the pose transformation process, using a margin α > 0 can prevent complete identity transfer. We take α = 0.4. Balance the feature and pixel l 1 The parameter β for the distance is taken as β = 10 during training.

[0106] Step 5: Train the model and test the data. The specific steps are as follows:

[0107] 5-1. Select a suitable dataset, and then preprocess the dataset as described in Step 1, such as the Market-1501 dataset (1501 identities) or the DukeMTMC-reID dataset (1404 identities).

[0108] 5-2. Use the training constraints proposed in Step 4 as the objective function to train the UPT network in Step 3. During training, we use the Adam optimizer, set the learning rate to 0.0002, β 1 coefficient to 0.5, β 2 coefficient to 0.999, and the batch size to 32. During the first 37500 iterations of the training process For the next 50000 iterations, with a 70% probability 30% probability

[0109] 5-3. To verify the effectiveness of the proposed method, compare it with traditional methods such as blurring, pixelation, and face removal. Additionally, compare it with deep learning methods such as CIAGAN and DG-Net. To compare the effectiveness of our differential privacy identity delegation (DPA), we apply DPA to CIAGAN and DG-Net for comparison, denoted as CIAGAN-DPA and DG-Net-DPA. Also, use random identity delegation for comparison on our UPT model, CIAGAN, and DG-Net, denoted as UCS-R, CIAGAN-R, and DG-Net-R. In addition, we also compare it with the feature-level differential privacy method, denoted as UCS-FDP.

[0110] 5-4. We evaluate the privacy protection and data availability. For privacy protection, we use a pre-trained person re-identification model to calculate the ReId rate (Rank@1, mAP). For data availability, we use a pre-trained attribute classifier to evaluate the attribute preservation rate (APR), pose preservation rate (PCKh), structural similarity (SSIM), and the and dataset FID score of the dataset. In addition, to avoid randomness, we conduct 10 experiments on methods with randomness.

[0111] Experimental results

[0112] 1. Table 1 shows the comparison experiment results of this method with other methods on the Market-1501 dataset.

[0113] 2. Table 2 shows the comparison experiment results of this method with other methods on the DukeMTMC-reID dataset.

[0114] Table 1 Comparison experiment results of this method with other methods on the Market-1501 dataset.

[0115]

[0116] Table 2 Comparison experiment results of this method with other methods on the DukeMTMC-reID dataset.

[0117]

Claims

1. A differential privacy human body anonymization synthesis method with consistent usability, characterized in that The source human body dataset Through the delegated human body dataset And the availability-maintained pose conversion network to obtain an anonymous human body dataset The specific implementation steps are as follows: Step 1: Preprocessing; Step 2: Assigning a delegated identity; Step 3: Constructing a usability-preserving pose conversion network; Step 4: Training constraints; Step 5: Model training and testing; Step 1 preprocessing, the specific steps are as follows: 1-1. Delegated human body dataset Collect and use a publicly available dataset or a synthetic dataset as the delegated human body dataset 1-2. Image marking, non-sensitive attributes required for marking human body images in the delegated human body dataset in the human body image; 1-3. Use a pose parser to parse the human body images in the delegated human body dataset to obtain human body key points; in the human body images to obtain human body key points; 1-4. Use an instance segmentation model to segment the human body image to obtain a human body region mask image; Step 2 assigning a delegated identity, the specific steps are as follows: 2-1. Construct a candidate set with consistent attributes; Use the pre-trained attribute classifier to classify the original human image I x Attribute A x Make predictions; attribute consistent candidate set S x From the delegated human dataset Select attribute and A x A consistent delegation identity is constructed, the selected attributes are all non-sensitive, and are selected by maximizing the attribute consistency selection method; Use a pre-trained attribute classifier to predict the attribute A of the original image I in the source human body dataset ; Select images with attributes consistent with the non-sensitive attribute A from the delegated human body dataset x to construct a candidate set S with consistent attributes x ; Among them ; Select images with attributes consistent with the non-sensitive attribute A from the delegated human body dataset x to construct a candidate set S with consistent attributes x ; where The prediction result of the i-th non-sensitive attribute of the attribute classifier on the original image I x is which represents the softmax score of the corresponding non-sensitive attribute, and m represents the number of corresponding non-sensitive attributes; let Delegate the human body image s when ρ(x, d) = m d and the original image I x are a perfect match, where is the i-th attribute label of the delegated human body image s d ; when the number n of perfectly matched delegated human body images is less than the preset threshold t, t - n additional images will be added to the attribute-consistent candidate set S x , and the rule is to select the delegated human body image s from the delegated human body dataset that maximizes the attribute consistency d , and the attribute consistency is expressed as 2-2. Differential privacy identity delegation To find a suitable delegated human body image s d , an image-level differential random identity mapping φ is introduced. For a random mechanism K, if it satisfies, for any adjacent mappings φ 1 and φ 2 and all possible outputs O: Pr[K(φ 1 ) ∈ O] ≤ e ∈ Pr[K(φ 2 ) ∈ O] Then the random mechanism K is said to satisfy ∈-differential privacy; The exponential mechanism is used to achieve ∈-differential privacy, and its definition is as follows: When the randomized algorithm K(φ, u, R) selects and outputs an element with probability proportional to exp(∈u(φ, r) / 2Δu) then K satisfies ∈-differential privacy; where represents the availability of mapping the x-th image from the source human body dataset to the d-th image in the delegated human body dataset, k represents the index of all possible mapped images of the x-th image in the source human body dataset on the delegated human body dataset, and ξ(k, x) = |f(s k ) - f(I x )| 2 , f is a pre-trained human feature extractor, s k is the k-th image in the delegated human body dataset; Finally, perform random sampling to select identity delegation with a probability of ; since Pr[φ|x→d] is proportional to exp(∈u(φ,r) / 2Δu), the identity delegation method satisfies the exponential mechanism, that is, it satisfies ∈-differential privacy. Step 3 constructing a usability-preserving pose conversion network, the specific steps are as follows: For a given original human body image First, use the steps in Step 2 to assign the delegated human body image corresponding to the delegated identity Then, use the UPT network to replace the human body in the original human body image I x to obtain an anonymous target image The UPT network is as follows: 3-1. Construct a generator; The generator consists of several encoders and a decoder, including encoders E x , E p , E a , E f and a decoder where E x and E p are composed of 3 convolutional layers, E f is composed of 2 convolutional layers, E a is composed of 7 fully connected layers, is composed of a progressive pose transfer model; 3-2. Construct a discriminator; The UPT network employs two discriminators D p and D t for adversarial training of the generator; discriminator D p focuses on determining whether the pose of the target image corresponds to the pose of the source human body and is composed of multiple convolutional layers and residual blocks; discriminator D t uses the same 7-layer fully connected layer as encoder E a to discriminate the authenticity and attribute consistency of the generated human body images; 3-3. Background restoration module; Paste the background of the original human body image I x onto the generated human body image and then input it into the discriminator D p and D t In this way, the original background can be well restored; The described generator has six inputs: the delegated human body image s d 、I x 's key point heat map P x 、s d 's key point heat map Non-sensitive attribute A x 、background B x and the human body region mask M x ; P x and Stack the input E p Output the pose feature E p ,non-sensitive attribute A x Input E a Output the attribute feature F a ,B x and s d Stack the input E again x Output the image feature F x ,attribute feature F a and image feature F x Stack the input E f Output the fusion feature F f ,pose feature F p and fusion feature F f Stack the input Get I′ x ,the final human body region mask M x 、background B x and I′ x Input the background restoration module to get the anonymized image T x ; The discriminator D t has two inputs: an anonymous image T x and a non-sensitive attribute A x ; the discriminator D t first encodes A using seven fully connected layers x , then encodes T using two convolutional layers x , and finally stacks the encoding results of A x and T x and inputs them into two convolutional layers, three residual blocks, and one sigmoid layer to output a discrimination score Discriminator D p has two inputs: the source image I x 's key-point heatmap P x and the anonymized image T x ; P x and T x are stacked and input into 2 convolutional layers, 3 ResNet residual blocks, and 1 sigmoid layer to output a discrimination score Step 4 training constraints, the specific steps are as follows: 4-1. The complete objective function of UPT is defined as: Among them L GD is the adversarial loss between the generator and the discriminator, L s is the identity exchange loss, L r is the reconstruction loss, λ 1 、λ 2 and λ 3 are parameters for balancing the importance of different losses; 4-2. Adversarial Loss L GD ; The adversarial loss is expressed as follows: Among them A x and P x are the attribute and pose heat maps sampled from real data, M x is the original human body image I x in the human body region mask, is the pose heat map of the delegated human body image s d ; 4-3. Identity exchange loss; When the identity of the original human body image I x and the identity of the delegated human body image s d come from different humans in the training samples, we define the identity exchange loss: Among them and respectively represent the source identity corresponding to the original human body image I x and the delegated identity corresponding to the delegated human body image s d H(T x , s d ) is a relaxed identity feature constraint, defined as: H(T x , s d ) = max{|f(s d ) - f(T x )| 1 - α, 0} (4) where f is a pre-trained human body feature extractor; during the pose conversion process, using a margin α>0 can prevent complete identity transfer; 4-4. Reconstruction loss; When the source identity and the delegated identity come from the same person in the training samples, the reconstruction loss L r is defined as the l 1 distance at the feature level and the pixel level: Among them, the feature-level l 1 distance U(T x , I x ) = |f(T x ) - f(I x )| 1 , and the pixel-level l 1 distance V(T x , I x ) = |T x - I x | 1 , where β is a parameter that balances the feature and pixel l 1 distances.

Images