A cyclic reversible anonymous face synthesis method for preventing identity leakage

Abstract

The application discloses a cycle reversible anonymous face synthesis method for preventing identity leakage, and comprises the following steps: step 1, data set pretreatment; step 2, constructing an identity modification network and obtaining modified identity features through an identity modification model; step 3, constructing an attribute control network and obtaining an attribute-controllable initial anonymous image through an attribute control model; step 4, constructing a feature steganography network and obtaining steganographic features through a feature steganography model; step 5, training and testing using a public data set; and step 6, completing image anonymity and recovery using the trained model. The method has the advantages that the face region of the image to be anonymized can be replaced without infringing on the privacy of others, the generated anonymous image can be reversible to restore the original image, and the anonymous image is more visually friendly.

Description

Technical Field

[0001] This invention relates to the field of facial image privacy protection, specifically to a method for cyclic reversible anonymized face synthesis to prevent identity leakage. Background Technology

[0002] In the information age, with the rapid advancement of internet technology, computer technologies such as data processing and analysis have developed rapidly, leading to a surge in intelligent devices and applications that provide people with personalized and diversified services. For example, platforms such as Alipay, WeChat, and Facebook have become an integral part of people's daily lives.

[0003] Technology is a double-edged sword. While it brings convenience to our daily lives, it also necessitates safeguarding our data security. Although the unprecedented development of internet technology has brought convenience and intelligent work and lifestyles, it has also left people with nowhere to hide their privacy, leading to frequent data breaches. In 2019, 1.08 billion cases of telecom fraud were intercepted nationwide, primarily due to data leaks. Marriott International suffered billions of dollars in market value losses due to a series of customer data breaches. Cambridge Analytica, a British data analytics company, was fined $5 billion and ultimately went bankrupt after misappropriating 87 million Facebook profiles. These examples have had a significant negative impact on individuals, businesses, and society as a whole, making the resolution of privacy and security issues an urgent matter.

[0004] To address image privacy concerns, many countries and regions have enacted laws and regulations to protect public privacy, and some have even reclaimed certain facial datasets. While these methods offer some protection for facial image privacy, they do not fundamentally solve the public's privacy problem. On the contrary, these measures increase the difficulty for researchers. In fact, tasks like face detection and pedestrian tracking do not require the identity information in facial images. Therefore, a reliable anonymization method can not only hide the original identity information but also allow the anonymized images to be used for other computer vision tasks.

[0005] In recent years, with the rapid development of deep learning, generative networks, represented by Generative Adversarial Networks (GANs) and Variational Auto-Encoders (VAEs), have provided the technological foundation and conditions for intelligent privacy protection. By utilizing generator networks, images to be anonymized can be effectively anonymized through methods such as feature decoupling or identity control.

[0006] In summary, using generative adversarial networks to anonymize facial images is a worthwhile research direction. This patent aims to address the difficulties and key issues in current methods by focusing on several key aspects of this task.

[0007] A key aspect of image-based facial anonymization is ensuring the effectiveness of face anonymization. However, most current methods only anonymize a limited range of faces, failing to achieve diversity in anonymization, or the anonymized image cannot be reversibly restored. While they achieve anonymization, they cannot provide reversible anonymity. Specifically, there are two main key points:

[0008] (1) The effectiveness of anonymized images. Current methods mainly achieve anonymity by face swapping or using someone else's identity to merge with one's own data. However, face swapping or identity replacement will still reveal the target face or identity information. Therefore, how to anonymize one's own facial image without revealing the identity information of others is an urgent problem to be solved.

[0009] (2) Reversibility of anonymized images. Recovering the original image from an anonymized image is currently a difficult problem. Summary of the Invention

[0010] To address the shortcomings of existing technologies, this invention proposes a reversible anonymized face synthesis method to prevent identity leakage. By introducing an identity modification network and a feature steganography network, the identity features of the image are modified and steganographically represented, thereby achieving reversible anonymity of the image.

[0011] This invention provides a cyclic reversible anonymous face synthesis method for preventing identity leakage, comprising the following steps:

[0012] Step 1: Dataset preprocessing;

[0013] Step 2: Construct an identity modification network and obtain modified identity features through the identity modification model.

[0014] First, the preprocessed image is input into the real identity feature extractor to obtain the real identity features of the image; then, Gaussian noise is added to the obtained identity features and input into the identity modification generator to obtain the modified identity features; finally, the generated modified identity features are input into the identity modification discriminator to distinguish between real and fake.

[0015] Step 3: Construct an attribute-controlled network and obtain an initial anonymous graph with controllable attributes through the attribute-controlled model.

[0016] First, a target image is randomly input into the style extractor, identity feature extractor, and expression extractor to obtain the corresponding style features, identity features, and expression features. Then, the preprocessed image and the obtained style features, identity features, and expression features are input into the attribute control generator to obtain the generated image. At the same time, the generated image is input into the attribute control discriminator to determine the authenticity of the initial anonymous image. The above method is repeated to finally obtain a visually realistic initial anonymous image.

[0017] Step 4: Construct a feature steganalysis network and obtain the steganalytic features through the feature steganalysis model.

[0018] First, the initial anonymized image and real features are input into the steganalysis network to obtain the stegated image. Then, the stegated image and Gaussian noise are input into the steganalysis network to obtain the stegated features. Through the training process described above, a visually realistic anonymized image is finally obtained.

[0019] Step 5: Train and test using a public dataset;

[0020] Step 6: Use the trained model to complete image anonymization and restoration.

[0021] Preferably, the dataset preprocessing in step 1 includes the following steps:

[0022] 1-1. Use an existing pre-trained face segmentation network to process the original image dataset and generate a mask for face segmentation in each image of the original image dataset.

[0023] 1-2. Use existing pre-trained face recognition models to extract the identity features of each image in the original image dataset, and use the k-means clustering algorithm to cluster these identity features to obtain the identity features of the cluster center of each image.

[0024] Preferably, in the identity modification network, the real identity feature extractor adopts an existing pre-trained face recognition network. This network structure includes convolutional layers, normalization layers, and four ResBlock residual blocks, which can extract the identity features of the input image. The identity modification generator consists of five fully connected layers, with ReLU layers between the fully connected layers for activation, used to modify the identity features. The identity modification discriminator consists of four blocks, each block having a fully connected layer, a ReLU activation layer, and Dropout. The last block also has a Sigmoid layer for outputting the probability value of the result being true or false.

[0025] Preferably, the objective function of the identity modification network includes the objective function of GAN, the distance preservation objective function, and the uniqueness objective function.

[0026] The objective function of GAN uses the GAN concept to control feature generation, ensuring that the generated features have the same data distribution as the real features. This generates modified identity features to distinguish between real and fake features. Mathematically, this can be expressed as:

[0027]

[0028] Among them, z id G represents the real identity features output by the real identity feature extractor, where G is the generator and D is the discriminator.

[0029] The distance-preserving objective function employs the idea of ​​k-anonymity, using a loss function to ensure that the generated image after modifying identity features is relatively close to its k-means cluster centers within a certain range. Mathematically, this can be expressed as:

[0030]

[0031]

[0032] Among them, z center For z new The cluster center of the cluster it belongs to. For z new and z center The cosine similarity is given by α, which is a hyperparameter and is half the maximum value of the cosine similarity among all cluster centers in the current dataset.

[0033] The uniqueness objective function, which uses this loss function to generate identity features that do not exist in the real world, can be mathematically expressed as:

[0034]

[0035] in, Let D represent the j-th identity feature in the current cluster set. FS represents all identity features of a given cluster, i.e., the set of identity features for that cluster. num This represents the number of features in FS.

[0036] Preferably, in the attribute control network, the style extractor consists of four ResBlock residual blocks, each of which includes two ConvLayer convolutional layers and one skip layer, which can ultimately extract the style of the input image.

[0037] The identity feature extractor uses an existing pre-trained face recognition network. The network structure includes a convolutional layer, a normalization layer, and four ResBlock residual blocks, which can extract the identity features of the input image.

[0038] The facial expression extractor uses an existing pre-trained facial expression extraction network, specifically the MobileFaceNet structure, which extracts the facial expression features of the input image.

[0039] The attribute control generator consists of nine symmetrical residual blocks, each with a convolutional layer, InstanceNorm2d normalization, and LeakyReLU activation function. The last five residual blocks each receive three 512-dimensional feature information, which is injected using AdaIN, and its expression is:

[0040]

[0041] Where, θ i Represents the image, μ(θ) i ) represents the mean of the image, σ(θ) i ) represents the standard deviation of the image, μ(s) i σ(s) represents the mean of the feature information input into the generator. i ) represents the standard deviation of feature information;

[0042] The attribute-controlled discriminator consists of multiple ConvLayer convolutional layers and five ResBlock residual blocks, and finally outputs through two fully connected layers to determine the authenticity of the input image.

[0043] The mask generator consists of multiple convolutional layers, normalization layers, and activation layers. In addition, Gaussian convolution kernels are added to the image to smooth the mask boundaries, thereby giving the generated mask a soft boundary.

[0044] Preferably, the objective function of the attribute control network includes the objective function of Generative Adversarial Network (GAN), the objective function of identity features, the objective function of style features, the objective function of facial expression features, and the objective function of masking.

[0045] The objective function of a Generative Adversarial Network (GAN) uses the GAN concept to control the generation of anonymized images. Different initial anonymized images are obtained by inputting different images and features. Specifically, the preprocessed image, its style features, the generated image, and facial expression features are input into an attribute-controlled generator to obtain the initial anonymized image for the corresponding target. Then, the stitched image of the initial anonymized image and the original background is input into an attribute-controlled discriminator to determine whether it is real or fake. Mathematically, this can be represented as:

[0046]

[0047] X fuse =G(X,z) id ,z style ,z exp )⊙(1-M)+X⊙M

[0048] Where X represents the image, z id z represents the output of the identity extractor. style This represents the output of the style extractor, z exp The output of the facial expression extractor is represented by M, the output of the mask generator is represented by G, and the discriminator is represented by D.

[0049] The identity feature objective function, to ensure that the generated image and the identity features input to the generator have the same identity, calculates the cosine similarity between their identity features, which can be expressed mathematically as:

[0050]

[0051] Among them, z id z represents the identity features input into the generator. id′ This indicates the identity features obtained by inputting the generated graph into the identity extractor;

[0052] The style feature objective function, to ensure that the generated image and the style features input to the generator have the same style, calculates the distance between their style features, which can be expressed mathematically as follows:

[0053]

[0054] Among them, z style z represents the style features input into the generator. style′ This represents the style features obtained by inputting the generated image into the style extractor;

[0055] The objective function for facial expression features is as follows: To ensure that the generated image and the facial expression features input to the generator have the same expression, we calculate the distance between their facial expression features, which can be expressed mathematically as follows:

[0056]

[0057] Among them, z exp z represents the facial features input into the generator. exp′ This indicates the facial features obtained by inputting the generated image into the facial expression extractor;

[0058] The masking objective function employs a pre-trained face segmentation network to obtain image masks during data preprocessing. To achieve smoother mask boundaries, a [missing information - likely a specific feature or function] is proposed. To guide the mask generator, mathematically it can be represented as:

[0059]

[0060] Where maskG represents the mask generator, and M′ represents the mask obtained using a pre-trained face segmentation network.

[0061] Preferably, the feature steganalysis network is implemented using a reversible neural network structure. By stegatizing the features, the feature steganalysis network can steg the features during forward transmission and extract the features during reverse transmission.

[0062] Preferably, the objective function of the feature steganalysis network includes a forward consistency objective function, a reverse consistency objective function, and a low-frequency wavelet objective function.

[0063] The objective function for positive consistency is to steg the features into the image during the forward hiding stage, resulting in a stegated image that is consistent with the original image. Therefore, we designed a hiding loss, which can be mathematically expressed as:

[0064]

[0065] Among them, X cover X represents the image before steganography. stego This represents the steganographic image;

[0066] The reverse consistency objective function, in the reverse display stage, aims to generate the original features based on the steganalyzed image and Gaussian noise. To achieve this objective, we designed a display loss, which can be mathematically expressed as:

[0067]

[0068] Where feature is hidden in X cover In the context of X, feature′ represents the characteristic from X. stego The features shown in the image.

[0069] The low-frequency wavelet objective function ensures that most of the information is hidden in the high-frequency components, resulting in the stegatized image X. stego The low-frequency components need to be compared with the image X before steganalysis. cover The low-frequency components are similar, and can be expressed mathematically as:

[0070]

[0071] in, This indicates the operation of extracting low-frequency components after wavelet decomposition.

[0072] Preferably, step 5 uses a public dataset for training and testing, and the specific steps are as follows:

[0073] 5-1 Prepare the dataset. Use the VGGFACE2 and CelebaHQ public datasets and process them according to the preprocessing procedure described in step 1 to obtain the face mask and cluster identity features of each image.

[0074] 5-2 Input the image and mask into the attribute control network for training, and test it using test data;

[0075] 5-3 Input the identity features of the image, mask, and cluster center into the identity modification network for training, and test it using test data;

[0076] 5-4 Input the image and features into the feature steganalysis network for forward and backward training, and test it using test data.

[0077] Preferably, step 6 uses the model trained in step 5 to complete image anonymization and restoration, and the specific steps are as follows:

[0078] 6-1 Perform the preprocessing of the image to be anonymized in step 1 to obtain its mask and the identity features of the cluster centers;

[0079] 6-2 Input it into the identity modification network to obtain the modified identity features, and then input its image and the obtained modified identity features into the attribute control network to obtain the initial anonymity image;

[0080] 6-3 Input the real identity features of the image and the initial anonymous image obtained in the previous stage into the feature steganalysis network to obtain the final anonymous image after steganography.

[0081] The benefits of this invention are:

[0082] This method modifies the identity of faces in images to achieve facial anonymity. The modified identity is maintained uniquely through an objective function, thus avoiding privacy violations. Furthermore, this method achieves reversible facial anonymity, making it more effective and visually appealing than previous methods. Experimental results clearly demonstrate the efficiency and practicality of the proposed method. In conclusion, the proposed method offers more efficient and aesthetically pleasing privacy protection for portrait images. Attached Figure Description

[0083] Figure 1 This is a flowchart of the implementation steps of the method of the present invention;

[0084] Figure 2 This is a diagram of the overall network architecture for implementing the method of this invention. Detailed Implementation

[0085] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0086] Step 1: The specific steps are as follows:

[0087] 1-1 We use an existing pre-trained face segmentation network to process the image dataset and generate a mask for face segmentation in each image.

[0088] 1-2 uses an existing pre-trained face recognition network to extract the identity features of each image, and uses the k-means clustering algorithm to cluster these identity features to obtain the identity features of the cluster center of each image.

[0089] Step 2 is detailed below:

[0090] like Figure 2 As shown in the lower left corner, the identity modification network includes a generator, an identity feature extractor, and a discriminator. First, the image is input into the identity feature extractor to obtain the image's identity features. Then, Gaussian noise is added to the obtained identity features and input into the generator to obtain modified identity features. Finally, the modified identity features are input into the discriminator to determine whether the image is genuine or fake.

[0091] 2-1 Constructing the Generator. The generator consists of five fully connected layers, with ReLU layers between each fully connected layer for activation, thereby modifying identity features.

[0092] 2-2 Constructing the Discriminator. The discriminator consists of four blocks, each with a fully connected layer, a ReLU activation layer, and a Dropout layer. The last block also has a Sigmoid layer to output the probability values ​​of the result as true or false.

[0093] 2-3 Construct an identity feature extractor. This extractor uses an existing pre-trained face recognition network, whose structure includes convolutional layers, normalization layers, and four ResBlock residual blocks. It can extract the identity features of the input image.

[0094] Step 3 is detailed below:

[0095] The objective functions of the identity modification network include the objective function of GAN, the distance preservation objective function, and the uniqueness objective function.

[0096] 3-1 GAN Objective Function. The GAN approach is used to control feature generation, ensuring that the generated features have the same data distribution as the real features. Specifically, the image is first input into an identity feature extractor to obtain identity features, then these features are input into a generator to obtain modified identity features, and finally, the modified features are input into a discriminator to determine whether the image is real or fake. Mathematically, this can be represented as:

[0097]

[0098] Among them, z id G is the output of the identity feature extractor, G is the generator, and D is the discriminator.

[0099] 3-2 Distance Preservation Objective Function. Since relying solely on network anonymity cannot guarantee true anonymity, we employ the concept of k-anonymity. This involves using a loss function to ensure that the modified identity features are relatively close to their k-means cluster centers within a certain range. Mathematically, this can be expressed as:

[0100]

[0101]

[0102] Among them, z center For z new The cluster center of the cluster it belongs to. For z new and z center The cosine similarity is given by α, which is a hyperparameter and is half the maximum value of the cosine similarity among all cluster centers in the current dataset.

[0103] 3-3 Uniqueness Objective Function. For identity modification networks, we aim to generate new identities that are distinct from any identity in the original dataset. Without additional constraints, during the random generation of identity features, the generated features might resemble existing identity features in the dataset. Therefore, we innovatively design a uniqueness loss to avoid this possibility. Theoretically, if the database is large enough, the uniqueness loss can be used to generate identity features that do not exist in the real world. Mathematically, this can be expressed as:

[0104]

[0105] in, Let D represent the j-th identity feature in the current cluster set. FS(the set of identity features for a given cluster) represents all identity features in a given cluster. num This represents the number of features in FS.

[0106] Step 4 is detailed below:

[0107] like Figure 2 As shown on the left, the attribute control network includes a generator, a style extractor, an identity feature extractor, an expression feature extractor, a discriminator, and a mask generator. First, the image is input into the style extractor, identity feature extractor, and expression feature extractor respectively to obtain the corresponding style features, identity features, and expression features. Then, the image along with the obtained style features, identity features, and expression features is input into the generator to obtain a generated image. Simultaneously, the generated image is input into the discriminator to determine its authenticity. Through training using the above process, a visually realistic initial anonymized image is finally obtained.

[0108] 4-1 Constructing the Generator. The generator consists of nine symmetric residual blocks. Each residual block has a convolutional layer, InstanceNorm2d normalization, and LeakyReLU activation function. The last five residual blocks each receive three 512-dimensional feature messages, which are injected using AdaIN, with the expression:

[0109]

[0110] Where, θ i Represents the image, μ(θ) i ) represents the mean of the image, σ(θ) i ) represents the standard deviation of the image, s i Representing feature information, μ(s) i σ(s) represents the mean of the feature information input into the generator. i ) represents the standard deviation of feature information.

[0111] 4-2 Constructing a Style Extractor. The style extractor consists of four ResBlock residual blocks, each containing two ConvLayer convolutional layers and one skip layer. It can extract the style of the input image.

[0112] 4-3 Constructing an Identity Feature Extractor. This extractor uses an existing pre-trained face recognition network, whose structure includes convolutional layers, normalization layers, and four ResBlock residual blocks. It can extract the identity features of the input image.

[0113] 4-4 Constructing an Facial Expression Feature Extractor. This extractor uses an existing pre-trained facial expression extraction network, primarily the MobileFaceNet structure, which can extract facial expression features from an input image.

[0114] 4-5 Constructing the discriminator. The discriminator consists of multiple ConvLayer convolutional layers and five ResBlock residual blocks, and finally outputs through two fully connected layers to determine whether the input image is real or fake.

[0115] 4-6 Mask Generator. The mask generator consists of multiple convolutional layers, normalization layers, and activation layers. In addition, Gaussian convolution kernels are added to the image to smooth the mask boundaries, thus giving the generated mask a soft edge.

[0116] Step 5 is detailed below:

[0117] The objective functions of the attribute control network include the objective function of Generative Adversarial Network (GAN), the objective function of identity features, the objective function of style features, the objective function of facial expression features, and the objective function of masking.

[0118] 5-1 GAN's objective function:

[0119] This paper employs the concept of Generative Adversarial Networks (GANs) to control the generation of anonymized images. Different anonymized images are obtained by inputting different images and features. Specifically, the image, its style features, identity features, and facial expression features are input into the generator to obtain the corresponding generated image. Then, the spliced ​​image of the generated image and the original image's background is input into the discriminator to determine whether it is real or fake. Mathematically, this can be represented as:

[0120]

[0121] X fuse =G(X,z) id ,z style ,z exp )⊙(1-M)+X⊙M

[0122] Where X represents the image, z id z represents the output of the identity extractor. style This represents the output of the style extractor, z exp The output of the facial feature extractor is represented by , M is the output of the mask generator, G represents the generator, and D represents the discriminator.

[0123] 5-2 Identity Feature Objective Function

[0124] To ensure that the generated image shares the same identity features as the input identity features to the generator, we calculated the cosine similarity between their identity features, which can be mathematically expressed as:

[0125]

[0126] Among them, z id z represents the identity features input into the generator.id′ This indicates the identity features obtained by inputting the generated graph into the identity extractor.

[0127] 5-3 Style Feature Objective Function

[0128] To ensure that the generated image shares the same style as the style features input to the generator, we calculated the distance between their style features, which can be mathematically represented as:

[0129]

[0130] Among them, z style z represents the style features input into the generator. style′ This indicates the style features obtained by inputting the generated image into the style extractor.

[0131] 5-4 Objective Function for Facial Expression Features

[0132] To ensure that the generated image and the facial expression features input to the generator have the same expression, we calculated the distance between their facial expression features, which can be expressed mathematically as:

[0133]

[0134] Among them, z exp z represents the facial features input into the generator. exp′ This indicates the facial features obtained by inputting the generated image into the facial feature extractor.

[0135] 5-5 Masking Objective Function

[0136] During data preprocessing, we use a pre-trained face segmentation network to obtain image masks. However, we need the masks to focus on more detailed areas of the image, making the mask boundaries smoother. Therefore, we propose... To guide the mask generator, mathematically it can be represented as:

[0137]

[0138] Where maskG represents the mask generator, and M′ represents the mask obtained using a pre-trained face segmentation network.

[0139] Step 6 is detailed below:

[0140] like Figure 2 As shown on the right, the feature steganalysis network consists only of a steganalysis network. First, the image and the features to be steganalyzed are input into the steganalysis network to obtain the stegated image. Then, the steganalysis image and Gaussian noise are input into the steganalysis network to obtain the stegated features. Through training using the above process, a visually realistic generated image is finally obtained.

[0141] 6-1 Constructing a Stegating Network. The stegating network is implemented using a reversible neural network structure. When using a reversible neural network for steganography, it often steganographically inserts one image into another. However, this may cause significant loss to the original image. Therefore, I adopted feature steganography, which allows the stegating network to insert features during forward propagation and extract features during reverse propagation.

[0142] Step 7 is detailed below:

[0143] The objective functions of the feature steganalysis network include a forward consistency objective function, a reverse consistency objective function, and a low-frequency wavelet objective function.

[0144] 7-1 Forward Consistency Objective Function. The goal of the forward hiding stage is to steg the features into the image to obtain the stegated image. The stegated image must be consistent with the original image. Therefore, we designed a hiding loss, which can be mathematically expressed as:

[0145]

[0146] Among them, X cover X represents the image before steganography. stego This represents the steganographic image.

[0147] 7-2 Reverse Consistency Objective Function. In the reverse display stage, the original features are generated based on the steglated image and Gaussian noise. To achieve this objective, we designed a display loss, which can be mathematically expressed as:

[0148]

[0149] Where feature is hidden in X cover In the context of X, feature′ represents the characteristic from X. stego The features shown in the image.

[0150] 7-3 Low-frequency wavelet objective function. Since we are staging highly sensitive features into the original image, to prevent the recognition network from directly detecting these features, we incorporate a low-frequency wavelet loss. To ensure that most information is hidden in the high-frequency components, the staging image X... stego The low-frequency components need to be compared with the image X before steganalysis. cover The low-frequency components are similar, and can be expressed mathematically as:

[0151]

[0152] in, This indicates the operation of extracting low-frequency components after wavelet decomposition.

[0153] Step 8 is detailed below:

[0154] 8-1 Prepare the dataset, for example, using public datasets such as VGGFACE2 and CelebAHq and processing them according to the preprocessing procedure described in step 1, to obtain the face mask and cluster identity features for each image.

[0155] 8-2 Input the image and mask into the attribute control network for training, and test it using test data.

[0156] 8-3 The identity features of the image, mask, and cluster center are input into the identity modification network for training, and the network is tested using test data.

[0157] 8-4 Input the image and features into the feature steganalysis network for forward and backward training, and test it using test data.

[0158] Step 9 is detailed below:

[0159] 9-1 The image that needs to be anonymized is preprocessed in step 1 to obtain its mask and the identity features of the cluster center.

[0160] 9-2 Input it into the identity modification network to obtain the modified identity features, and then input its image and the obtained modified identity features into the attribute control network to obtain the anonymity graph.

[0161] 9-3 Input the unmodified identity features of the image and the anonymous image obtained in the previous stage into the feature steganalysis network to obtain the steganalyzed image.

[0162] This embodiment applies the above-described technical solution to obtain...

[0163] Experimental results:

[0164] 1. The recognition rates of our method were compared with those of methods dealing with occlusion, blurring, pixelation, random noise, NEO, and CIAGAN. Detailed results are shown in Table 1.

[0165] Table 1. Experimental results of the de-identification rate of this method compared with other methods.

[0166] method Obstruction Vague Pixelation Random noise NEO CIAGAN This method Recognition rate 1.000 0.6878 0.9130 0.2288 0.8269 0.9556 0.9990

[0167] A higher de-identification rate indicates better anonymization results. As shown in the table above, compared with other generative anonymization models, this method has the highest de-identification rate and better anonymization effect.

[0168] 2. The identity swapping rates of our method and other methods including occlusion, blurring, pixelation, random noise, NEO, and CIAGAN were tested. Detailed data results are shown in Table 2.

[0169] Table 2 shows the experimental results of identity exchange rate between this method and other methods.

[0170] method Obstruction Vague Pixelation Random noise NEO CIAGAN This method Identity exchange rate 0.0000 0.3466 0.1272 0.9869 0.2394 0.1561 0.0842

[0171] The identity exchange rate metric indicates the degree to which anonymized graphs reveal other identities. As shown in the table above, compared with other generative anonymization models, this method has the lowest identity exchange rate and the degree of leakage to other identities is smaller.

[0172] 3. The face recognition efficiency of our method was tested against those of methods with occlusion, blurring, pixelation, random noise, NEO, and CIAGAN. Detailed results are shown in Table 3.

[0173] Table 3 shows the experimental results of the effectiveness of this method in facial recognition compared to other methods.

[0174]

[0175] The face recognition efficiency is used to evaluate the usability of the generated image for other computer vision tasks. As shown in the table above, the face recognition efficiency of this method is the highest, and it can still be used for other computer vision tasks.

[0176] 4. The FID image quality metrics of our method and other methods such as occlusion, blurring, pixelation, random noise, NEO, and CIAGAN were tested. Detailed data results are shown in Table 4.

[0177] Table 4. Experimental results of image quality using this method compared to other methods.

[0178] method Obstruction Vague Pixelation Random noise NEO CIAGAN This method Image quality 2389.97 431.91 515.67 732.11 76.34 67.47 21.60

[0179] FID image quality is used to evaluate the quality of the image generated by the model. The lower the FID value, the better the image quality. As shown in the table above, the image generated by this method has a lower FID and better image quality.

Claims

1. A method for preventing identity leakage through cyclic reversible anonymized face synthesis, characterized in that, Includes the following steps: Step 1: Dataset preprocessing; Step 2: Construct an identity modification network and obtain the modified identity features through the identity modification network. The specific steps are as follows: First, the preprocessed image is input into the real identity feature extractor to obtain the real identity features of the image; Then, Gaussian noise is added to the obtained identity features and input into the identity modification generator to obtain the modified identity features. Finally, the generated modified identity features are input into the identity modification discriminator to distinguish between true and false identities. Step 3: Construct an attribute control network and obtain an initial anonymous graph with controllable attributes through the attribute control network. The specific steps are as follows: First, a target image is randomly input into the style extractor, identity feature extractor, and expression extractor respectively to obtain the corresponding style features, identity features, and expression features; Then, the preprocessed image, along with the corresponding style features, identity features, and facial expression features, is input into the attribute control generator to generate a generated image. Simultaneously, the generated image is input into the attribute control discriminator to determine the authenticity of the initial anonymous image. The above steps of generating an image and determining authenticity are repeated to finally obtain a visually realistic initial anonymous image. Step 4: Construct a feature steganalysis network and obtain the steganalytic features through the feature steganalysis network. The specific steps are as follows: First, the initial anonymous image and the real identity features are input into the feature steganalysis network to obtain the steganalysis image. Then, the steganalysis image and Gaussian noise are input into the feature steganalysis network to obtain the steganalysis features. After training the feature steganalysis network, a visually realistic anonymous image is finally obtained. Step 5: Train and test using a public dataset; Step 6: Use the trained model to complete image anonymization and restoration.

2. The cyclic reversible anonymized face synthesis method for preventing identity leakage according to claim 1, characterized in that, The dataset preprocessing in step 1 involves the following steps: 1-1. A pre-trained face segmentation network is used to process the original image dataset and generate a mask for face segmentation in each image of the original image dataset. 1-2. A pre-trained face recognition model is used to extract the identity features of each image in the original image dataset, and the k-means clustering algorithm is used to cluster these identity features to obtain the identity features of the cluster center of each image.

3. The cyclic reversible anonymized face synthesis method for preventing identity leakage according to claim 1, characterized in that, In the identity modification network, the real identity feature extractor uses a pre-trained face recognition network. This network structure includes convolutional layers, normalization layers, and four ResBlock residual blocks, which can extract the identity features of the input image. The identity modification generator consists of five fully connected layers, with ReLU layers between each fully connected layer for activation, used to modify the identity features. The identity modification discriminator consists of four blocks, each block having a fully connected layer, a ReLU activation layer, and Dropout. The last block also has a Sigmoid layer to output the probability value of the result being true or false.

4. The cyclic reversible anonymized face synthesis method for preventing identity leakage according to claim 3, characterized in that, The objective functions of the identity modification network include the objective function of GAN, the distance preservation objective function, and the uniqueness objective function. The objective function of GAN uses the GAN concept to control feature generation, ensuring that the generated features have the same data distribution as the real features. It generates modified identity features to distinguish between real and fake features. Mathematically, this is expressed as: in The real identity features output by the real identity feature extractor For generator, For discriminators; The distance-preserving objective function employs the idea of ​​k-anonymity, using a loss function to ensure that the generated image after modifying identity features is relatively close to its k-means cluster centers within a certain range. Mathematically, this can be expressed as: in for The cluster center of the cluster it belongs to. for and cosine similarity, is a hyperparameter, which is half the maximum value of the cosine similarity among all cluster centers in the current dataset; The uniqueness objective function, which uses this loss function to generate identity features that do not exist in the real world, is mathematically expressed as: in Indicates the th element in the current cluster set. There are 10 identity features, and all identity features of a given cluster are 100. That is, the set of identity features of a certain cluster. express Number of features.

5. The cyclic reversible anonymized face synthesis method for preventing identity leakage according to claim 1, characterized in that, In the attribute control network The style extractor consists of four ResBlock residual blocks, each of which includes two ConvLayer convolutional layers and one skip layer. It can extract the style of the input image. The identity feature extractor uses a pre-trained face recognition network. The network structure includes convolutional layers, normalization layers, and four ResBlock residual blocks, which can extract the identity features of the input image. The facial expression extractor uses a pre-trained facial expression extraction network, specifically the MobileFaceNet structure, which extracts the facial expression features of the input image. The attribute control generator consists of nine symmetrical residual blocks, each with a convolutional layer, InstanceNorm2d normalization, and LeakyReLU activation function. The last five residual blocks each receive three 512-dimensional feature information, which is injected using AdaIN, and its expression is: in, Represents an image. Represents the mean of the image. The standard deviation of the image is represented. This represents the mean of the feature information input into the generator. The standard deviation representing feature information; The attribute-controlled discriminator consists of multiple ConvLayer convolutional layers and five ResBlock residual blocks, and finally outputs through two fully connected layers to determine the authenticity of the input image. The attribute control network also includes a mask generator, which generates a mask for the face region. The mask generator consists of multiple convolutional layers, normalization layers, and activation layers. In addition, Gaussian convolution kernels are added to the image to smooth the mask boundaries, so that the generated mask has soft boundaries.

6. The cyclic reversible anonymized face synthesis method for preventing identity leakage according to claim 5, characterized in that, The objective functions of the attribute control network include the objective function of Generative Adversarial Network (GAN), the objective function of identity features, the objective function of style features, the objective function of facial expression features, and the objective function of masking. The objective function of Generative Adversarial Networks (GANs) uses the GAN concept to control the generation of anonymized images. Different initial anonymized images are obtained by inputting different images and features. Mathematically, this can be expressed as: in, Represents an image. This represents the output of the identity extractor. This represents the output of the style extractor. This represents the output of the facial expression extractor. It is the output of the mask generator. Represents a generator. Indicates the discriminator; The identity feature objective function, in order to ensure that the generated image and the identity features input to the generator have the same identity, calculates the cosine similarity between their identity features, which can be expressed mathematically as: in This represents the identity features input into the generator. This indicates the identity features obtained by inputting the generated graph into the identity extractor; The style feature objective function, in order to ensure that the generated image has the same style as the style features input to the generator, calculates the distance between their style features, which can be expressed mathematically as follows: in This represents the style features input into the generator. This represents the style features obtained by inputting the generated image into the style extractor; The objective function for facial expression features, in order to ensure that the generated image and the facial expression features input to the generator have the same expression, calculates the distance between their facial expression features, which can be expressed mathematically as follows: in This represents the facial features input into the generator. This indicates the facial features obtained by inputting the generated image into the facial expression extractor; The masking objective function involves using a pre-trained face segmentation network to obtain image masks during data preprocessing. To achieve smoother mask boundaries, a [missing information - likely a specific mechanism or method] is proposed. To guide the mask generator, mathematically represented as: in Indicates a mask generator. This represents the mask obtained using a pre-trained face segmentation network.

7. The cyclic reversible anonymized face synthesis method for preventing identity leakage according to claim 1, characterized in that, The feature steganalysis network is implemented using a reversible neural network structure. By stegatizing features, the feature steganalysis network can steg the features during forward transmission and extract the features during reverse transmission.

8. The cyclic reversible anonymized face synthesis method for preventing identity leakage according to claim 7, characterized in that, The objective functions of the feature steganalysis network include a forward consistency objective function, a backward consistency objective function, and a low-frequency wavelet objective function. The objective function for positive consistency is to hide the features in the image during the forward hiding stage, resulting in a stegated image. This stegated image must be consistent with the original image. Therefore, a hiding loss is designed, mathematically expressed as: in This represents the image before steganography. This represents the steganographic image; The reverse consistency objective function, in the reverse display stage, aims to generate the original features based on the steganalyzed image and Gaussian noise. To achieve this objective, a display loss is designed, mathematically expressed as: in To hide Features in Indicates from Features shown in the image; The low-frequency wavelet objective function ensures that most of the information is hidden in the high-frequency components, resulting in a steglated image. The low-frequency components need to be compared with the image before steganalysis. The low-frequency components are similar, which can be expressed mathematically as: in This indicates the operation of extracting low-frequency components after wavelet decomposition.

9. The cyclic reversible anonymized face synthesis method for preventing identity leakage according to claim 2, characterized in that, Step 5 uses a public dataset for training and testing. The specific steps are as follows: 5-1 Prepare the dataset by using the VGGFACE2 and CelebaHQ public datasets and processing them according to the preprocessing procedure described in step 1 to obtain the mask and cluster identity features of each image. 5-2 Input the image and mask into the attribute control network for training, and test it using test data; 5-3 Input the identity features of the image, mask, and cluster center into the identity modification network for training, and test it using test data; 5-4 Input the image and features into the feature steganalysis network for forward and backward training, and test it using test data.

10. The cyclic reversible anonymized face synthesis method for preventing identity leakage according to claim 9, characterized in that, Step 6 uses the model trained in step 5 to complete image anonymization and restoration. The specific steps are as follows: 6-1 Perform the preprocessing of the image to be anonymized in step 1 to obtain its mask and the identity features of the cluster centers; 6-2 Input it into the identity modification network to obtain the modified identity features, and then input its image and the obtained modified identity features into the attribute control network to obtain the initial anonymity image; 6-3 Input the real identity features of the image and the initial anonymous image obtained in the previous stage into the feature steganalysis network to obtain the final anonymous image after steganography.

Images