A method for protecting sensitive face data based on elastic anti-interference privacy protection mechanism

Abstract

The present invention discloses a method for protecting sensitive facial data based on a flexible anti-interference privacy protection mechanism, which belongs to the field of information processing. In the data preprocessing stage, a dynamic adaptive sampling strategy based on information entropy is proposed, which adjusts the sampling rate according to the distribution characteristics of the facial data set and the weight of sensitive information, and maximizes the model performance under the premise of ensuring user privacy. In order to achieve better privacy protection effects, an entropy-adjusted differential privacy mechanism is designed. After dynamically and adaptively sampling the facial data set, different degrees of Gaussian noise are adaptively added according to the information entropy value of the facial data, thereby achieving refined privacy protection of sensitive facial data under differential privacy constraints. In the model training stage, a flexible anti-interference training method is proposed, which dynamically generates diverse adversarial samples at different stages of model training, while enhancing the robustness of the model to various types of adversarial attacks and reducing the overfitting risk of the model.

Description

Technical Field

[0001] The present invention belongs to the field of information processing, and specifically relates to a method for protecting sensitive facial data based on a flexible anti-interference privacy protection mechanism. Background Art

[0002] Compared to other types of data, sensitive facial data contains a wealth of personal information, such as identity, age, and gender. It can even reveal a person's emotions and health through facial expressions and subtle features. However, with the widespread application of facial recognition technology, the risk of personal privacy leaks is increasing. Facial data, as a highly sensitive biometric feature, poses a serious threat to personal privacy, security, and interests if leaked or misused.

[0003] Some existing research uses data obfuscation techniques to protect sensitive facial data. This technique reduces image resolution, adds random noise, or applies filters to render facial features difficult to discern, thereby achieving privacy protection. While this method is simple and easy to implement, the obfuscated data still faces the risk of being reconstructed by advanced reconstruction algorithms, potentially rendering privacy protection ineffective.

[0004] In addition, some research has used generative adversarial networks (GANs) to generate images that resemble the original image but lack real-identity information, further enhancing user privacy. This approach can effectively mask a user's real-identity information, but the model is difficult to train, and the generated images can still be restored or identified under certain attacks.

[0005] In summary, while existing methods have made some progress in protecting sensitive facial data, several challenges remain. First, in terms of privacy protection, existing methods are vulnerable to attacks such as reverse engineering. This is particularly true given the continued advancement of image reconstruction algorithms and the increasing risk of side-channel attacks, which continue to threaten data privacy. Second, most current technologies focus on hiding or encrypting the data itself. However, when attackers generate adversarial examples to mislead models for data theft or model destruction, existing methods often struggle to effectively defend against them. Summary of the Invention

[0006] To address the aforementioned issues, this paper proposes a method for protecting sensitive facial data based on a flexible anti-interference privacy protection mechanism. This method adaptively samples and dynamically adds noise to facial image data by evaluating its entropy. This method then uses adversarial samples generated by a composite adversarial attack to train models, thereby achieving privacy protection for sensitive facial data in face recognition. Specifically, during the data preprocessing phase, a dynamic, adaptive sampling strategy based on entropy is proposed. The sampling rate is adjusted based on the distribution characteristics of the facial dataset and the weight of sensitive information, maximizing model performance while preserving privacy. To achieve even better privacy protection, an entropy-modulated differential privacy mechanism is designed. After dynamically and adaptively sampling the facial dataset, varying degrees of Gaussian noise are adaptively added based on the entropy of the facial data, thereby achieving refined privacy protection for sensitive facial data under differential privacy constraints. During the model training phase, a flexible anti-interference training method is proposed. Diverse adversarial samples are dynamically generated at different stages of model training, enhancing the model's robustness against various types of adversarial attacks while reducing the risk of overfitting. Furthermore, to enhance the network's feature extraction capabilities and improve the model's generalization performance, this paper proposes a dynamic entropy-aware privacy-preserving adversarial model. By introducing dynamic convolution kernels into the ResNet-18 network, the convolution operation can be adaptively adjusted based on the characteristics of the input data, thereby achieving more flexible and efficient feature learning. Finally, to demonstrate the effectiveness of this paper, experiments were conducted on a gender classification task based on facial data.

[0007] The technical solutions of the present invention are as follows:

[0008] A method for protecting sensitive face data based on a flexible anti-interference privacy protection mechanism includes the following steps:

[0009] Step 1: Obtain the original face image dataset and preprocess it. Divide the preprocessed dataset into a training set and a test set in proportion. Calculate the information entropy of each image. Dynamically and adaptively sample the dataset based on the information entropy to obtain an updated training set.

[0010] Step 2: Allocate a privacy budget for facial image data and use the entropy-modulated differential privacy protection mechanism to add Gaussian noise to each image in the first-updated training set to obtain a second-updated training set.

[0011] Step 3: Construct a dynamic convolutional layer and replace the standard convolutional layer in the original model network ResNet18 with the constructed dynamic convolutional layer to form a dynamic entropy-aware privacy adversarial model;

[0012] Step 4: Use the twice updated training set to pre-train the model;

[0013] Step 5: Adopting a flexible anti-interference training method, a variety of adversarial samples are dynamically generated at different stages of the dynamic entropy-aware privacy adversarial model training. After mixing the normal sample set of each round in the secondary updated training set with the generated adversarial sample set in proportion, the model is trained.

[0014] Step 6: Continue a new round of dynamic entropy-aware privacy adversarial model training. In each round of training, dynamic adaptive sampling and facial image data noise processing are performed based on information entropy. Adversarial samples and ordinary samples are again generated to form a mixed training set for model training.

[0015] Furthermore, the specific process of step 1 is:

[0016] Step 1.1, the original face image dataset obtained is D = {X1, X2, ..., X i ,…,X N}, N is the total number of images, X i is the i-th image; each image is an m×n×3 three-dimensional matrix, where m and m represent the height and width of the image respectively, and 3 represents the dimension, corresponding to the three RGB color channels;

[0017] Step 1.2: Perform Z-score normalization on the three RGB color channels of each image. Divide the normalized pre-processed dataset into a training set and a test set in proportion. The calculation formula for Z-score normalization is as follows:

[0018]

[0019]

[0020]

[0021] Where x′ i (a, b, c) is x i (a, b, c) normalized pixel values; x i (a, b, c) represents the pixel value of the color channel c at position (a, b) of the i-th image; a is the row index of the image in the spatial dimension; b is the column index of the image in the spatial dimension; the color channel c∈{R1, R2, R3}, R1, R2, R3 correspond to the RGB three color channels of the color image respectively; μ c represents the mean value of all pixels on color channel c; σ c Represents the standard deviation on color channel c;

[0022] Step 1.3: Calculate the information entropy of the image based on the distribution of pixel values.

[0023] Step 1.4: Perform dynamic adaptive sampling of face images in the training set based on information entropy to obtain an updated training set.

[0024] Furthermore, the specific process of step 1.3 is as follows:

[0025] Step 1.3.1. Calculate the distribution probability of each pixel value in the image. Assume a pixel value k in the range [0, K], where K is the maximum value of the pixel value k. The distribution probability is calculated as follows:

[0026]

[0027] Among them, p i,k，c is the distribution probability of pixel value k in color channel c of the i-th image; |{(a, b)|x i (a, b, c) = k}| represents the i-th image X i The number of pixels with pixel value k in color channel c;

[0028] Step 1.3.2. Calculate the information entropy of each image in each color channel using the formula:

[0029]

[0030] Among them, H(X i , c) is the i-th image X i Information entropy on color channel c;

[0031] Step 1.3.3: Average the information entropy of each image in each color channel to obtain the final total information entropy of each image. The formula is:

[0032]

[0033] Among them, H total (X i ) is the i-th image X i The final total information entropy.

[0034] Furthermore, the specific process of step 1.4 is as follows:

[0035] Step 1.4.1. Based on the information entropy of each image, dynamically assign a sampling probability to each image. The sampling probability assignment formula is as follows:

[0036]

[0037] in, is the sampling probability of the i-th image; φ≥1 is a tuning parameter that controls the degree of influence of entropy on the sampling probability; j is the image number different from i; X j is the jth image;

[0038] Step 1.4.2: Perform random sampling based on the sampling probability of the image. First, generate a random number for each image, compare the random number with the sampling probability, and update the training set. Let the random number generated for the i-th image be r i ∈[0,1], if Then choose to add the i-th image to the updated training set D′ train ; Otherwise, it is not selected; Finally, an updated training set D′ is obtained train ,

[0039] Furthermore, the specific process of step 2 is as follows:

[0040] Step 2.1: Allocate a privacy budget to each image using the following formula:

[0041]

[0042] Among them, ∈ i is the privacy budget of the i-th image; ∈ 0 is the basic privacy budget; H max is the maximum entropy value of the samples in the data set; γ is the smoothing constant;

[0043] After allocating the privacy budget to each image, the privacy budget of each image is summed up to ensure that the sum of the privacy budgets of all samples does not exceed the total privacy budget ∈ total If the total privacy budget is exceeded, the privacy budgets of all images need to be scaled. The final privacy budget for each image is as follows:

[0044]

[0045] in, is the entropy adjustment privacy budget of the i-th image, i.e., the final privacy budget; ∈ j is the privacy budget of the jth image; M is the updated training set D′ train The total number of images in ;

[0046] Step 2.2: After allocating the privacy budget, use the entropy-modulated differential privacy protection mechanism to add Gaussian noise to each image in the updated training set. The specific process is as follows:

[0047] Step 2.2.1. Calculate the standard deviation of the Gaussian noise to be added to each image based on the final privacy budget and global sensitivity of each image in the training set. The formula is as follows:

[0048]

[0049] in, is the standard deviation of the Gaussian noise of the i-th image; Δf is the global sensitivity; δ is the failure probability parameter in the differential privacy mechanism;

[0050] Step 2.2.2: After obtaining the standard deviation of each image, generate Gaussian noise for each image. Gaussian noise follows a normal distribution:

[0051]

[0052] Among them, g i is the Gaussian noise of the i-th image; is the normal distribution function;

[0053] Step 2.2.3: Add Gaussian noise to the face image to obtain the noisy image. The formula is as follows:

[0054] X″ i =X′ i +g i (12);

[0055] Among them, X″ i is the i-th image after adding noise; X′ i is the i-th image to which the standard deviation is added;

[0056] Step 2.2.4: All the noisy face images constitute the second updated training set D″ train ={X″1,X″2,...,X″ i ,...,X″ M Since the second update only adds noise disturbance to the images in the dataset and does not change the number of datasets, the training set D″ of the second update is train The total number of images in is still M.

[0057] Furthermore, in step 3, the dynamic convolution layer uses the He initialization method to initialize the basic convolution kernel weights and adaptively allocates the convolution kernel weights according to the input image data; the specific working process of the dynamic convolution layer is as follows:

[0058] Step 3.1, assume that the dynamic convolution layer has G basic convolution kernels, the size of each basic convolution kernel is h×h, and the number of channels of the input feature is C in , the number of channels of the output feature is C out The dynamic convolution layer uses the He initialization method to initialize the basic convolution kernel weights of the dynamic convolution layer. The specific initialization formula is as follows:

[0059]

[0060] in, represents the basic convolution kernel weight between the d-th input channel and the e-th output channel in the l-th layer; is the number of channels of the input feature in the lth layer; h(l).h(l) represents the size of the basic convolution kernel in the lth layer. It is a common number when calculated here, so it is represented by dot product;

[0061] Step 3.2: A lightweight sub-network is designed in the dynamic convolution layer. This sub-network consists of a global pooling layer, two fully connected layers, and an activation function. The global information of the input features is extracted through the global pooling layer, and linear transformation is performed through the first fully connected layer. The second fully connected layer generates dynamic weight coefficients and normalizes them through Softmax. Finally, an adaptive weight coefficient is assigned to each convolution kernel. This sub-network is based on D″ train The final weight coefficient set α=[α1,α2,...,α s ,...,α G ], α G is the final weight coefficient of the Gth basic convolution kernel, α s is the final weight coefficient of the sth basic convolution kernel; the specific working process of the subnetwork is:

[0062] Step 3.2.1: The noisy face image first passes through the global average pooling layer to obtain the pooled features. The formula is as follows:

[0063]

[0064] in, is the i-th image after noise addition X″ i The features after pooling; GAP(·) is the global average pooling layer; m and n are the height and width of the image respectively; a is the row index of the image in the spatial dimension; b is the column index of the image in the spatial dimension; x″ i (a, b) represents the pixel value of the i-th image at position (a, b) after noise addition;

[0065] Step 3.2.2: The pooled features are linearly transformed through the first fully connected layer to obtain hidden features. The formula is as follows:

[0066]

[0067] Among them, z is the hidden feature of the pooled feature; is the weight of the first fully connected layer; b1 is the bias of the first fully connected layer; X gp Represents the features after image pooling;

[0068] Step 3.2.3: Use the ReLU activation function to perform nonlinear mapping on the hidden features to obtain the mapped hidden features. The formula is as follows:

[0069] z′=ReLU(z)(16);

[0070] Where z′ is the mapped hidden feature; ReLU(·) is the ReLU activation function;

[0071] Step 3.2.4: Map the hidden features through the second fully connected layer to generate the initial weight coefficient β of the basic convolution kernel, and use the Softmax function to normalize it to obtain the final weight coefficient. The formula is as follows:

[0072]

[0073] Among them, β s is the initial weight coefficient of the sth basic convolution kernel; α s is the final weight coefficient of the sth basic convolution kernel; is the weight of the second fully connected layer; b2 is the bias of the second fully connected layer; Softmax(·) is the Softmax function; ep is the serial number of the basic convolution kernel; exp(·) is the exponential function with e as the base; z′ s is the mapping hidden feature of the s-th basic convolution kernel; β ep is the initial weight coefficient of the epth basic convolution kernel;

[0074] Step 3.3: Dynamically adjust the final weight coefficient of the basic convolution kernel by weighted summation. The dynamic convolution kernel is obtained by weighted summing the final weight coefficients of multiple basic convolution kernels generated by the lightweight sub-network above. The formula is as follows:

[0075]

[0076] Among them, W dyn is the dynamic convolution kernel; W s is the sth basic convolution kernel.

[0077] Furthermore, during the pre-training process of step 4, the noisy image X″ undergoes convolution and pooling operations to generate a feature map, and then global features are extracted through a global pooling layer and a fully connected layer. Finally, the model parameters are adjusted by optimizing the cross entropy loss function of common samples; the formula of the cross entropy loss function of common samples is as follows:

[0078]

[0079] Among them, L CE is the cross entropy loss function value of the common sample; M′ is the number of images in each batch; y iis the true label of the i-th image; is the predicted probability of the i-th image.

[0080] Furthermore, the specific process of step 5 is as follows:

[0081] Step 5.1: Initialize the perturbation amplitude, step size, and first round of adversarial samples, using D″ train The first round of adversarial samples is initialized from the noisy face image. The formula is as follows:

[0082]

[0083] in, is the first round of adversarial samples; X″ is the face image after adding noise;

[0084] Step 5.2: Define the loss function for adversarial samples. The cross entropy loss function is also used. The specific formula is as follows:

[0085]

[0086] Among them, L adv is the cross entropy loss function value of the adversarial sample; M″ is the number of adversarial samples in each round; represents the model's predicted probability of the adversarial sample corresponding to the i-th image; y i represents the true label of the i-th image;

[0087] Step 5.3: Adaptively adjust the step size and perturbation amplitude based on the model feedback to ensure that the step size and perturbation amplitude are the optimal values ​​after dynamic adjustment. The dynamic adjustment formula is as follows:

[0088]

[0089] Among them, c i is the perturbation amplitude of the adversarial sample corresponding to the i-th image, which is used to control the perturbation amount applied to the sample during the adversarial sample generation process; C base is the initial disturbance amplitude; λ′ is the disturbance amplitude adjustment coefficient, which is used to control the adjustment intensity of the disturbance amplitude; is the average loss value of the current batch of adversarial samples, and the formula is as follows:

[0090]

[0091] Among them, N adv is the number of adversarial samples in the current batch;

[0092] According to H total (X i ) Adaptively adjust the generation iteration step of the adversarial sample, the formula is as follows:

[0093] η i =η base ·(1+λ″·H total (X i )) (25);

[0094] Among them, η i is the step size of the adversarial sample corresponding to the i-th image; η base is the initial step size; λ″ is the step size adjustment coefficient, which is used to control the influence of information entropy on step size adjustment;

[0095] Step 5.4: Dynamically select an attack strategy for generating adversarial samples based on the current performance of the model.

[0096] Furthermore, the specific process of step 5.4 is as follows:

[0097] Step 5.4.1. Calculate the loss change rate using the following formula:

[0098]

[0099] in, is the rate of change of cross entropy loss of adversarial samples in round t; are the cross entropy loss function values ​​of the adversarial samples in round t and round t-1 respectively; u is a constant;

[0100] Step 5.4.2: Dynamically switch the attack strategy for generating adversarial samples based on the cross entropy loss change rate of the model; pre-set the first threshold τ fgsm and the second threshold τ pgd , used to determine which attack strategy to choose; it includes three attack strategies, namely the fast gradient sign method, the projected gradient descent method, and the deep deception method; the specific judgment rules are as follows:

[0101] In the early stages of training, if Use the fast gradient sign method to generate adversarial samples. The formula is as follows:

[0102]

[0103] in, is the adversarial example corresponding to the i-th image in round t; is the i-th image after the t-th round of noise addition; is the step size of the adversarial sample corresponding to the i-th image in the t-th round; yes The loss gradient of ; sign is the sign of the gradient; represents the model's predicted probability of the adversarial sample corresponding to the i-th image in the t-th round; represents the true label of the i-th image in the t-th round;

[0104] In the middle of training, if Use the projected gradient descent method to generate adversarial samples and gradually optimize the perturbation. The formula is as follows:

[0105]

[0106] in, is the adversarial example corresponding to the i-th image in the t+1th round; It is a projection operation, which projects the updated adversarial sample back into the range to ensure that the adversarial sample is within the restricted area; is the i-th image after the t-th round of noise addition; Therefore A spherical area with the center and the radius of the disturbance amplitude c; yes The loss gradient of

[0107] In the later stages of training, if Use the deep deception method to generate adversarial samples. The formula is as follows:

[0108]

[0109] Among them, f df (·) is the classification output score of the current sample; It is a sample The gradient of the classification output score; is the square of the L2 norm of the gradient;

[0110] Step 5.5, D″ train The common sample set D in each round normal With adversarial sample set D adv After mixing in proportion, the training is performed with the following formula:

[0111] D train =ζ·D normal +(1-ζ)·D adv (30);

[0112] Among them, D train is the mixed training set; ζ is the mixing ratio;

[0113] The total loss function L of the model total as follows:

[0114]

[0115] in, is the mixing ratio;

[0116] After training, the dynamic entropy-aware privacy adversarial model is evaluated using the test set to measure the performance of the model.

[0117] Furthermore, the specific process of step 6 is as follows:

[0118] Step 6.1: Based on the feedback from each round of dynamic entropy-aware privacy adversarial model training, first perform dynamic adaptive sampling of the data based on information entropy; then, add adaptive Gaussian noise to the sampled data based on the information entropy of the sampled data;

[0119] Step 6.2: Based on the performance of the current dynamic entropy-aware privacy adversarial model, generate adversarial samples from the noisy data. Then, combine these adversarial samples with the noisy normal samples in proportion to form a mixed training set.

[0120] Step 6.3: Use the mixed training set to train the dynamic entropy-aware privacy adversarial model;

[0121] Step 6.4: After each iteration, fine-tune the sampling, noise addition, and adversarial training processes based on the training results of the dynamic entropy-aware privacy adversarial model, gradually improving the model performance until the preset gender recognition accuracy is reached or the maximum number of training rounds is met under the condition of differential privacy protection.

[0122] The beneficial technical effects brought about by the present invention are as follows:

[0123] 1. By using a flexible anti-disturbance training method, the dynamic entropy-aware privacy adversarial model can conduct targeted adversarial training, thereby gradually strengthening its defense capabilities against adversarial attacks. This method effectively avoids the limitations of fixed perturbations and ensures that the model remains stable in the face of various adversarial attacks. In addition, the introduction of dynamic convolution kernels enables the model to have better expressive power when processing different inputs and can automatically adjust to adversarial examples or noise interference, thereby improving its robustness.

[0124] 2. By rationally allocating privacy budgets based on the information entropy of sensitive facial data and implementing an adaptive differential privacy noise addition strategy for this data, we can effectively prevent the leakage of facial data privacy. By flexibly adjusting the noise intensity in an adaptive manner, we can protect data privacy while minimizing the negative impact on model performance and ensuring high model accuracy.

[0125] 3. Dynamically and adaptively sampling facial datasets based on the information entropy of sensitive facial data effectively measures the importance of each sample within the dataset, thereby constructing a diverse and representative training dataset. This approach not only reduces redundant data and accelerates model training, but also reduces the risk of sensitive information leakage, effectively protecting sensitive facial data. BRIEF DESCRIPTION OF THE DRAWINGS

[0126] Figure 1 This is a flow chart of the face-sensitive data protection method based on the elastic anti-interference privacy protection mechanism of the present invention.

[0127] Figure 2 This is a comparison chart of the loss value changes of the present invention and two baseline methods in the experiment of the present invention.

[0128] Figure 3 This is a comparison chart of the accuracy changes of the present invention and two baseline methods in the experiment of the present invention.

[0129] Figure 4 This is a comparison chart of the recall rate changes of the present invention and two baseline methods in the experiment of the present invention.

[0130] Figure 5 This is a comparison chart of the F1 score changes of the present invention and two baseline methods in the experiment of the present invention. DETAILED DESCRIPTION

[0131] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0132] like Figure 1 As shown, the present invention proposes a method for protecting sensitive facial data based on a flexible anti-interference privacy protection mechanism. The flexible anti-interference privacy protection mechanism dynamically adjusts the step size and perturbation size through elastic anti-interference training to prevent overfitting or underfitting caused by too small or too large perturbations, thereby achieving a privacy protection mechanism for sensitive facial data. The method specifically includes the following steps:

[0133] Step 1: Obtain and preprocess the original facial image dataset. The preprocessed dataset is then divided into a training set and a test set proportionally. The information entropy of each image is calculated. Dynamically and adaptively sample the dataset based on the information entropy to obtain an updated training set. During the data preprocessing phase, this paper proposes a dynamic adaptive sampling strategy based on information entropy. This strategy adjusts the sampling rate based on the distribution characteristics of the facial dataset and the weight of sensitive information, maximizing model performance while preserving privacy. The specific process is as follows:

[0134] Step 1.1: The original face image dataset obtained is D = {X1, X2, ..., X i ,...,X N}, N is the total number of images, X i is the i-th image, and the true label corresponding to the i-th image is yi. Each image is a three-dimensional matrix of m×n×3, where m and n represent the height and width of the image respectively, and 3 represents the dimension, corresponding to the three RGB color channels.

[0135] Step 1.2: Perform Z-score normalization on each of the three RGB color channels of each image to eliminate deviations between different images due to brightness or contrast. Split the normalized dataset into a training set and a test set in an 8:2 ratio. The calculation formula for Z-score normalization is as follows:

[0136]

[0137] Where x′ i (a, b, c) is x i (a, b, c) normalized pixel values; x i (a, b, c) represents the pixel value of the color channel c at position (a, b) of the i-th image; a is the row index of the image in the spatial dimension; b is the column index of the image in the spatial dimension; the color channel c∈{R1, R2, R3}, R1, R2, R3 correspond to the RGB three color channels of the color image respectively; μ c represents the mean value of all pixels on color channel c; σ c Indicates the standard deviation on color channel c.

[0138] Step 1.3: Calculate the information entropy based on the distribution of image pixel values. The specific process is as follows:

[0139] Step 1.3.1. First, calculate the distribution probability of each pixel value in the image. Assume a pixel value k in the range [0, K], where K is the maximum value of the pixel value k. The distribution probability is calculated as follows:

[0140]

[0141] Among them, p i,k,c is the distribution probability of pixel value k in color channel c of the i-th image; |{(a, b)|x′ i (a, b, c) = k}| represents the i-th image X i The number of pixels with value k in color channel c.

[0142] Step 1.3.2. Then, calculate the information entropy of each image on each color channel using the formula:

[0143]

[0144] Among them, H(X i , c) is the i-th image X i Information entropy on color channel c;

[0145] Step 1.3.3. Finally, the information entropy of each image in each color channel is averaged and weighted to obtain the final total information entropy of each image. The formula is:

[0146]

[0147] Among them, H total (X i ) is the i-th image X i The final total information entropy;

[0148] Step 1.4: Perform dynamic adaptive sampling of face images in the training set based on information entropy to obtain an updated training set. The specific process is as follows:

[0149] Step 1.4.1. Based on the information entropy of each image, dynamically assign a sampling probability to each image. The sampling probability is adjusted according to the change in entropy value. The higher the entropy value, the higher the sampling frequency. The sampling probability assignment formula is as follows:

[0150]

[0151] in, is the sampling probability of the i-th image; φ≥1 is a tuning parameter that controls the degree of influence of entropy on the sampling probability. A larger φ value will result in a higher sampling weight for images with higher entropy values; j is the image number different from i; X j is the jth image.

[0152] Step 1.4.2: Randomly sample the face image based on its sampling probability. First, generate a random number for each image, compare the random number with the sampling probability, and update the training set. Let the random number generated for the i-th image be r i ∈[0, 1], if Then choose to add the i-th image to the updated training set D′ train Otherwise, it is not selected. Finally, we get an updated training set D′ train ,

[0153] Step 2: Allocate a privacy budget for facial image data and use an entropy-modulated differential privacy protection mechanism to add Gaussian noise to each image in the first-updated training set to obtain a second-updated training set. To achieve better privacy protection, this paper designs an entropy-modulated differential privacy protection mechanism. After dynamically and adaptively sampling the facial dataset, it adaptively adds varying degrees of Gaussian noise based on the information entropy of the facial data, thereby achieving refined privacy protection for sensitive facial data under differential privacy constraints. The specific process is as follows:

[0154] Step 2.1. This invention uses the information entropy of facial images to dynamically adjust the privacy budget. Images with high information entropy contain more information, so they generally require stronger privacy protection, while images with low information entropy are relatively simple, so their privacy protection is relatively less. The formula for allocating the privacy budget for each image is as follows:

[0155]

[0156] Among them, ∈ i is the privacy budget of the i-th image; ∈ 0 is the basic privacy budget, defined as the maximum allowed privacy budget; H max is the maximum entropy value of the samples in the data set, that is, H max =max{H total (X i ));γ is a smoothing constant to avoid the denominator being zero;

[0157] After allocating the privacy budget for each face image, the privacy budget of each image is summed up to ensure that the sum of the privacy budgets of all samples does not exceed the total privacy budget ∈ total If the total privacy budget is exceeded, the privacy budgets of all images need to be scaled, and the final privacy budget for each image is as follows:

[0158]

[0159] in, is the entropy adjustment privacy budget of the i-th image, i.e., the final privacy budget; ∈ j is the privacy budget of the jth image; M is the updated training set D′ train The total number of images in .

[0160] Step 2.2: After allocating the privacy budget, use the entropy-modulated differential privacy protection mechanism to add Gaussian noise to each image in the updated training set. The specific process is as follows:

[0161] Step 2.2.1. Calculate the standard deviation of the Gaussian noise to be added to each image based on the final privacy budget and global sensitivity of each image in the training set. The formula is as follows:

[0162]

[0163] in, is the standard deviation of the Gaussian noise of the i-th image; Δf is the global sensitivity, which is 2 in the present invention; δ is the failure probability parameter in the differential privacy mechanism, which is 10 in the present invention. -6 .

[0164] Step 2.2.2. After obtaining the standard deviation of each image, generate Gaussian noise that conforms to the following distribution:

[0165]

[0166] Among them, g i is the Gaussian noise of the i-th image, and its dimension is consistent with that of the face image; is the normal distribution function;

[0167] Step 2.2.3: Add Gaussian noise to the face image to obtain the noisy image. The formula is as follows:

[0168] X″ i =X′ i +g i (12);

[0169] Among them, X″ i is the i-th image after adding noise; X′ i is the i-th image to which the standard deviation is added;

[0170] Step 2.2.4: All the noisy face images constitute the second updated training set D″ train ={X″1,X″2,…,X″ i ,…,X″ M Since the second update only adds noise perturbations to the images in the dataset and does not change the number of datasets, the training set D″ of the second update is train The total number of images in is still M.

[0171] Step 3: Construct a dynamic entropy-aware privacy adversarial model. First, construct a dynamic convolutional layer. Then, replace the standard convolutional layer in the original model network ResNet18 with the constructed dynamic convolutional layer to form a new model. This new model is the dynamic entropy-aware privacy adversarial model. The dynamic convolutional layer enhances the flexibility and adaptability of the model network in feature extraction, helping the model learn the differences in characteristics of different genders. The dynamic convolutional layer uses the He initialization method to initialize the basic convolution kernel weights and adaptively assigns the convolution kernel weights based on the input facial image data. The specific working process of the dynamic convolutional layer is as follows:

[0172] Step 3.1, assume that the dynamic convolution layer has G basic convolution kernels, the size of each basic convolution kernel is h×h, and the number of channels of the input feature is C in , the number of channels of the output feature is C out The basic convolution kernel weight is defined as In order to ensure the numerical stability of network training, the present invention uses the He initialization method to initialize the basic convolution kernel weights of the dynamic convolution layer. The specific initialization formula is as follows:

[0173]

[0174] in, represents the basic convolution kernel weight between the d-th input channel and the e-th output channel in the l-th layer; is the number of channels of input features in the lth layer; h (l) .h (l) Represents the size of the basic convolution kernel in the lth layer. It is a common number when calculated here, so it is represented by dot product.

[0175] Step 3.2: To enable the weights of the basic convolution kernels to be adaptively adjusted based on the input data, the present invention designs a lightweight subnetwork within the dynamic convolution layer. This subnetwork consists of a global pooling layer, two fully connected layers, and an activation function. The global pooling layer extracts global information about the input features, which is then linearly transformed by the first fully connected layer. The second fully connected layer generates dynamic weight coefficients, which are then normalized by Softmax. Ultimately, adaptive weight coefficients are assigned to each convolution kernel.

[0176] The subnetwork is based on D″ train The final weight coefficient set α=[α1,α2,...,α s ,...,α G ], α G is the final weight coefficient of the Gth basic convolution kernel, α s is the final weight coefficient of the sth basic convolution kernel. The following is the specific working process of the sub-network:

[0177] Step 3.2.1: The face image after adding noise first passes through the global average pooling layer to obtain the pooled feature X gp , the formula is as follows:

[0178]

[0179] in, is the i-th image after noise addition X″ i Features after pooling; GAP(·) is the global average pooling operation; m and n are the height and width of the image respectively; a is the row index of the image in the spatial dimension; b is the column index of the image in the spatial dimension. x″ i (a, b) represents the pixel value of the i-th image at position (a, b) after noise addition.

[0180] Step 3.2.2: The pooled features are linearly transformed through the first fully connected layer to obtain hidden features. The formula is as follows:

[0181]

[0182] Among them, z is the hidden feature of the pooled feature; is the weight of the first fully connected layer; b1 is the bias of the first fully connected layer; X gp Represents the features after image pooling;

[0183] Step 3.2.3: Use the ReLU activation function to perform nonlinear mapping on the hidden features to obtain the mapped hidden features. The formula is as follows:

[0184] z′=ReLU(z)(16);

[0185] Where z′ is the mapped hidden feature; ReLU(·) is the ReLU activation function;

[0186] Step 3.2.4: Map the hidden features through the second fully connected layer to generate the initial weight coefficient β of the basic convolution kernel s , and use the Softmax function to normalize it to get the final weight coefficient. The formula is as follows:

[0187]

[0188] Among them, β s is the initial weight coefficient of the sth basic convolution kernel; α s is the final weight coefficient of the sth basic convolution kernel; is the weight of the second fully connected layer; b2 is the bias of the second fully connected layer; Softmax(·) is the Softmax function; ep is the serial number of the basic convolution kernel; exp(·) is the exponential function with e as the base; z′ s is the hidden feature of the e-map of the s-th basic convolution kernel; β ep is the initial weight coefficient of the epth basic convolution kernel.

[0189] Step 3.3: Dynamically adjust the final weight coefficient of the basic convolution kernel by weighted summation. The dynamic convolution kernel is obtained by weighted summing the final weight coefficients of multiple basic convolution kernels generated by the lightweight sub-network above. The formula is as follows:

[0190]

[0191] Among them, W dyn is the dynamic convolution kernel; W s is the sth basic convolution kernel.

[0192] Step 4: To improve the generalization ability of the dynamic entropy-aware privacy adversarial model in gender classification tasks and to provide a reference for subsequent adversarial samples and D″ train The samples in (in order to facilitate the comparison and distinction with the adversarial samples, D″ trainThe samples in the training set are called normal samples) and the training set D″ after the noise is added is used to update the training set D″. train Perform model pre-training. The main goal of pre-training is to enable the model to fully learn the basic features of facial image data on standard samples. In this process, the noisy image X″ undergoes convolution, pooling and other operations to generate a feature map X tr , then extract global features through the global pooling layer and the fully connected layer, and finally adjust the model parameters by optimizing the cross entropy loss function of common samples, so that the classification accuracy of the model on common samples is improved. The formula of the cross entropy loss function of common samples is as follows:

[0193]

[0194] Among them, L CE is the cross entropy loss function value of the common sample; M′ is the number of images in each batch; y i is the true label of the i-th image; is the predicted probability of the i-th image.

[0195] Step 5: During the training phase of the dynamic entropy-aware privacy adversarial model, the present invention proposes a flexible anti-interference training method that dynamically generates diverse adversarial samples at different stages of model training, enhancing the model's robustness against various types of adversarial attacks while reducing the risk of overfitting. After completing the pre-training of the model, adversarial samples are dynamically generated based on the model's performance, and the normal sample set and adversarial sample set from each round of the secondary updated training set are mixed in a ratio of 7:3 before model training. The specific process is as follows:

[0196] Step 5.1. Initialize the perturbation amplitude c base , step length η base And the first round of adversarial samples. In this invention, the initial value of the perturbation amplitude is set to 0.03, the initial value of the step length is set to 0.015, and D″ is used train The first round of adversarial samples is initialized from the noisy face image. The formula is as follows:

[0197]

[0198] in, is the first round of adversarial samples; X″ is the face image after adding noise;

[0199] Step 5.2: Define the loss function for adversarial samples. The cross entropy loss function is also used. The specific formula is as follows:

[0200]

[0201] Among them, L advis the cross entropy loss function value of the adversarial sample; M″ is the number of adversarial samples in each round; represents the model's predicted probability of the adversarial sample corresponding to the i-th image; y i represents the true label of the i-th image.

[0202] Step 5.3: Adaptively adjust the step size and perturbation amplitude based on the model's feedback to ensure that these parameters are optimal after dynamic adjustment. In this invention, the cross entropy loss function value of the model's current adversarial sample is used as feedback to adjust the perturbation amplitude. When the model's gradient is large, the perturbation amplitude is appropriately increased, and when the model's gradient is small, the perturbation amplitude is appropriately reduced. The dynamic adjustment formula is as follows:

[0203]

[0204] Among them, c i is the perturbation amplitude of the adversarial sample corresponding to the i-th image, which is used to control the perturbation amount applied to the sample during the adversarial sample generation process; c base is the initial disturbance amplitude; λ′ is the disturbance amplitude adjustment coefficient, which is used to control the adjustment intensity of the disturbance amplitude. In the present invention, the value of λ′ is 0.1. is the average loss value of the current batch of adversarial samples, and the formula is as follows:

[0205]

[0206] Among them, N adv is the number of adversarial examples in the current batch.

[0207] According to H total (X i ) Adaptively adjust the generation iteration step of the adversarial sample, the formula is as follows:

[0208] η i =η base ·(1+λ″·H total (X i )) (25);

[0209] Among them, η i is the step size of the adversarial sample corresponding to the i-th image, which is used to control the size of the perturbation update during the adversarial sample generation process; η base is the initial step length; λ″ is the step length adjustment coefficient, which is used to control the influence of information entropy on the step length adjustment. In the present invention, the value of λ″ is 0.1.

[0210] Step 5.4: Dynamically select an attack strategy for generating adversarial samples based on the current performance of the model. In this invention, the effectiveness of the current strategy is determined based on the changes in the loss function of the adversarial samples at different training stages. By monitoring the changes in the loss function of the adversarial samples, it is determined whether the current attack strategy has failed, and thus whether to switch to a stronger attack strategy to ensure the effectiveness of the currently generated adversarial samples. The specific process is as follows:

[0211] Step 5.4.1. Calculate the loss change rate using the following formula:

[0212]

[0213] in, is the rate of change of cross entropy loss of adversarial samples in round t; are the cross entropy loss function values ​​of the adversarial samples in the tth and t-1th rounds, respectively; u is a constant, which takes a smaller value to prevent the denominator from being zero.

[0214] Step 5.4.2: Dynamically switch the attack strategy for generating adversarial samples based on the cross entropy loss change rate of the model. In the present invention, the cross entropy loss change rate is calculated by five consecutive iterations to determine whether to switch the attack strategy. The first threshold τ is pre-set. fgsm and the second threshold τ pgd , used to determine which attack strategy to choose; in the present invention, τ fgsm Set to 0.01, τ pgd Set to 0.001.

[0215] The present invention includes three attack strategies, which respectively adopt the Fast Gradient Sign Method (FGSM), the Projected Gradient Descent (PGD), and the DeepFool method; in the early stage of training, the model parameters have not yet converged, and the use of FGSM can quickly generate simple adversarial samples, so that the model can perceive and adapt to the adversarial perturbations as early as possible. In the middle of training, the model begins to converge and the loss changes tend to be stable. At this time, the use of PGD can continuously optimize the adversarial perturbations to make them more aggressive, thereby further improving the robustness of the model. In the later stage of training, the model has tended to converge, and the use of DeepFool can effectively verify its resistance to extreme adversarial perturbations. The specific judgment rules are as follows:

[0216] In the early stages of training, the model loss varies greatly. Use the fast gradient sign method to quickly generate adversarial samples. The formula is as follows:

[0217]

[0218] in, is the adversarial example corresponding to the i-th image in round t; is the i-th image after the t-th round of noise addition; is the step size of the adversarial sample corresponding to the i-th image in the t-th round, which is used to control the update amplitude of each step; yes The loss gradient of ; sign is the sign of the gradient, which converts each element of the gradient vector into its sign, that is, if the gradient is positive, the sign is +1; if it is negative, the sign is -1; if it is zero, the sign is 0. represents the model's predicted probability of the adversarial sample corresponding to the i-th image in the t-th round; represents the true label of the i-th image in the t-th round.

[0219] In the middle of training, the model loss tends to stabilize but has not yet converged. If Use the projected gradient descent method to generate adversarial samples and gradually optimize the perturbation. The formula is as follows:

[0220]

[0221] in, is the adversarial example corresponding to the i-th image in the t+1th round; It is a projection operation, which projects the updated adversarial sample back into the range to ensure that the adversarial sample is within the restricted area; is the i-th image after the t-th round of noise addition; Therefore A spherical area with the center and the radius of the disturbance amplitude c; yes The loss gradient of .

[0222] In the later stages of training, the model loss tends to converge. If Use the deep deception method to generate stronger adversarial samples. The formula is as follows:

[0223]

[0224] Among them, f df (·) is the classification output score of the current sample. It is a sample Gradient of the classification output score. is the square of the L2 norm of the gradient.

[0225] Step 5.5, D″ train The common sample set D in each round normal With adversarial sample set D adv After mixing in a certain ratio, training is performed. The formula is as follows:

[0226] D train =ζ·D normal +(1-ζ)·D adv (30);

[0227] Among them, D train is the mixed training set; ζ is the mixing ratio, which is 0.7 in the present invention.

[0228] The total loss function L of the model total as follows:

[0229]

[0230] in, Is the mixing ratio, in the present invention It is 0.7.

[0231] After training, the dynamic entropy-aware privacy adversarial model is evaluated using the test set to measure the performance of the model and ensure its reliability in practical applications.

[0232] Step 6: Continue a new round of dynamic entropy-aware privacy adversarial model training. In each round of training, dynamic adaptive sampling and face image data noise processing are performed based on information entropy. Adversarial samples and normal samples are again generated to form a mixed training set for model training. The specific process is as follows:

[0233] Step 6.1: Based on the feedback from each round of dynamic entropy-aware privacy-preserving adversarial model training, the data is first dynamically and adaptively sampled based on information entropy to select samples with high information content. Next, adaptive Gaussian noise is added to the sampled data based on its information entropy to blur the details of sensitive facial features, thereby reducing the risk of individual privacy leaks.

[0234] Step 6.2: Based on the performance of the current dynamic entropy-aware privacy adversarial model, generate adversarial samples X based on the noisy data. adv Then, these adversarial samples are combined with ordinary samples after differential privacy noise processing in a certain proportion to form a mixed training set.

[0235] Step 6.3: Use the mixed training set to train the dynamic entropy-aware privacy adversarial model. With the dual protection of differential privacy and adversarial examples, the model's robustness to privacy noise and adversarial perturbations is improved, thereby enhancing the effectiveness of protecting facial data privacy.

[0236] Step 6.4: After each iteration, fine-tune the sampling, noise addition, and adversarial training processes based on the training results of the dynamic entropy-aware privacy adversarial model, gradually improving the model performance until the preset gender recognition accuracy is reached or the maximum number of training rounds is met under the condition of differential privacy protection.

[0237] In order to prove the feasibility and rationality of the present invention in theory, the following proof and analysis are given.

[0238] (1) The present invention effectively ensures the privacy and security of data.

[0239] Analysis: Using information entropy to adjust the sampling probability can indirectly protect sensitive data. For samples with higher entropy values, it means that the model uncertainty is greater. Such samples are more critical to model training, so their sampling probability should be increased. For samples with lower entropy values, their sampling probability can be appropriately reduced. Assuming that some samples have lower entropy values, this means that the model is more confident in these samples and may have fully learned their features. At this time, by dynamically adjusting the sampling probability, the frequency of these samples in training can be reduced, thereby reducing the model's dependence on these samples and the potential risk of privacy leakage. The entropy-based sampling probability can be defined as: p s (X i )∝exp(ξH(X i )); where p s (X i ) is X i The sampling probability of the sample is greater than that of the original sample. ξ is a control parameter used to adjust the degree of influence of entropy on the sampling probability. For samples with larger entropy values, the sampling probability will be greater, while for samples with smaller entropy values, the sampling probability will be reduced accordingly.

[0240] Assuming that the entropy value of sensitive data is low, according to the above sampling strategy, the sampling probability will be reduced, which can reduce the frequent sampling of sensitive data and thus reduce the risk of privacy leakage.

[0241] By introducing noise into the data and individually allocating a privacy budget, the entropy-modulated differential privacy mechanism provides stronger protection for sensitive facial data. Specifically, as the sensitivity based on information entropy increases, the privacy budget decreases accordingly, increasing the magnitude of the noise added to the data. This mechanism ensures that the model does not over-rely on or learn from sensitive data during training, effectively protecting data privacy.

[0242] (2) The present invention improves the robustness and stability of the model.

[0243] Analysis: Elastic anti-disturbance training dynamically adjusts the step size and perturbation magnitude based on model performance, effectively addressing gradient variations at different training stages. This elastic mechanism makes training more flexible and prevents overfitting or underfitting due to excessively small or large perturbations. In adversarial training, the loss function is often unstable. Adaptive adjustment stabilizes the convergence of the loss by dynamically controlling the perturbation strength.

[0244] In adversarial training, the loss function is often unstable. As can be seen from formula (23), the adjustment of the perturbation size depends on the current loss change. We stabilize the convergence of the loss by dynamically controlling the perturbation intensity, avoiding the convergence difficulties that may occur in adversarial training. At the same time, adaptive adversarial training keeps the model within a reasonable perturbation range through dynamic adjustment, effectively preventing gradient vanishing or gradient exploding, and ensuring gradient stability.

[0245] By adopting three different adversarial sample generation strategies to simulate adversarial attacks of different intensities and complexities, the model can effectively cope with adversarial attacks of different types and intensities, improving its robustness and adversarial capabilities in practical applications.

[0246] In order to demonstrate the feasibility and superiority of the present invention in practice, the following comparative experiments are given.

[0247] This paper conducts experiments using the Gender Classification Dataset, a face dataset on the Kaggle website, as an example. This dataset is obtained by uploading photos from the CelebA dataset through screening and processing. Therefore, the photos only contain a face, with very little irrelevant information such as the background and other body parts. This prevents irrelevant information in the photos from interfering with the model classification.

[0248] The present invention uses a dynamic entropy-aware privacy adversarial model for training, which can effectively identify and classify data sets; using the total loss function L total As a loss function that measures the difference between the predicted value and the label; set multiple evaluation indicators to measure the performance of the model, including accuracy, precision, recall and F1 score, the formulas are:

[0249]

[0250] Among them, Acc represents the accuracy index; Prec represents the accuracy index; Recall represents the recall index; F1 represents the F1 score index; TP, TN, FP, and FN represent the number of true positives, true negatives, false positives, and false negatives during the model training process, respectively.

[0251] All baseline solutions used in this paper are based on a dynamic entropy-aware privacy adversarial model to protect sensitive personal data. Table 1 shows a functional comparison of the solutions of this paper and the baseline solutions.

[0252] Table 1 Function comparison table of the present invention solution and the baseline solution

[0253]

[0254]

[0255] In the experiment, the optimizer used was Adam, the learning rate was 0.001, the iteration round was 60 rounds, and the training hyperparameters remained the same. The comparison charts of loss value, accuracy, recall rate and F1 score in the experiment are as follows: Figure 2 、 Figure 3 、 Figure 4 、 Figure 5 shown.

[0256] The experimental results show that the proposed solution can protect privacy while maintaining model performance comparable to the baseline method. This experiment demonstrates the feasibility of the proposed solution in practice.

[0257] Of course, the above description is not a limitation of the present invention, and the present invention is not limited to the above examples. Changes, modifications, additions or substitutions made by technicians in this technical field within the essential scope of the present invention should also fall within the scope of protection of the present invention.

Claims

1. A method for protecting sensitive face data based on a flexible anti-interference privacy protection mechanism, characterized in that: The steps include: Step 1: Obtain the original face image dataset and preprocess it. Divide the preprocessed dataset into a training set and a test set in proportion. Calculate the information entropy of each image. Dynamically and adaptively sample the dataset based on the information entropy to obtain an updated training set. Step 2: Allocate a privacy budget for facial image data and use the entropy-modulated differential privacy protection mechanism to add Gaussian noise to each image in the first-updated training set to obtain a second-updated training set. Step 3: Construct a dynamic convolutional layer and replace the standard convolutional layer in the original model network ResNet18 with the constructed dynamic convolutional layer to form a dynamic entropy-aware privacy adversarial model; Step 4: Use the twice updated training set to pre-train the model; Step 5: Adopting the elastic anti-interference training method, a variety of adversarial samples are dynamically generated at different stages of the dynamic entropy-aware privacy adversarial model training. After mixing the normal sample set of each round in the second-updated training set with the generated adversarial sample set in proportion, the model training is carried out. The specific process is as follows: Step 5.1: Initialize the perturbation amplitude, step size, and first round of adversarial samples, using D″ train The first round of adversarial samples is initialized from the noisy face image. The formula is as follows: in, is the first round of adversarial samples; X″ is the face image after adding noise; Step 5.2: Define the loss function for adversarial samples. The cross entropy loss function is also used. The specific formula is as follows: Among them, L adv is the cross entropy loss function value of the adversarial sample; M″ is the number of adversarial samples in each round; represents the model's predicted probability of the adversarial sample corresponding to the i-th image; y i represents the true label of the i-th image; Step 5.3: Adaptively adjust the step size and perturbation amplitude based on the model feedback to ensure that the step size and perturbation amplitude are the optimal values ​​after dynamic adjustment. The dynamic adjustment formula is as follows: Among them, c i is the perturbation amplitude of the adversarial sample corresponding to the i-th image, which is used to control the perturbation amount applied to the sample during the adversarial sample generation process; c base is the initial perturbation amplitude; λ ′ is the disturbance amplitude adjustment coefficient, which is used to control the adjustment intensity of the disturbance amplitude; is the average loss value of the current batch of adversarial samples, and the formula is as follows: Among them, N adv is the number of adversarial samples in the current batch; According to H total (X i ) Adaptively adjust the generation iteration step of the adversarial sample, the formula is as follows: or i =the base ·(1+λ″·H total (X i )) (25); Among them, η i is the step size of the adversarial sample corresponding to the i-th image; η base is the initial step size; λ″ is the step size adjustment coefficient, which is used to control the influence of information entropy on step size adjustment; Step 5.4: Dynamically select an attack strategy for generating adversarial samples based on the model's current performance. The specific process is as follows: Step 5.4.

1. Calculate the loss change rate using the following formula: in, is the rate of change of cross entropy loss of adversarial samples in round t; are the cross entropy loss function values ​​of the adversarial samples in round t and round t-1 respectively; u is a constant; Step 5.4.2: Dynamically switch the attack strategy for generating adversarial samples based on the cross entropy loss change rate of the model; pre-set the first threshold τ fgsm and the second threshold τ pgd , used to determine which attack strategy to choose; it includes three attack strategies, namely the fast gradient sign method, the projected gradient descent method, and the deep deception method; the specific judgment rules are as follows: In the early stages of training, if Use the fast gradient sign method to generate adversarial samples. The formula is as follows: in, is the adversarial example corresponding to the i-th image in round t; is the i-th image after the t-th round of noise addition; is the step size of the adversarial sample corresponding to the i-th image in the t-th round; yes The loss gradient of ; sign is the sign of the gradient; represents the model's predicted probability of the adversarial sample corresponding to the i-th image in the t-th round; represents the true label of the i-th image in the t-th round; In the middle of training, if Use the projected gradient descent method to generate adversarial samples and gradually optimize the perturbation. The formula is as follows: in, is the adversarial example corresponding to the i-th image in the t+1th round; It is a projection operation, which projects the updated adversarial sample back into the range to ensure that the adversarial sample is within the restricted area; is the i-th image after the t-th round of noise addition; Therefore A spherical area with the center and the radius of the disturbance amplitude c; yes The loss gradient of In the later stages of training, if Use the deep deception method to generate adversarial samples. The formula is as follows: Among them, f df (·) is the classification output score of the current sample; It is a sample The gradient of the classification output score; is the square of the L2 norm of the gradient; Step 5.5, D″ train The common sample set D in each round normal With adversarial sample set D adv After mixing in proportion, the training is performed with the following formula: D train =ζ·D normal +(1-ζ)·D adv (30); Among them, D train is the mixed training set; ζ is the mixing ratio; The total loss function L of the model total as follows: in, is the mixing ratio; After training, the dynamic entropy-aware privacy adversarial model is evaluated using the test set to measure the performance of the model; Step 6: Continue a new round of dynamic entropy-aware privacy adversarial model training. In each round of training, dynamic adaptive sampling and facial image data noise processing are performed based on information entropy. Adversarial samples and normal samples are again generated to form a mixed training set for model training. The specific process is as follows: Step 6.1: Based on the feedback from each round of dynamic entropy-aware privacy adversarial model training, first perform dynamic adaptive sampling of the data based on information entropy; then, add adaptive Gaussian noise to the sampled data based on the information entropy of the sampled data; Step 6.2: Based on the performance of the current dynamic entropy-aware privacy adversarial model, generate adversarial samples from the noisy data. Then, combine these adversarial samples with the noisy normal samples in proportion to form a mixed training set. Step 6.3: Use the mixed training set to train the dynamic entropy-aware privacy adversarial model; Step 6.4: After each iteration, fine-tune the sampling, noise addition, and adversarial training processes based on the training results of the dynamic entropy-aware privacy adversarial model, gradually improving the model performance until the preset gender recognition accuracy is reached or the maximum number of training rounds is met under the condition of differential privacy protection.

2. The method for protecting sensitive face data based on a flexible anti-interference privacy protection mechanism according to claim 1 is characterized in that: The specific process of step 1 is: Step 1.1, the original face image dataset obtained is D = {X1, X2, ..., X i ,…,X N }, N is the total number of images, X i is the i-th image; each image is a three-dimensional matrix of m×n×3, where m and n represent the height and width of the image respectively, and 3 represents the dimension, corresponding to the three RGB color channels; Step 1.2: Perform Z-score normalization on the three RGB color channels of each image. Divide the normalized pre-processed dataset into a training set and a test set in proportion. The calculation formula for Z-score normalization is as follows: Where x′ i (a,b,c) is x i (a, b, c) normalized pixel values; x i (a, b, c) represents the pixel value of the color channel c at position (a, b) of the i-th image; a is the row index of the image in the spatial dimension; b is the column index of the image in the spatial dimension; color channel c∈{R1, R2, R3}, R1, R2, R3 correspond to the RGB three color channels of the color image respectively; μ c represents the mean value of all pixels on color channel c; σ c Represents the standard deviation on color channel c; Step 1.3: Calculate the information entropy of the image based on the distribution of pixel values. Step 1.4: Perform dynamic adaptive sampling of face images in the training set based on information entropy to obtain an updated training set.

3. The method for protecting sensitive face data based on a flexible anti-interference privacy protection mechanism according to claim 2 is characterized in that: The specific process of step 1.3 is as follows: Step 1.3.

1. Calculate the distribution probability of each pixel value in the image. Assume a pixel value k in the range [0, K], where K is the maximum value of the pixel value k. The distribution probability is calculated as follows: Among them, p i,k,c is the distribution probability of pixel value k in color channel c of the i-th image; |{(a,b)|x i (a, b, c) = k}| represents the i-th image X i The number of pixels with pixel value k in color channel c; Step 1.3.

2. Calculate the information entropy of each image in each color channel using the formula: Among them, H(X i ,c) is the i-th image X i Information entropy on color channel c; Step 1.3.3: Average the information entropy of each image in each color channel to obtain the final total information entropy of each image. The formula is: Among them, H total (X i ) is the i-th image X i The final total information entropy.

4. The method for protecting sensitive facial data based on a flexible anti-interference privacy protection mechanism according to claim 3 is characterized in that: The specific process of step 1.4 is as follows: Step 1.4.

1. Based on the information entropy of each image, dynamically assign a sampling probability to each image. The sampling probability assignment formula is as follows: in, is the sampling probability of the i-th image; φ≥1 is a tuning parameter that controls the degree of influence of entropy on the sampling probability; j is the image number different from i; X j is the jth image; Step 1.4.2: Perform random sampling based on the sampling probability of the image. First, generate a random number for each image, compare the random number with the sampling probability, and update the training set. Let the random number generated for the i-th image be r i ∈[0,1], if Then choose to add the i-th image to the updated training set D′ train ; Otherwise, it is not selected; Finally, an updated training set D′ is obtained train , 5. The method for protecting sensitive face data based on a flexible anti-interference privacy protection mechanism according to claim 4 is characterized in that: The specific process of step 2 is: Step 2.1: Allocate a privacy budget to each image using the following formula: Among them, ∈ i is the privacy budget of the i-th image; ∈ 0 is the basic privacy budget; H max is the maximum entropy value of the samples in the data set; γ is the smoothing constant; After allocating the privacy budget to each image, the privacy budget of each image is summed up to ensure that the sum of the privacy budgets of all samples does not exceed the total privacy budget ∈ total If the total privacy budget is exceeded, the privacy budgets of all images need to be scaled. The final privacy budget for each image is as follows: in, is the entropy adjustment privacy budget of the i-th image, i.e., the final privacy budget; ∈ j is the privacy budget of the jth image; M is the updated training set D′ train The total number of images in ; Step 2.2: After allocating the privacy budget, use the entropy-modulated differential privacy protection mechanism to add Gaussian noise to each image in the updated training set. The specific process is as follows: Step 2.2.

1. Calculate the standard deviation of the Gaussian noise to be added to each image based on the final privacy budget and global sensitivity of each image in the training set. The formula is as follows: in, is the standard deviation of the Gaussian noise of the i-th image; Δf is the global sensitivity; δ is the failure probability parameter in the differential privacy mechanism; Step 2.2.2: After obtaining the standard deviation of each image, generate Gaussian noise for each image. Gaussian noise follows a normal distribution: Among them, g i is the Gaussian noise of the i-th image; is the normal distribution function; Step 2.2.3: Add Gaussian noise to the face image to obtain the noisy image. The formula is as follows: X″ i =X′ i +g i (12); Among them, X″ i is the i-th image after adding noise; X′ i is the i-th image to which the standard deviation is added; Step 2.2.4: All the noisy face images constitute the second updated training set D″ train ={X″1,X″2,…,X″ i ,…,X″ M Since the second update only adds noise disturbance to the images in the dataset and does not change the number of datasets, the training set D″ of the second update is train The total number of images in is still M.

6. The method for protecting sensitive face data based on a flexible anti-interference privacy protection mechanism according to claim 5, characterized in that: In step 3, the dynamic convolution layer uses the He initialization method to initialize the basic convolution kernel weights and adaptively allocates the convolution kernel weights according to the input image data. The specific working process of the dynamic convolution layer is as follows: Step 3.1, assume that the dynamic convolution layer has G basic convolution kernels, the size of each basic convolution kernel is h×h, and the number of channels of the input feature is C in , the number of channels of the output feature is C out The dynamic convolution layer uses the He initialization method to initialize the basic convolution kernel weights of the dynamic convolution layer. The specific initialization formula is as follows: in, represents the basic convolution kernel weight between the d-th input channel and the e-th output channel in the l-th layer; is the number of channels of input features in the lth layer; h (l) ·h (l) Indicates the size of the basic convolution kernel in the lth layer. It is a common number when calculated here, so it is represented by dot multiplication; Step 3.2: A lightweight sub-network is designed in the dynamic convolution layer. This sub-network consists of a global pooling layer, two fully connected layers, and an activation function. The global information of the input features is extracted through the global pooling layer, and linear transformation is performed through the first fully connected layer. The second fully connected layer generates dynamic weight coefficients and normalizes them through Softmax. Finally, an adaptive weight coefficient is assigned to each convolution kernel. This sub-network is based on D″ train The final weight coefficient set α=[α1,α2,…,α s ,…,α G ],ɑ G is the final weight coefficient of the Gth basic convolution kernel, α s is the final weight coefficient of the sth basic convolution kernel; the specific working process of the subnetwork is: Step 3.2.1: The noisy face image first passes through the global average pooling layer to obtain the pooled features. The formula is as follows: in, is the i-th image after noise addition X″ i The features after pooling; GAP(·) is the global average pooling layer; m and n are the height and width of the image respectively; a is the row index of the image in the spatial dimension; b is the column index of the image in the spatial dimension; x″ i (a, b) represents the pixel value of the i-th image at position (a, b) after noise addition; Step 3.2.2: The pooled features are linearly transformed through the first fully connected layer to obtain hidden features. The formula is as follows: Among them, z is the hidden feature of the pooled feature; is the weight of the first fully connected layer; b1 is the bias of the first fully connected layer; X gp Represents the features after image pooling; Step 3.2.3: Use the ReLU activation function to perform nonlinear mapping on the hidden features to obtain the mapped hidden features. The formula is as follows: z′=ReLU(z) (16); Where z′ is the mapped hidden feature; ReLU(·) is the ReLU activation function; Step 3.2.4: Map the hidden features through the second fully connected layer to generate the initial weight coefficient β of the basic convolution kernel, and use the Softmax function to normalize it to obtain the final weight coefficient. The formula is as follows: Among them, β s is the initial weight coefficient of the sth basic convolution kernel; α s is the final weight coefficient of the sth basic convolution kernel; is the weight of the second fully connected layer; b2 is the bias of the second fully connected layer; Softmax(·) is the Softmax function; ep is the serial number of the basic convolution kernel; exp(·) is the exponential function with e as the base; z′ s is the mapping hidden feature of the s-th basic convolution kernel; β ep is the initial weight coefficient of the epth basic convolution kernel; Step 3.3: Dynamically adjust the final weight coefficient of the basic convolution kernel by weighted summation. The dynamic convolution kernel is obtained by weighted summing the final weight coefficients of multiple basic convolution kernels generated by the lightweight sub-network above. The formula is as follows: Among them, W dyn is the dynamic convolution kernel; W s is the sth basic convolution kernel.

7. The method for protecting sensitive facial data based on a flexible anti-interference privacy protection mechanism according to claim 6, characterized in that: During the pre-training process of step 4, the noisy image X″ undergoes convolution and pooling operations to generate a feature map, and then global features are extracted through a global pooling layer and a fully connected layer. Finally, the model parameters are adjusted by optimizing the cross entropy loss function of common samples; the formula of the cross entropy loss function of common samples is as follows: Among them, L CE is the cross entropy loss function value of the common sample; M′ is the number of images in each batch; y i is the true label of the i-th image; is the predicted probability of the i-th image.

Images