Face sensitive data protection method based on elastic anti-interference privacy protection mechanism

Abstract

The invention discloses a human face sensitive data protection method based on an elastic anti-interference privacy protection mechanism, belongs to the field of information processing, and provides a dynamic self-adaptive sampling strategy based on information entropy in a data preprocessing stage, and the sampling rate is adjusted according to the distribution characteristics of a human face data set and the weight of sensitive information. And the model performance is maximized on the premise of ensuring user privacy. In order to obtain a better privacy protection effect, an entropy adjustment differential privacy mechanism is designed, and Gaussian noise of different degrees is adaptively added according to an information entropy value of face data after a face data set is dynamically and adaptively sampled, so that refined privacy protection of face sensitive data under differential privacy constraint is realized. In a model training stage, an elastic anti-interference training method is provided, diversified adversarial samples are dynamically generated in different stages of model training, and the over-fitting risk of the model is reduced while the robustness of the model to various types of adversarial attacks is enhanced.

Description

Technical Field

[0001] The present invention belongs to the field of information processing, and in particular relates to a method for protecting sensitive face data based on a flexible anti-interference privacy protection mechanism. Background Art

[0002] Compared with other types of data, sensitive facial data contains rich personal information, such as identity, age, gender, etc., and can even reflect a person's emotions and health status through facial expressions and subtle features. However, with the popularization and application of facial recognition technology, the risk of personal privacy leakage is also increasing. As a highly sensitive biometric information, facial data, once leaked or abused, will pose a serious threat to personal privacy, security and interests.

[0003] Some existing studies use data blurring technology to ensure the security of sensitive facial data. This technology makes facial features difficult to identify by reducing image resolution, adding random noise, or applying filters, so as to achieve the purpose of privacy protection. Although this method is simple and easy to implement, the blurred data still has the risk of being restored when facing advanced reconstruction algorithms, which may lead to the failure of privacy protection.

[0004] In addition, some studies have used generative adversarial network technology (GANs) to generate images that are similar in appearance to the original images but do not contain real identity information to further enhance user privacy protection. This method can effectively conceal the user's real identity information, however, the model is difficult to train, and the generated images may still be restored or recognized when subjected to specific attack methods.

[0005] In summary, although existing methods have made some progress in protecting sensitive facial data, there are still some problems. First, in terms of privacy protection, existing methods are easily cracked by attacks such as reverse engineering. Especially with the continuous advancement of image reconstruction algorithms and the increased risk of side channel attacks, the privacy security of data is still threatened. Secondly, most current technologies focus on how to hide or encrypt the data itself. However, when attackers use the generation of adversarial samples to mislead the model in order to achieve the purpose of data theft or model destruction, existing methods are often difficult to defend effectively. Summary of the invention

[0006] In order to solve the above problems, the present invention proposes a face sensitive data protection method based on elastic anti-disturbance privacy protection mechanism, which realizes adaptive sampling and dynamic noise addition by evaluating the information entropy of face image data, and combines the adversarial samples generated by composite adversarial attacks to perform model training, so as to realize the privacy protection of face sensitive data in the field of face recognition. Specifically, in the data preprocessing stage, a dynamic adaptive sampling strategy based on information entropy is proposed, and the sampling rate is adjusted according to the distribution characteristics of the face data set and the weight of sensitive information, so as to maximize the model performance under the premise of privacy protection. In order to achieve better privacy protection effect, an entropy-adjusted differential privacy mechanism is designed. After dynamically adaptive sampling of the face data set, different degrees of Gaussian noise are adaptively added according to the information entropy size of the face data, so as to realize the refined privacy protection of face sensitive data under differential privacy constraints. In the model training stage, a flexible anti-disturbance training method is proposed, which dynamically generates diversified adversarial samples at different stages of model training, while enhancing the robustness of the model to various types of adversarial attacks and reducing the overfitting risk of the model. In addition, in order to enhance the network's ability in feature extraction and improve the generalization performance of the model, the present invention proposes a dynamic entropy-aware privacy adversarial model. By introducing dynamic convolution kernels in the ResNet18 network, the convolution operation can be adaptively adjusted according to the characteristics of the input data, thereby achieving more flexible and efficient feature learning. Finally, in order to prove the effectiveness of the present invention, experiments were conducted in the gender classification task based on face data.

[0007] The technical solution of the present invention is as follows:

[0008] A method for protecting sensitive face data based on a flexible anti-interference privacy protection mechanism comprises the following steps:

[0009] Step 1: Obtain the original face image dataset and preprocess it, divide the preprocessed dataset into a training set and a test set in proportion, calculate the information entropy of each image, dynamically and adaptively sample the dataset according to the information entropy, and obtain an updated training set;

[0010] Step 2: Allocate a privacy budget for face image data, use the entropy-modulated differential privacy protection mechanism to add Gaussian noise to each image in the first updated training set, and obtain the second updated training set;

[0011] Step 3: Construct a dynamic convolution layer and replace the standard convolution layer in the original model network ResNet18 with the constructed dynamic convolution layer to form a dynamic entropy-aware privacy adversarial model.

[0012] Step 4: Use the twice updated training set to pre-train the model;

[0013] Step 5: Adopt the elastic anti-disturbance training method to dynamically generate diverse adversarial samples at different stages of the dynamic entropy-aware privacy adversarial model training. Mix the normal sample set of each round in the second-updated training set with the generated adversarial sample set in proportion and then conduct model training.

[0014] Step 6: Continue a new round of dynamic entropy-aware privacy adversarial model training. In each round of training, dynamic adaptive sampling and face image data noise processing are performed based on information entropy, and adversarial samples and ordinary samples are generated again to form a mixed training set for model training.

[0015] Furthermore, the specific process of step 1 is as follows:

[0016] Step 1.1: The original face image dataset obtained is D = {X1, X2, ..., X i ,…,X N}, N is the total number of images, X i is the i-th image; each image is an m×n×3 three-dimensional matrix, where m and m represent the height and width of the image respectively, and 3 represents the dimension, corresponding to the three RGB color channels;

[0017] Step 1.2: Perform Z-score normalization on the three RGB color channels of each image; divide the data set after normalization preprocessing into a training set and a test set in proportion; the calculation formula for Z-score normalization is as follows:

[0018]

[0019]

[0020]

[0021] Among them, x′ i (a, b, c) is x i (a, b, c) Normalized pixel values; x i (a, b, c) represents the pixel value of the i-th image at position (a, b) on color channel c; a is the row index of the image in the spatial dimension; b is the column index of the image in the spatial dimension; color channel c∈{R1, R2, R3}, R1, R2, R3 correspond to the RGB three color channels of the color image respectively; μ c represents the mean value of all pixels on color channel c; σ c Represents the standard deviation on color channel c;

[0022] Step 1.3, calculate the information entropy of the image based on the distribution of pixel values;

[0023] Step 1.4: Perform dynamic adaptive sampling of face images in the training set based on information entropy to obtain an updated training set.

[0024] Furthermore, the specific process of step 1.3 is as follows:

[0025] Step 1.3.1, count the distribution probability of each image pixel value, assuming a pixel value k in the range [0, K], K is the maximum value of the pixel value k; the calculation formula of the distribution probability is as follows:

[0026]

[0027] Among them, p i,k，c is the distribution probability of pixel value k in color channel c of the i-th image; |{(a, b)|x i (a, b, c) = k}| represents the i-th image X i The number of pixels with pixel value k in color channel c;

[0028] Step 1.3.2: Calculate the information entropy of each image in each color channel. The formula is:

[0029]

[0030] Among them, H(X i , c) is the i-th image X i Information entropy on color channel c;

[0031] Step 1.3.3: Average the information entropy of each image in each color channel to obtain the final total information entropy of each image. The formula is:

[0032]

[0033] Among them, H total (X i ) is the i-th image X i The final total information entropy.

[0034] Furthermore, the specific process of step 1.4 is as follows:

[0035] Step 1.4.1, based on the information entropy of each image, dynamically assign a sampling probability to each image; the sampling probability assignment formula is as follows:

[0036]

[0037] in, is the sampling probability of the i-th image; φ≥1 is a tuning parameter that controls the influence of entropy value on the sampling probability; j is the image number different from i; X j is the jth image;

[0038] Step 1.4.2: Perform random sampling according to the sampling probability of the image. First, generate a random number for each image, compare the random number with the sampling probability, and update the training set. Let the random number generated for the i-th image be r i ∈[0,1], if Then choose to add the i-th image to the updated training set D′ train ; Otherwise, it is not selected; Finally, an updated training set D′ is obtained train ,

[0039] Furthermore, the specific process of step 2 is as follows:

[0040] Step 2.1: Allocate a privacy budget to each image using the following formula:

[0041]

[0042] Among them, ∈ i is the privacy budget of the i-th image; ∈0 is the basic privacy budget; H max is the maximum entropy value of the samples in the data set; γ is the smoothing constant;

[0043] After allocating the privacy budget to each image, the privacy budget of each image is added together to ensure that the sum of the privacy budgets of all samples does not exceed the total privacy budget ∈ total ; If the total privacy budget is exceeded, the privacy budget of all images needs to be scaled, and the final privacy budget for each image is as follows:

[0044]

[0045] in, is the entropy adjustment privacy budget of the i-th image, i.e., the final privacy budget; ∈ j is the privacy budget of the jth image; M is the updated training set D′ train The total number of images in ;

[0046] Step 2.2: After allocating the privacy budget, use the entropy-modulated differential privacy protection mechanism to add Gaussian noise to each image in the updated training set. The specific process is as follows:

[0047] Step 2.2.1. Calculate the standard deviation of the Gaussian noise to be added to each image based on the final privacy budget and global sensitivity of each image in the training set. The formula is as follows:

[0048]

[0049] in, is the standard deviation of the Gaussian noise of the i-th image; Δf is the global sensitivity; δ is the failure probability parameter in the differential privacy mechanism;

[0050] Step 2.2.2, after obtaining the standard deviation of each image, generate Gaussian noise for each image. Gaussian noise follows a normal distribution:

[0051]

[0052] Among them, g i is the Gaussian noise of the i-th image; is the normal distribution function;

[0053] Step 2.2.3, add Gaussian noise to the face image to obtain the noisy image. The formula is as follows:

[0054] X″ i =X′ i +g i (12);

[0055] Among them, X″ i is the i-th image after adding noise; X′ i is the i-th image to which the standard deviation is added;

[0056] Step 2.2.4: All the noisy face images constitute the second updated training set D″ train ={X″1,X″2,...,X″ i , ..., X″ M}; Since the second update only adds noise disturbance to the images in the data set and does not change the number of data sets, the training set D″ of the second update train The total number of images in is still M.

[0057] Furthermore, in step 3, the dynamic convolution layer uses the He initialization method to initialize the basic convolution kernel weights, and adaptively allocates the convolution kernel weights according to the input image data; the specific working process of the dynamic convolution layer is as follows:

[0058] Step 3.1: Assume that the dynamic convolution layer has G basic convolution kernels, each of which has a size of h×h and a number of input feature channels of C. in , the number of channels of the output feature is C out ; The dynamic convolution layer uses the He initialization method to initialize the basic convolution kernel weights of the dynamic convolution layer. The specific initialization formula is as follows:

[0059]

[0060] in, represents the basic convolution kernel weight between the d-th input channel and the e-th output channel in the l-th layer; is the number of channels of the input features in the lth layer; h(l).h(l) represents the size of the basic convolution kernel in the lth layer. It is a common number when calculated here, so it is represented by dot multiplication;

[0061] Step 3.2, a lightweight sub-network is designed in the dynamic convolution layer; the sub-network consists of a global pooling layer, two fully connected layers and an activation function. The global information of the input features is extracted through the global pooling layer, and the first fully connected layer performs linear transformation. The second fully connected layer generates dynamic weight coefficients and normalizes them through Softmax. Finally, an adaptive weight coefficient is assigned to each convolution kernel. The sub-network is based on D″ train The final weight coefficient set α = [α1, α2, ..., α s , ..., α G ], α G is the final weight coefficient of the Gth basic convolution kernel, α s is the final weight coefficient of the sth basic convolution kernel; the specific working process of the subnetwork is:

[0062] Step 3.2.1: The noisy face image first passes through the global average pooling layer to obtain the pooled features. The formula is as follows:

[0063]

[0064] in, is the i-th image after adding noise X″ i Features after pooling; GAP(·) is the global average pooling layer; m and n are the height and width of the image, respectively; a is the row index of the image in the spatial dimension; b is the column index of the image in the spatial dimension; x″ i (a, b) represents the pixel value of the i-th image at position (a, b) after adding noise;

[0065] Step 3.2.2: The pooled features are linearly transformed through the first fully connected layer to obtain hidden features. The formula is as follows:

[0066]

[0067] Among them, z is the hidden feature of the pooled feature; is the weight of the first fully connected layer; b1 is the bias of the first fully connected layer; X gp Represents the features after image pooling;

[0068] Step 3.2.3: Use the ReLU activation function to perform nonlinear mapping on the hidden features to obtain the mapped hidden features. The formula is as follows:

[0069] z′=ReLU(z)(16);

[0070] Where z′ is the mapping hidden feature; ReLU(·) is the ReLU activation function;

[0071] Step 3.2.4, map the hidden features through the second fully connected layer to generate the initial weight coefficient β of the basic convolution kernel, and use the Softmax function to normalize it to get the final weight coefficient. The formula is as follows:

[0072]

[0073] Among them, β s is the initial weight coefficient of the sth basic convolution kernel; α s is the final weight coefficient of the sth basic convolution kernel; is the weight of the second fully connected layer; b2 is the bias of the second fully connected layer; Softmax(·) is the Softmax function; ep is the serial number of the basic convolution kernel; exp(·) is the exponential function with e as the base; z′ s is the mapping hidden feature of the sth basic convolution kernel; β ep is the initial weight coefficient of the epth basic convolution kernel;

[0074] Step 3.3: Dynamically adjust the final weight coefficient of the basic convolution kernel by weighted summation. Weighted summation of multiple basic convolution kernels according to the final weight coefficients generated by the lightweight subnetwork above is performed to obtain a dynamic convolution kernel. The formula is as follows:

[0075]

[0076] Among them, W dyn is the dynamic convolution kernel; W s is the sth basic convolution kernel.

[0077] Furthermore, in the pre-training process of step 4, the noisy image X″ is subjected to convolution and pooling operations to generate a feature map, and then the global features are extracted through the global pooling layer and the fully connected layer. Finally, the model parameters are adjusted by optimizing the cross entropy loss function of the common sample; the formula of the cross entropy loss function of the common sample is as follows:

[0078]

[0079] Among them, L CE is the cross entropy loss function value of the common sample; M′ is the number of images in each batch; y iis the true label of the i-th image; is the predicted probability of the ith image.

[0080] Furthermore, the specific process of step 5 is as follows:

[0081] Step 5.1: Initialize the perturbation amplitude, step size, and first round of adversarial samples, using D″ train The first round of adversarial samples are initialized with the noisy face image. The formula is as follows:

[0082]

[0083] in, is the first round of adversarial samples; X″ is the face image after adding noise;

[0084] Step 5.2: Define the loss function of the adversarial sample. The cross entropy loss function is also used. The specific formula is as follows:

[0085]

[0086] Among them, L adv is the cross entropy loss function value of the adversarial sample; M″ is the number of adversarial samples in each round; represents the model's predicted probability of the adversarial sample corresponding to the i-th image; y i represents the true label of the i-th image;

[0087] Step 5.3: Adaptively adjust the step size and disturbance amplitude according to the feedback of the model to ensure that the step size and disturbance amplitude are the optimal values ​​after dynamic adjustment; the dynamic adjustment formula is as follows:

[0088]

[0089] Among them, c i is the perturbation amplitude of the adversarial sample corresponding to the i-th image, which is used to control the perturbation amount applied to the sample during the generation of the adversarial sample; C base is the initial disturbance amplitude; λ′ is the disturbance amplitude adjustment coefficient, which is used to control the adjustment intensity of the disturbance amplitude; is the average loss value of the current batch of adversarial samples, and the formula is as follows:

[0090]

[0091] Among them, N adv is the number of adversarial samples in the current batch;

[0092] According to H total (X i ) Adaptively adjust the generation iteration step of the adversarial sample, the formula is as follows:

[0093] η i =η base ·(1+λ″·H total (X i )) (25);

[0094] Among them, η i is the step size of the adversarial sample corresponding to the i-th image; η base is the initial step size; λ″ is the step size adjustment coefficient, which is used to control the influence of information entropy on step size adjustment;

[0095] Step 5.4: Dynamically select the attack strategy for generating adversarial samples based on the current performance of the model.

[0096] Furthermore, the specific process of step 5.4 is as follows:

[0097] Step 5.4.1. Calculate the loss change rate using the following formula:

[0098]

[0099] in, is the rate of change of the cross entropy loss of the adversarial sample in the tth round; are the cross entropy loss function values ​​of the adversarial samples in the tth and t-1th rounds respectively; u is a constant;

[0100] Step 5.4.2: Dynamically switch the attack strategy for generating adversarial samples according to the cross entropy loss change rate of the model; pre-set the first threshold τ fgsm and the second threshold τ pgd , used to determine which attack strategy to choose; it includes three attack strategies, namely the fast gradient sign method, the projected gradient descent method, and the deep deception method; the specific judgment rules are as follows:

[0101] In the early stages of training, if Use the fast gradient sign method to generate adversarial samples. The formula is as follows:

[0102]

[0103] in, is the adversarial sample corresponding to the i-th image in round t; is the i-th image after the t-th round of noise addition; is the step size of the adversarial sample corresponding to the i-th image in the t-th round; yes The loss gradient of ; sign is the sign of the gradient; represents the model's predicted probability of the adversarial sample corresponding to the i-th image in the t-th round; represents the true label of the i-th image in the t-th round;

[0104] In the middle of training, if Use the projected gradient descent method to generate adversarial samples and gradually optimize the perturbation. The formula is as follows:

[0105]

[0106] in, is the adversarial sample corresponding to the i-th image in the t+1th round; It is a projection operation, which projects the updated adversarial sample back into the range to ensure that the adversarial sample is within the restricted area; is the i-th image after the t-th round of noise addition; So A spherical area with the center at and the radius at the disturbance amplitude c; yes The loss gradient of

[0107] In the later stages of training, if Use the deep deception method to generate adversarial samples. The formula is as follows:

[0108]

[0109] Among them, f df (·) is the classification output score of the current sample; It is a sample The gradient of the classification output score; is the square of the L2 norm of the gradient;

[0110] Step 5.5: D″ train The common sample set D in each round normal With adversarial sample set D adv After mixing in proportion, the training is performed with the following formula:

[0111] D train = ζ·D normal +(1-ζ)·D adv (30);

[0112] Among them, D train is the mixed training set; ζ is the mixing ratio;

[0113] The total loss function L of the model total as follows:

[0114]

[0115] in, is the mixing ratio;

[0116] After training, the dynamic entropy-aware privacy adversarial model is evaluated using the test set to measure the performance of the model.

[0117] Furthermore, the specific process of step 6 is as follows:

[0118] Step 6.1: Based on the feedback of each round of dynamic entropy-aware privacy adversarial model training results, first dynamically and adaptively sample the data based on information entropy; then, add adaptive Gaussian noise to the sampled data based on the information entropy of the sampled data;

[0119] Step 6.2: Based on the performance of the current dynamic entropy-aware privacy adversarial model, generate adversarial samples based on the noisy data; then, combine these adversarial samples with the noisy normal samples in proportion to form a mixed training set;

[0120] Step 6.3: Use the mixed training set to train the dynamic entropy-aware privacy adversarial model;

[0121] Step 6.4: After each round of iteration, fine-tune the sampling, noise addition, and adversarial training process based on the training results of the dynamic entropy-aware privacy adversarial model to gradually improve the performance of the model until the preset gender recognition accuracy is reached or the maximum number of training rounds is met under the condition of differential privacy protection.

[0122] The beneficial technical effects brought by the present invention are as follows:

[0123] 1. By using the elastic anti-disturbance training method, the dynamic entropy-aware privacy adversarial model can conduct adversarial training in a targeted manner, thereby gradually enhancing its defense capabilities against adversarial attacks. This method effectively avoids the limitations of fixed perturbations and ensures that the model can remain stable in the face of multiple adversarial attacks. In addition, the introduction of dynamic convolution kernels enables the model to have better expressiveness when processing different inputs, and can automatically adjust under adversarial samples or noise interference, thereby improving its robustness.

[0124] 2. Reasonably allocate the privacy budget through the information entropy of sensitive face data and implement an adaptive differential privacy noise addition strategy for sensitive face data, which can effectively prevent the leakage of face data privacy. By flexibly adjusting the intensity of noise in an adaptive manner, it can protect data privacy while minimizing the negative impact on model performance and ensuring that the model maintains a high level of accuracy.

[0125] 3. Dynamically and adaptively sampling the face dataset through the information entropy of sensitive face data can effectively measure the importance of each sample in the dataset, thereby building a diverse and representative training dataset. This method can not only reduce redundant data and increase the speed of model training, but also reduce the risk of sensitive information leakage and effectively protect sensitive face data. BRIEF DESCRIPTION OF THE DRAWINGS

[0126] Figure 1 This is a flow chart of the face sensitive data protection method based on the elastic anti-interference privacy protection mechanism of the present invention.

[0127] Figure 2 This is a comparison chart of the loss value changes of the present invention and two baseline methods in the experiment of the present invention.

[0128] Figure 3 This is a comparison chart of the accuracy changes of the present invention and two baseline methods in the experiment of the present invention.

[0129] Figure 4 This is a comparison chart of the recall rate changes of the present invention and two baseline methods in the experiment of the present invention.

[0130] Figure 5 This is a comparison chart of the F1 score changes of the present invention and two baseline methods in the experiment of the present invention. DETAILED DESCRIPTION

[0131] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0132] like Figure 1 As shown, the present invention proposes a method for protecting sensitive face data based on an elastic anti-disturbance privacy protection mechanism. The elastic anti-disturbance privacy protection mechanism is a mechanism for protecting sensitive face data privacy by dynamically adjusting the step size and disturbance size through elastic anti-disturbance training to prevent overfitting or underfitting caused by too small or too large disturbances. The method specifically includes the following steps:

[0133] Step 1: Obtain the original face image dataset and preprocess it, divide the preprocessed dataset into a training set and a test set in proportion, calculate the information entropy of each image, dynamically and adaptively sample the dataset based on the information entropy, and obtain an updated training set; In the data preprocessing stage, the present invention proposes a dynamic adaptive sampling strategy based on information entropy, adjusts the sampling rate according to the distribution characteristics of the face dataset and the weight of sensitive information, and maximizes the model performance under the premise of privacy protection. The specific process is as follows:

[0134] Step 1.1: The original face image dataset obtained is D = {X1, X2, ..., X i , ..., X N}, N is the total number of images, X i is the i-th image, and the true label corresponding to the i-th image is yi. Each image is a three-dimensional matrix of m×n×3, where m and n represent the height and width of the image respectively, and 3 represents the dimension, corresponding to the three color channels of RGB.

[0135] Step 1.2: Perform Z-score normalization on the three RGB color channels of each image to eliminate the deviation caused by brightness or contrast between different images. The standardized preprocessed data set is divided into a training set and a test set in a ratio of 8:2. The calculation formula for Z-score normalization is as follows:

[0136]

[0137] Among them, x′ i (a, b, c) is x i (a, b, c) Normalized pixel values; x i (a, b, c) represents the pixel value of the i-th image at position (a, b) on color channel c; a is the row index of the image in the spatial dimension; b is the column index of the image in the spatial dimension; color channel c∈{R1, R2, R3}, R1, R2, R3 correspond to the RGB three color channels of the color image respectively; μ c represents the mean value of all pixels on color channel c; σ c Represents the standard deviation on color channel c.

[0138] Step 1.3: Calculate the information entropy based on the distribution of image pixel values; the specific process is:

[0139] Step 1.3.1. First, count the distribution probability of each image pixel value. Assume a pixel value k in the range [0, K], where K is the maximum value of the pixel value k. The calculation formula for the distribution probability is as follows:

[0140]

[0141] Among them, p i,k,c is the distribution probability of pixel value k in color channel c of the i-th image; |{(a, b)|x′ i (a, b, c) = k}| represents the i-th image X i The number of pixels with pixel value k in color channel c.

[0142] Step 1.3.2, then calculate the information entropy of each image on each color channel, the formula is:

[0143]

[0144] Among them, H(X i , c) is the i-th image X i Information entropy on color channel c;

[0145] Step 1.3.3. Finally, the information entropy of each image in each color channel is averaged and weighted to obtain the final total information entropy of each image. The formula is:

[0146]

[0147] Among them, H total (X i ) is the i-th image X i The final total information entropy;

[0148] Step 1.4: Perform dynamic adaptive sampling of face images in the training set based on information entropy to obtain an updated training set. The specific process is:

[0149] Step 1.4.1. Based on the information entropy of each image, dynamically assign sampling probability to each image. The sampling probability is adjusted according to the change of entropy value. The higher the entropy value, the higher the sampling frequency. The allocation formula of sampling probability is as follows:

[0150]

[0151] in, is the sampling probability of the i-th image; φ≥1 is a tuning parameter that controls the degree of influence of entropy value on sampling probability. A larger φ value will make images with high entropy values ​​obtain higher sampling weights; j is the image number different from i; X j is the jth image.

[0152] Step 1.4.2: Perform random sampling based on the sampling probability of face images. First, generate a random number for each image, compare the random number with the sampling probability, and update the training set. Let the random number generated for the i-th image be r i ∈[0, 1], if Then choose to add the i-th image to the updated training set D′ train ; Otherwise, it is not selected. Finally, we get an updated training set D′ train ,

[0153] Step 2: Allocate a privacy budget for face image data, use the entropy-adjusted differential privacy protection mechanism to add Gaussian noise to each image in the once-updated training set, and obtain a second-updated training set; in order to achieve better privacy protection, the present invention designs an entropy-adjusted differential privacy protection mechanism, which dynamically and adaptively samples the face data set and adaptively adds different degrees of Gaussian noise according to the information entropy of the face data, thereby achieving refined privacy protection of sensitive face data under differential privacy constraints. The specific process is as follows:

[0154] Step 2.1. The present invention uses the information entropy of the face image to dynamically adjust the privacy budget. Images with high information entropy contain more information, so they usually require stronger privacy protection, while images with low information entropy are relatively simple, so their privacy protection is relatively less. The allocation formula for the privacy budget of each image is as follows:

[0155]

[0156] Among them, ∈ i is the privacy budget of the i-th image; ∈0 is the basic privacy budget, defined as the maximum allowed privacy budget; H max is the maximum entropy value of the samples in the data set, that is, H max =max{H total (X i ));γ is a smoothing constant to avoid the denominator being zero;

[0157] After allocating the privacy budget for each face image, the privacy budget of each image is added together to ensure that the sum of the privacy budgets of all samples does not exceed the total privacy budget ∈ total If the total privacy budget is exceeded, the privacy budget of all images needs to be scaled, and the final privacy budget for each image is as follows:

[0158]

[0159] in, is the entropy adjustment privacy budget of the i-th image, i.e., the final privacy budget; ∈ j is the privacy budget of the jth image; M is the updated training set D′ train The total number of images in .

[0160] Step 2.2: After allocating the privacy budget, use the entropy-modulated differential privacy protection mechanism to add Gaussian noise to each image in the updated training set. The specific process is:

[0161] Step 2.2.1. Calculate the standard deviation of the Gaussian noise to be added to each image based on the final privacy budget and global sensitivity of each image in the training set. The formula is as follows:

[0162]

[0163] in, is the standard deviation of the Gaussian noise of the i-th image; Δf is the global sensitivity, and the value of the global sensitivity in the present invention is 2; δ is the failure probability parameter in the differential privacy mechanism, and δ is 10 in the present invention. -6 .

[0164] Step 2.2.2. After obtaining the standard deviation of each image, generate Gaussian noise that conforms to the following distribution:

[0165]

[0166] Among them, g i is the Gaussian noise of the i-th image, and its dimension is consistent with that of the face image; is the normal distribution function;

[0167] Step 2.2.3, add Gaussian noise to the face image to obtain the noisy image. The formula is as follows:

[0168] X″ i =X′ i +g i (12);

[0169] Among them, X″ i is the i-th image after adding noise; X′ i is the i-th image to which the standard deviation is added;

[0170] Step 2.2.4: All the noisy face images constitute the second updated training set D″ train ={X″1,X″2,…,X″ i ,…,X″ M Since only noise disturbance is added to the images in the dataset during the second update, the number of datasets is not changed, so the training set D″ of the second update train The total number of images in is still M.

[0171] Step 3: Construct a dynamic entropy-aware privacy adversarial model. First, construct a dynamic convolution layer, and then replace the standard convolution layer in the original model network ResNet18 with the constructed dynamic convolution layer to form a new model. The new model is the dynamic entropy-aware privacy adversarial model. The dynamic convolution layer enhances the flexibility and adaptability of the model network in feature extraction, which helps the model learn the differences in characteristics of different genders. The dynamic convolution layer uses the He initialization method to initialize the basic convolution kernel weights, and adaptively allocates the convolution kernel weights according to the input face image data. The specific working process of the dynamic convolution layer is as follows:

[0172] Step 3.1: Assume that the dynamic convolution layer has G basic convolution kernels, each of which has a size of h×h and a number of input feature channels of C. in , the number of channels of the output feature is C out The basic convolution kernel weight is defined as In order to ensure the numerical stability of network training, the present invention uses the He initialization method to initialize the basic convolution kernel weights of the dynamic convolution layer. The specific initialization formula is as follows:

[0173]

[0174] in, represents the basic convolution kernel weight between the d-th input channel and the e-th output channel in the l-th layer; is the number of channels of input features in the lth layer; h (l) .h (l) Represents the size of the basic convolution kernel in the lth layer. It is a common number when calculated here, so it is represented by dot multiplication.

[0175] Step 3.2, in order to make the weight of the basic convolution kernel adaptively adjusted according to the input data, the present invention designs a lightweight sub-network in the dynamic convolution layer. The sub-network consists of a global pooling layer, two fully connected layers and an activation function. The global information of the input features is extracted through the global pooling layer, and the linear transformation is performed through the first fully connected layer. The second fully connected layer generates a dynamic weight coefficient, and is normalized by Softmax, and finally an adaptive weight coefficient is assigned to each convolution kernel.

[0176] The subnetwork is based on D″ train The final weight coefficient set α = [α1, α2, ..., α s , ..., α G ], α G is the final weight coefficient of the Gth basic convolution kernel, α s is the final weight coefficient of the sth basic convolution kernel. The following is the specific working process of the sub-network:

[0177] Step 3.2.1: The noisy face image first passes through the global average pooling layer to obtain the pooled feature X gp , the formula is as follows:

[0178]

[0179] in, is the i-th image after adding noise X″ i Features after pooling; GAP(·) is the global average pooling operation; m and n are the height and width of the image respectively; a is the row index of the image in the spatial dimension; b is the column index of the image in the spatial dimension. x″ i (a, b) represents the pixel value of the i-th image at position (a, b) after noise addition.

[0180] Step 3.2.2: The pooled features are linearly transformed through the first fully connected layer to obtain hidden features. The formula is as follows:

[0181]

[0182] Among them, z is the hidden feature of the pooled feature; is the weight of the first fully connected layer; b1 is the bias of the first fully connected layer; X gp Represents the features after image pooling;

[0183] Step 3.2.3: Use the ReLU activation function to perform nonlinear mapping on the hidden features to obtain the mapped hidden features. The formula is as follows:

[0184] z′=ReLU(z)(16);

[0185] Where z′ is the mapping hidden feature; ReLU(·) is the ReLU activation function;

[0186] Step 3.2.4: Map the hidden features through the second fully connected layer to generate the initial weight coefficient β of the basic convolution kernel s , and use the Softmax function to normalize to get the final weight coefficient. The formula is as follows:

[0187]

[0188] Among them, β s is the initial weight coefficient of the sth basic convolution kernel; α s is the final weight coefficient of the sth basic convolution kernel; is the weight of the second fully connected layer; b2 is the bias of the second fully connected layer; Softmax(·) is the Softmax function; ep is the serial number of the basic convolution kernel; exp(·) is the exponential function with e as the base; z′ s is the hidden feature of the e-map of the s-th basic convolution kernel; β ep is the initial weight coefficient of the epth basic convolution kernel.

[0189] Step 3.3: Dynamically adjust the final weight coefficient of the basic convolution kernel by weighted summation. Weighted summation of multiple basic convolution kernels according to the final weight coefficients generated by the lightweight subnetwork above is performed to obtain a dynamic convolution kernel. The formula is as follows:

[0190]

[0191] Among them, W dyn is the dynamic convolution kernel; W s is the sth basic convolution kernel.

[0192] Step 4: In order to improve the generalization ability of the dynamic entropy-aware privacy adversarial model in gender classification tasks and to provide a reference for subsequent adversarial samples and D″ train The samples in (in order to facilitate the comparison and distinction with the adversarial samples, D″ trainThe samples in the training set are called normal samples) and the training set D″ is updated twice after adding noise. train Perform model pre-training. The main goal of pre-training is to enable the model to fully learn the basic features of face image data on standard samples. In this process, the noisy image X″ undergoes convolution, pooling and other operations to generate a feature map X tr , then extract global features through the global pooling layer and the fully connected layer, and finally adjust the model parameters by optimizing the cross entropy loss function of common samples, so that the classification accuracy of the model on common samples is improved. The formula of the cross entropy loss function of common samples is as follows:

[0193]

[0194] Among them, L CE is the cross entropy loss function value of the common sample; M′ is the number of images in each batch; y i is the true label of the i-th image; is the predicted probability of the ith image.

[0195] Step 5: In the training phase of the dynamic entropy-aware privacy adversarial model, the present invention proposes a flexible anti-disturbance training method, which dynamically generates diversified adversarial samples at different stages of model training, while enhancing the robustness of the model to various types of adversarial attacks and reducing the risk of overfitting of the model. After completing the pre-training of the model, adversarial samples are dynamically generated according to the performance of the model, and the normal sample set and the adversarial sample set of each round in the second updated training set are mixed in a ratio of 7:3 for model training; the specific process is as follows:

[0196] Step 5.1: Initialize the disturbance amplitude c base , step length η base And the first round of adversarial samples. In the present invention, the initial value of the perturbation amplitude is set to 0.03, the initial value of the step length is set to 0.015, and D″ train The first round of adversarial samples are initialized with the noisy face image. The formula is as follows:

[0197]

[0198] in, is the first round of adversarial samples; X″ is the face image after adding noise;

[0199] Step 5.2: Define the loss function of the adversarial sample. The cross entropy loss function is also used. The specific formula is as follows:

[0200]

[0201] Among them, L advis the cross entropy loss function value of the adversarial sample; M″ is the number of adversarial samples in each round; represents the model's predicted probability of the adversarial sample corresponding to the i-th image; y i represents the true label of the i-th image.

[0202] Step 5.3, adaptively adjust the step size and perturbation amplitude according to the feedback of the model to ensure that these parameters are the optimal values ​​after dynamic adjustment. In the present invention, the cross entropy loss function value of the current adversarial sample of the model is used as feedback to adjust the perturbation amplitude. When the gradient of the model is large, the perturbation amplitude is appropriately increased, and when the gradient of the model is small, the perturbation amplitude is appropriately reduced. The dynamic adjustment formula is as follows:

[0203]

[0204] Among them, c i is the perturbation amplitude of the adversarial sample corresponding to the i-th image, which is used to control the perturbation amount applied to the sample during the generation of the adversarial sample; c base is the initial disturbance amplitude; λ′ is the disturbance amplitude adjustment coefficient, which is used to control the adjustment intensity of the disturbance amplitude. In the present invention, the value of λ′ is 0.1. is the average loss value of the current batch of adversarial samples, and the formula is as follows:

[0205]

[0206] Among them, N adv is the number of adversarial examples in the current batch.

[0207] According to H total (X i ) Adaptively adjust the generation iteration step of the adversarial sample, the formula is as follows:

[0208] η i =η base ·(1+λ″·H total (X i )) (25);

[0209] Among them, η i is the step size of the adversarial sample corresponding to the i-th image, which is used to control the size of the perturbation update during the adversarial sample generation process; η base is the initial step length; λ″ is the step length adjustment coefficient, which is used to control the influence of information entropy on the step length adjustment. In the present invention, the value of λ″ is 0.1.

[0210] Step 5.4: Dynamically select an attack strategy for generating adversarial samples based on the current performance of the model. In the present invention, the effectiveness of the current strategy is judged based on the changes in the loss function of the adversarial samples at different training stages. By monitoring the changes in the loss function of the adversarial samples, it is determined whether the current attack strategy has failed, thereby deciding whether to switch to a stronger attack strategy to ensure the effectiveness of the currently generated adversarial samples. The specific process is as follows:

[0211] Step 5.4.1. Calculate the loss change rate using the following formula:

[0212]

[0213] in, is the rate of change of the cross entropy loss of the adversarial sample in the tth round; are the cross entropy loss function values ​​of the adversarial samples in the tth and t-1th rounds respectively; u is a constant, which takes a smaller value to prevent the denominator from being zero.

[0214] Step 5.4.2: Dynamically switch the attack strategy for generating adversarial samples according to the cross entropy loss change rate of the model. In the present invention, the cross entropy loss change rate is calculated by five consecutive iterations to determine whether to switch the attack strategy. The first threshold τ is pre-set fgsm and the second threshold τ pgd , used to determine which attack strategy to choose; in the present invention, τ fgsm Set to 0.01, τ pgd Set to 0.001.

[0215] The present invention includes three attack strategies, which respectively use the Fast Gradient Sign Method (FGSM), the Projected Gradient Descent (PGD), and the DeepFool method; in the early stage of training, the model parameters have not yet converged, and FGSM can be used to quickly generate simple adversarial samples, so that the model can perceive and adapt to adversarial disturbances as early as possible. In the middle of training, the model begins to converge and the loss changes tend to be stable. At this time, PGD can be used to continuously optimize the adversarial disturbance to make it more aggressive, thereby further improving the robustness of the model. In the later stage of training, the model has tended to converge, and DeepFool can be used to effectively verify its resistance to extreme adversarial disturbances. The specific judgment rules are as follows:

[0216] In the early stages of training, the model loss varies greatly. Use the fast gradient symbol method to quickly generate adversarial samples. The formula is as follows:

[0217]

[0218] in, is the adversarial sample corresponding to the i-th image in round t; is the i-th image after the t-th round of noise addition; is the step size of the adversarial sample corresponding to the i-th image in the t-th round, which is used to control the update amplitude of each step; yes The loss gradient of ; sign is the sign of the gradient, which converts each element of the gradient vector into its sign, that is, if the gradient is positive, the sign is +1; if it is negative, the sign is -1; if it is zero, the sign is 0. represents the model's predicted probability of the adversarial sample corresponding to the i-th image in the t-th round; represents the true label of the i-th image in the t-th round.

[0219] In the middle of training, the model loss tends to stabilize but has not yet converged. If Use the projected gradient descent method to generate adversarial samples and gradually optimize the perturbation. The formula is as follows:

[0220]

[0221] in, is the adversarial sample corresponding to the i-th image in the t+1th round; It is a projection operation, which projects the updated adversarial sample back into the range to ensure that the adversarial sample is within the restricted area; is the i-th image after the t-th round of noise addition; So A spherical area with the center at and the radius at the disturbance amplitude c; yes The loss gradient.

[0222] In the later stages of training, the model loss tends to converge. If Use the deep deception method to generate stronger adversarial samples. The formula is as follows:

[0223]

[0224] Among them, f df (·) is the classification output score of the current sample. It is a sample Gradient of the classification output score. is the square of the L2 norm of the gradient.

[0225] Step 5.5: D″ train The common sample set D in each round normal With adversarial sample set D adv After mixing in a certain ratio, training is performed. The formula is as follows:

[0226] D train = ζ·D normal +(1-ζ)·D adv (30);

[0227] Among them, D train is the mixed training set; ζ is the mixing ratio, and ζ is 0.7 in the present invention.

[0228] The total loss function L of the model total as follows:

[0229]

[0230] in, is the mixing ratio, in the present invention It is 0.7.

[0231] After training, the dynamic entropy-aware privacy adversarial model is evaluated using a test set to measure the performance of the model and ensure its reliability in practical applications.

[0232] Step 6: Continue a new round of dynamic entropy-aware privacy adversarial model training. In each round of training, dynamic adaptive sampling and face image data noise processing are performed based on information entropy, and adversarial samples and ordinary samples are generated again to form a mixed training set for model training. The specific process is as follows:

[0233] Step 6.1: Based on the feedback of each round of dynamic entropy-aware privacy adversarial model training results, first dynamically and adaptively sample the data based on information entropy to filter out samples with high information content. Then, add adaptive Gaussian noise to the sampled data based on the information entropy of the sampled data to blur the details of sensitive facial features, thereby reducing the risk of individual privacy leakage.

[0234] Step 6.2: Based on the performance of the current dynamic entropy-aware privacy adversarial model, generate adversarial samples X based on the noisy data. adv Next, these adversarial samples are combined with ordinary samples after differential privacy noise processing in a certain proportion to form a mixed training set.

[0235] Step 6.3: Use the mixed training set to train the dynamic entropy-aware privacy adversarial model. Under the dual protection of differential privacy and adversarial samples, the robustness of the model under privacy noise and adversarial perturbations is improved, thereby improving the effect of protecting the privacy of face data.

[0236] Step 6.4: After each round of iteration, fine-tune the sampling, noise addition, and adversarial training process based on the training results of the dynamic entropy-aware privacy adversarial model to gradually improve the performance of the model until the preset gender recognition accuracy is reached or the maximum number of training rounds is met under the condition of differential privacy protection.

[0237] In order to prove the feasibility and rationality of the present invention in theory, the following proof and analysis are given.

[0238] (1) The present invention effectively ensures the privacy and security of data.

[0239] Analysis: Using information entropy to adjust the sampling probability can indirectly protect sensitive data. For samples with higher entropy values, it means that the model uncertainty is greater. Such samples are more critical to model training, so their sampling probability should be increased. For samples with lower entropy values, their sampling probability can be appropriately reduced. Assuming that some samples have lower entropy values, this means that the model is more confident about these samples and may have fully learned their features. At this time, by dynamically adjusting the sampling probability, the frequency of these samples in training can be reduced, thereby reducing the model's dependence on these samples and the potential risk of privacy leakage. The entropy-based sampling probability can be defined as: p s (X i )∝exp(ξH(X i )); where p s (X i ) is X i The sampling probability of ξ is the control parameter used to adjust the influence of entropy value on sampling probability. For samples with larger entropy value, the sampling probability will be greater, while for samples with smaller entropy value, the sampling probability will be reduced accordingly.

[0240] Assuming that the entropy value of sensitive data is low, according to the above sampling strategy, the sampling probability will be reduced, which can reduce the frequent sampling of sensitive data and thus reduce the risk of privacy leakage.

[0241] By introducing noise into the data and allocating privacy budgets in a personalized manner, the entropy-modulated differential privacy mechanism can provide stronger protection for sensitive facial data. Specifically, as the sensitivity based on information entropy increases, the privacy budget decreases accordingly, increasing the amplitude of the noise added to the data itself. This mechanism ensures that the model does not over-rely on or learn sensitive data during training, thereby effectively protecting data privacy.

[0242] (2) The present invention improves the robustness and stability of the model.

[0243] Analysis: Elastic anti-disturbance training dynamically adjusts the step size and disturbance size according to the performance of the model, effectively coping with gradient changes at different training stages. The elastic mechanism makes training more flexible and will not cause overfitting or underfitting due to too small or too large disturbances. In adversarial training, the loss function is usually unstable, and adaptive adjustment can stabilize the convergence of the loss by dynamically controlling the disturbance intensity.

[0244] In adversarial training, the loss function is usually unstable. As can be seen from formula (23), the adjustment of the perturbation size depends on the current loss change. We stabilize the convergence of the loss by dynamically controlling the perturbation intensity, thus avoiding the convergence difficulties that may occur in adversarial training. At the same time, adaptive adversarial training makes dynamic adjustments to keep the model within a reasonable perturbation range, thereby effectively preventing gradient vanishing or gradient exploding and ensuring the stability of the gradient.

[0245] By adopting three different adversarial sample generation strategies to simulate adversarial attacks of different intensities and complexities, the model can effectively cope with adversarial attacks of different types and intensities, thus improving its robustness and adversarial capabilities in practical applications.

[0246] In order to prove the feasibility and superiority of the present invention in practice, the following comparative experiment is given.

[0247] The present invention uses the Gender Classification Dataset, a face dataset on the Kaggle website, as an example to carry out experiments. The dataset is obtained by the uploader screening and processing photos in the CelebA dataset. Therefore, the photo content only contains a face, and there is very little irrelevant information such as the background and other body parts. This avoids the irrelevant information in the photo interfering with the model classification.

[0248] The present invention uses a dynamic entropy-aware privacy adversarial model for training, which can effectively identify and classify data sets; the total loss function L is used total As a loss function to measure the difference between the predicted value and the label; set multiple evaluation indicators to measure the performance of the model, including accuracy, precision, recall and F1 score, the formulas are:

[0249]

[0250] Among them, Acc represents the accuracy index; Prec represents the accuracy index; Recall represents the recall index; F1 represents the F1 score index; TP, TN, FP, and FN respectively represent the number of true positives, true negatives, false positives, and false negatives during the model training process.

[0251] All baseline solutions used in the present invention are sensitive data protection systems based on dynamic entropy-aware privacy adversarial models. The functional comparison table of the present invention solution and the baseline solution is shown in Table 1.

[0252] Table 1 Function comparison table of the solution of the present invention and the baseline solution

[0253]

[0254]

[0255] In the experiment, the optimizer used was Adam, the learning rate used was 0.001, the iteration round was 60 rounds, and the training hyperparameters remained the same. The loss value, accuracy, recall rate, and F1 score change comparison charts in the experiment are as follows: Figure 2 , Figure 3 , Figure 4 , Figure 5 shown.

[0256] The experimental results show that the scheme of the present invention can protect privacy while the performance of the model is not lower than that of the baseline method in all aspects. This experiment proves the feasibility of the scheme of the present invention in practice.

[0257] Of course, the above description is not a limitation of the present invention, and the present invention is not limited to the above examples. Changes, modifications, additions or substitutions made by technicians in this technical field within the essential scope of the present invention should also fall within the protection scope of the present invention.

Claims

1. A method for protecting sensitive face data based on a flexible anti-interference privacy protection mechanism, characterized in that: The steps include: Step 1: Obtain the original face image dataset and preprocess it, divide the preprocessed dataset into a training set and a test set in proportion, calculate the information entropy of each image, dynamically and adaptively sample the dataset according to the information entropy, and obtain an updated training set; Step 2: Allocate a privacy budget for face image data, use the entropy-modulated differential privacy protection mechanism to add Gaussian noise to each image in the first updated training set, and obtain the second updated training set; Step 3: Construct a dynamic convolution layer and replace the standard convolution layer in the original model network ResNet18 with the constructed dynamic convolution layer to form a dynamic entropy-aware privacy adversarial model. Step 4: Use the twice updated training set to pre-train the model; Step 5: Adopt the elastic anti-disturbance training method to dynamically generate diverse adversarial samples at different stages of the dynamic entropy-aware privacy adversarial model training. Mix the normal sample set of each round in the second-updated training set with the generated adversarial sample set in proportion and then conduct model training. Step 6: Continue a new round of dynamic entropy-aware privacy adversarial model training. In each round of training, dynamic adaptive sampling and face image data noise processing are performed based on information entropy, and adversarial samples and ordinary samples are generated again to form a mixed training set for model training.

2. According to claim 1, the method for protecting sensitive face data based on a flexible anti-interference privacy protection mechanism is characterized in that: The specific process of step 1 is as follows: Step 1.1: The original face image dataset obtained is D = {X1, X2, ..., X i ,…,X N }, N is the total number of images, X i is the i-th image; each image is a three-dimensional matrix of m×n×3, where m and n represent the height and width of the image respectively, and 3 represents the dimension, corresponding to the three color channels of RGB; Step 1.2: Perform Z-score normalization on the three RGB color channels of each image; divide the data set after normalization preprocessing into a training set and a test set in proportion; the calculation formula for Z-score normalization is as follows: Among them, x′ i (a,b,c) is x i (a, b, c) normalized pixel values; x i (a, b, c) represents the pixel value of the i-th image at position (a, b) on color channel c; a is the row index of the image in the spatial dimension; b is the column index of the image in the spatial dimension; color channel c∈{R1, R2, R3}, R1, R2, R3 correspond to the RGB three color channels of the color image respectively; μ c represents the mean value of all pixels on color channel c; σ c Represents the standard deviation on color channel c; Step 1.3, calculate the information entropy of the image based on the distribution of pixel values; Step 1.4: Perform dynamic adaptive sampling of face images in the training set based on information entropy to obtain an updated training set.

3. According to claim 2, the method for protecting sensitive face data based on a flexible anti-interference privacy protection mechanism is characterized in that: The specific process of step 1.3 is as follows: Step 1.3.1, count the distribution probability of each image pixel value, assuming a pixel value k in the range [0, K], K is the maximum value of the pixel value k; the calculation formula of the distribution probability is as follows: Among them, p i,k,c is the distribution probability of pixel value k in color channel c of the i-th image; |{(a,b)|x i (a, b, c) = k}| represents the i-th image X i The number of pixels with pixel value k in color channel c; Step 1.3.2: Calculate the information entropy of each image in each color channel. The formula is: Among them, H(X i ,c) is the i-th image X i Information entropy on color channel c; Step 1.3.3: Average the information entropy of each image in each color channel to obtain the final total information entropy of each image. The formula is: Among them, H total (X i ) is the i-th image X i The final total information entropy.

4. According to claim 3, the method for protecting sensitive face data based on a flexible anti-interference privacy protection mechanism is characterized in that: The specific process of step 1.4 is as follows: Step 1.4.1, based on the information entropy of each image, dynamically assign a sampling probability to each image; the sampling probability assignment formula is as follows: in, is the sampling probability of the i-th image; φ≥1 is a tuning parameter that controls the influence of entropy value on the sampling probability; j is the image number different from i; X j is the jth image; Step 1.4.2: Perform random sampling according to the sampling probability of the image. First, generate a random number for each image, compare the random number with the sampling probability, and update the training set. Let the random number generated for the i-th image be r i ∈[0,1], if Then choose to add the i-th image to the updated training set D′ train ; Otherwise, it is not selected; Finally, an updated training set D′ is obtained train , 5. According to claim 4, the method for protecting sensitive face data based on a flexible anti-interference privacy protection mechanism is characterized in that: The specific process of step 2 is: Step 2.1: Allocate a privacy budget to each image using the following formula: Among them, ∈ i is the privacy budget of the i-th image; ∈0 is the basic privacy budget; H max is the maximum entropy value of the samples in the data set; γ is the smoothing constant; After allocating the privacy budget to each image, the privacy budget of each image is added together to ensure that the sum of the privacy budgets of all samples does not exceed the total privacy budget ∈ total ; If the total privacy budget is exceeded, the privacy budget of all images needs to be scaled, and the final privacy budget for each image is as follows: in, is the entropy adjustment privacy budget of the i-th image, i.e., the final privacy budget; ∈ j is the privacy budget of the jth image; M is the updated training set D′ train The total number of images in ; Step 2.2: After allocating the privacy budget, use the entropy-modulated differential privacy protection mechanism to add Gaussian noise to each image in the updated training set. The specific process is as follows: Step 2.2.

1. Calculate the standard deviation of the Gaussian noise to be added to each image based on the final privacy budget and global sensitivity of each image in the training set. The formula is as follows: in, is the standard deviation of the Gaussian noise of the i-th image; Δf is the global sensitivity; δ is the failure probability parameter in the differential privacy mechanism; Step 2.2.2, after obtaining the standard deviation of each image, generate Gaussian noise for each image. Gaussian noise follows a normal distribution: Among them, g i is the Gaussian noise of the i-th image; is the normal distribution function; Step 2.2.3, add Gaussian noise to the face image to obtain the noisy image. The formula is as follows: X″ i =X′ i +g i (12); Among them, X″ i is the i-th image after adding noise; X′ i is the i-th image to which the standard deviation is added; Step 2.2.4: All the noisy face images constitute the second updated training set D″ train ={X″1,X″2,…,X″ i ,…,X″ M }; Since the second update only adds noise disturbance to the images in the data set and does not change the number of data sets, the training set D″ of the second update train The total number of images in is still M.

6. According to claim 5, the method for protecting sensitive face data based on a flexible anti-interference privacy protection mechanism is characterized in that: In step 3, the dynamic convolution layer uses the He initialization method to initialize the basic convolution kernel weights, and adaptively allocates the convolution kernel weights according to the input image data; the specific working process of the dynamic convolution layer is as follows: Step 3.1: Assume that the dynamic convolution layer has G basic convolution kernels, each of which has a size of h×h and a number of input feature channels of C. in , the number of channels of the output feature is C out ; The dynamic convolution layer uses the He initialization method to initialize the basic convolution kernel weights of the dynamic convolution layer. The specific initialization formula is as follows: in, represents the basic convolution kernel weight between the d-th input channel and the e-th output channel in the l-th layer; is the number of channels of input features in the lth layer; h (l) ·h (l) Indicates the size of the basic convolution kernel in the lth layer. It is a common number when calculated here, so it is represented by dot multiplication; Step 3.2, a lightweight sub-network is designed in the dynamic convolution layer; the sub-network consists of a global pooling layer, two fully connected layers and an activation function. The global information of the input features is extracted through the global pooling layer, and the first fully connected layer performs linear transformation. The second fully connected layer generates dynamic weight coefficients and normalizes them through Softmax. Finally, an adaptive weight coefficient is assigned to each convolution kernel. The sub-network is based on D″ train The final weight coefficient set α=[α1,α2,…,α s ,…,α G ], α G is the final weight coefficient of the Gth basic convolution kernel, α s is the final weight coefficient of the sth basic convolution kernel; the specific working process of the subnetwork is: Step 3.2.1: The noisy face image first passes through the global average pooling layer to obtain the pooled features. The formula is as follows: in, is the i-th image after adding noise X″ i Features after pooling; GAP(·) is the global average pooling layer; m and n are the height and width of the image, respectively; a is the row index of the image in the spatial dimension; b is the column index of the image in the spatial dimension; x″ i (a, b) represents the pixel value of the i-th image at position (a, b) after adding noise; Step 3.2.2: The pooled features are linearly transformed through the first fully connected layer to obtain hidden features. The formula is as follows: Among them, z is the hidden feature of the pooled feature; is the weight of the first fully connected layer; b1 is the bias of the first fully connected layer; X gp Represents the features after image pooling; Step 3.2.3: Use the ReLU activation function to perform nonlinear mapping on the hidden features to obtain the mapped hidden features. The formula is as follows: Z′=ReLU(z) (16); Where z′ is the mapping hidden feature; ReLU(·) is the ReLU activation function; Step 3.2.4, map the hidden features through the second fully connected layer to generate the initial weight coefficient β of the basic convolution kernel, and use the Softmax function to normalize it to get the final weight coefficient. The formula is as follows: Among them, β s is the initial weight coefficient of the sth basic convolution kernel; α s is the final weight coefficient of the sth basic convolution kernel; is the weight of the second fully connected layer; b2 is the bias of the second fully connected layer; Softmax(·) is the Softmax function; ep is the serial number of the basic convolution kernel; exp(·) is the exponential function with e as the base; z′ s is the mapping hidden feature of the sth basic convolution kernel; β ep is the initial weight coefficient of the epth basic convolution kernel; Step 3.3: Dynamically adjust the final weight coefficient of the basic convolution kernel by weighted summation. Weighted summation of multiple basic convolution kernels according to the final weight coefficients generated by the lightweight subnetwork above is performed to obtain a dynamic convolution kernel. The formula is as follows: Among them, W dyn is the dynamic convolution kernel; W s is the sth basic convolution kernel.

7. The method for protecting sensitive face data based on a flexible anti-interference privacy protection mechanism according to claim 6 is characterized in that: In the pre-training process of step 4, the noisy image X″ is subjected to convolution and pooling operations to generate a feature map, and then the global features are extracted through the global pooling layer and the fully connected layer. Finally, the model parameters are adjusted by optimizing the cross entropy loss function of the common sample; the formula of the cross entropy loss function of the common sample is as follows: Among them, L CE is the cross entropy loss function value of the common sample; M′ is the number of images in each batch; y i is the true label of the i-th image; is the predicted probability of the ith image.

8. The method for protecting sensitive face data based on a flexible anti-interference privacy protection mechanism according to claim 7 is characterized in that: The specific process of step 5 is as follows: Step 5.1: Initialize the perturbation amplitude, step size, and first round of adversarial samples, using D″ train The first round of adversarial samples are initialized with the noisy face image. The formula is as follows: in, is the first round of adversarial samples; X″ is the face image after adding noise; Step 5.2: Define the loss function of the adversarial sample. The cross entropy loss function is also used. The specific formula is as follows: Among them, L adv is the cross entropy loss function value of the adversarial sample; M″ is the number of adversarial samples in each round; represents the model's predicted probability of the adversarial sample corresponding to the i-th image; y i represents the true label of the i-th image; Step 5.3: Adaptively adjust the step size and disturbance amplitude according to the feedback of the model to ensure that the step size and disturbance amplitude are the optimal values ​​after dynamic adjustment; the dynamic adjustment formula is as follows: Among them, c i is the perturbation amplitude of the adversarial sample corresponding to the i-th image, which is used to control the perturbation amount applied to the sample during the generation of the adversarial sample; c base is the initial disturbance amplitude; λ′ is the disturbance amplitude adjustment coefficient, which is used to control the adjustment intensity of the disturbance amplitude; is the average loss value of the current batch of adversarial samples, and the formula is as follows: Among them, N adv is the number of adversarial samples in the current batch; According to H total (X i ) Adaptively adjust the generation iteration step of the adversarial sample, the formula is as follows: or i =the base ·(1+λ″·H total (X i )) (25); Among them, η i is the step size of the adversarial sample corresponding to the i-th image; η base is the initial step size; λ″ is the step size adjustment coefficient, which is used to control the influence of information entropy on step size adjustment; Step 5.4: Dynamically select the attack strategy for generating adversarial samples based on the current performance of the model.

9. The method for protecting sensitive face data based on a flexible anti-interference privacy protection mechanism according to claim 8, characterized in that: The specific process of step 5.4 is as follows: Step 5.4.

1. Calculate the loss change rate using the following formula: in, is the rate of change of the cross entropy loss of the adversarial sample in the tth round; are the cross entropy loss function values ​​of the adversarial samples in the tth and t-1th rounds respectively; u is a constant; Step 5.4.2: Dynamically switch the attack strategy for generating adversarial samples according to the cross entropy loss change rate of the model; pre-set the first threshold τ fgsm and the second threshold τ pgd , used to determine which attack strategy to choose; it includes three attack strategies, namely the fast gradient sign method, the projected gradient descent method, and the deep deception method; the specific judgment rules are as follows: In the early stages of training, if Use the fast gradient symbol method to generate adversarial samples. The formula is as follows: in, is the adversarial sample corresponding to the i-th image in round t; is the i-th image after the t-th round of noise addition; is the step size of the adversarial sample corresponding to the i-th image in the t-th round; yes The loss gradient of ; sign is the sign of the gradient; represents the model's predicted probability of the adversarial sample corresponding to the i-th image in the t-th round; represents the true label of the i-th image in the t-th round; In the middle of training, if Use the projected gradient descent method to generate adversarial samples and gradually optimize the perturbation. The formula is as follows: in, is the adversarial sample corresponding to the i-th image in the t+1th round; It is a projection operation, which projects the updated adversarial sample back into the range to ensure that the adversarial sample is within the restricted area; is the i-th image after the t-th round of noise addition; So A spherical area with the center at and the radius at the disturbance amplitude c; yes The loss gradient of In the later stages of training, if Use the deep deception method to generate adversarial samples. The formula is as follows: Among them, f df (·) is the classification output score of the current sample; It is a sample The gradient of the classification output score; is the square of the L2 norm of the gradient; Step 5.5: D″ train The common sample set D in each round normal With adversarial sample set D adv After mixing in proportion, the training is performed with the following formula: Among them, D train is the mixed training set; is the mixing ratio; The total loss function L of the model total as follows: in, is the mixing ratio; After training, the dynamic entropy-aware privacy adversarial model is evaluated using the test set to measure the performance of the model.

10. The method for protecting sensitive face data based on a flexible anti-interference privacy protection mechanism according to claim 9, characterized in that: The specific process of step 6 is as follows: Step 6.1: Based on the feedback of each round of dynamic entropy-aware privacy adversarial model training results, first dynamically and adaptively sample the data based on information entropy; then, add adaptive Gaussian noise to the sampled data based on the information entropy of the sampled data; Step 6.2: Based on the performance of the current dynamic entropy-aware privacy adversarial model, generate adversarial samples based on the noisy data; then, combine these adversarial samples with the noisy normal samples in proportion to form a mixed training set; Step 6.3: Use the mixed training set to train the dynamic entropy-aware privacy adversarial model; Step 6.4: After each round of iteration, fine-tune the sampling, noise addition, and adversarial training process based on the training results of the dynamic entropy-aware privacy adversarial model to gradually improve the performance of the model until the preset gender recognition accuracy is reached or the maximum number of training rounds is met under the condition of differential privacy protection.

Images