Multi-level weighted face image differential privacy protection method

Abstract

The invention discloses a multi-level weighted face image differential privacy protection method. The method comprises the following steps: firstly, acquiring a wavelet coefficient of a face image by applying multistage wavelet transform; secondly, calculating the influence of the noise of the wavelet coefficient of each sub-band of each level on the face features by using partial derivative; then, respectively quantizing privacy budget contributed by noise disturbing each level of weighted wavelet coefficient to face features and a local data utility loss function, decomposing a constraint optimization problem into a plurality of sub-problems level by level, and solving a preliminarily optimized noise scale parameter; then, according to the preliminarily optimized scale parameters, reconstructing a global constraint optimization problem, and further solving to obtain optimized scale parameters; and finally, obtaining a disturbed face image and a face feature vector according to a multi-level weighted differential privacy mechanism, hierarchical sampling of noise, disturbance of wavelet coefficients and inverse wavelet transformation. According to the method, sensitive face features in the public face image can be effectively protected, and meanwhile, the visual effect of the image is reserved.

Description

Technical Field

[0001] This invention belongs to the field of computer science, and in particular relates to a multi-level weighted differential privacy protection method for facial images. Background Technology

[0002] Facial features are crucial biometrics for identity verification, possessing uniqueness and resistance to replication, making them a primary target for attackers seeking to steal sensitive information. The widespread dissemination of user facial images on public platforms poses a serious threat to their privacy, making the application of privacy protection mechanisms to facial features essential. Currently, facial feature privacy protection methods during image publishing primarily focus on mosaic, data perturbation, and blurring. In recent years, differential privacy perturbation mechanisms have gained significant attention due to their reduced algorithmic complexity and verifiable privacy protection; however, existing differential privacy schemes have failed to effectively preserve image visual quality. We propose, for the first time, a multi-level weighted differential privacy protection method for facial images that, given a privacy budget, guarantees image integrity and quality, achieving a dynamic balance between privacy protection and application value.

[0003] Definitions:

[0004] Wavelet transform: By using a scalable and translational basis function, an image is transformed from the pixel domain to the more easily processed frequency domain, taking advantage of its inherent sparsity and hierarchy to achieve efficient and accurate noise perturbation.

[0005] Multi-level weighted differential privacy: After the face image undergoes an L-level wavelet transform, Laplacian noise is added to the weighted wavelet coefficients, and parameters are shared within the same level and subband. Specifically, in levels 2 to L-1, each weighted coefficient is perturbed; in level 1, the first K-1 weighted coefficients are perturbed, which is a privacy protection mechanism.

[0006] Privacy budget: a measure of privacy protection; the smaller the value, the greater the level of privacy protection.

[0007] Geometric superposition mechanism: For multiple variables that follow a geometric distribution, summation and other superposition operations are performed on these multiple random variables to generate a new variable that follows a Laplace distribution. Summary of the Invention

[0008] To address the aforementioned issues, this paper proposes a multi-level weighted differential privacy protection method for face images, called ML-WDP, to protect sensitive facial features.

[0009] The objective of this invention is achieved through the following technical solution:

[0010] Let p (0 < p < 1) be the probability parameter of the sampling geometry K in the geometric superposition mechanism, K ~ Geo(p). ε0 is a constant, representing the total privacy budget for a given face feature. The pixel matrix of the face image is P, with... The facial feature vector of a face image consists of [number] elements. have There are several elements. `i` is the index of an element in the face feature vector, and `f` is the index of that element. i Δ represents the i-th element of the face feature vector. i It is f i Sensitivity Let the constant term C p ,

[0011]

[0012] Step 1: The face image undergoes an L-level wavelet transform (Haar wavelet transform, HWT) to obtain wavelet coefficients representing facial features. The coefficients are grouped by level and sub-band. l represents the level index of the wavelet coefficients, l∈{1,…,L}. Each block of level l has… One coefficient, j represents the sub-band index of the coefficient. When the coefficient belongs to level l (l∈{1,…,L-1}), j∈{2,3,4}; when the coefficient belongs to level L (l=L), j∈{1,2,3,4}. k is the sequential index of the coefficient within the sub-band. This represents the k-th coefficient of the j-th subband of the l-th layer.

[0013] Step 2: Calculate the weights using partial derivative operations. It is the k-th coefficient of the j-th subband of the l-th layer, which is Laplacian noise with a mean of 0 and a scale parameter of arbitrary value. i ′ is the perturbed facial feature vector The i-th element.

[0014] Step 3: Using variance calculation, calculate the weighted wavelet coefficients of the perturbation from level L to level 1 on the perturbed face feature vector. The i-th element f i Privacy budget contribution

[0015] 1) The scaling parameter of the Laplace noise added to the wavelet coefficients of the j-th subband (j∈{1,2,3,4}) of level L is: The weighting coefficients of this level of disturbance affect f i The privacy budget contributed is

[0016]

[0017] 2) The scaling parameter of the Laplace noise added to the wavelet coefficients of the j-th subband (j∈{2,3,4}) of the l-th level (l∈{2,3,…,L-1}) in the intermediate layer is: The weighting coefficients of this level of disturbance affect f i The privacy budget contributed is

[0018]

[0019] 3) The scaling parameter of the Laplace noise added to the wavelet coefficients of the j-th subband (j∈{2,3,4}) of level 1 is: Based on the properties of the geometric superposition mechanism, the weighting coefficients of this level of perturbation affect f. i The privacy budget contributed is

[0020]

[0021] Step 4, assuming that in the wavelet coefficients of the L-th level perturbation, the privacy budget contribution of the coefficients of the l-th level to ε0 is ε (l) Let l ∈ {1,…,L}. Based on the structural decomposition of the L-th level wavelet coefficients, L constrained optimization subproblems are obtained. The l-th subproblem is to determine the coefficients of the l-th level perturbation relative to the perturbed face feature vector. The sum of the privacy budgets contributed by each element equals ε. (l) Under the given conditions, solve the data utility loss function for the l-th level. Minimum Right now,

[0022]

[0023] Using variance calculations, calculate the data utility loss function for each level.

[0024] 1) When l = L,

[0025] 2) When l∈{2,…,L-1},

[0026] 3) When l = 1,

[0027] Step 5: Construct the constant term f of the l-th level (l∈{1,…,L}). c (l,p),

[0028]

[0029] and the constant term C w ,

[0030]

[0031] Solve the L subproblems using the Lagrange multiplier method to calculate the privacy budget ε of the wavelet coefficient contribution of the scale parameter of each preliminary optimization layer with respect to the perturbation of that layer. (l) expression

[0032]

[0033] Further calculations yielded the data utility function with respect to ε. (l) The expression f loss (l) (ε (l) ):

[0034]

[0035] Step 6: The global constraint is that the sum of the privacy budget contributions of the wavelet coefficients at each level of perturbation equals the given total privacy budget ε0, i.e., The global data utility loss function is the sum of the data utility loss functions at each level, that is, Construct a globally constrained optimization problem: Under the constraints, solve for the loss function f. loss (ε (l) ) Minimum ε (l) Calculate ε using the Lagrange multiplier method. (l) (ε0), and substitute it into Optimized scale parameters are obtained

[0036]

[0037] Step 7: Add noise to the face image: First, generate a Laplacian noise set by hierarchical subbands based on the optimized scale parameter, adding noise to all weighted wavelet coefficients level by level and subband from level L to level 2; then, sample the geometric number K from the geometric distribution Geo(p), adding noise to the first K-1 sorted weighted coefficients; finally, perform inverse wavelet transform to obtain the perturbed face image pixel matrix P′, and obtain the perturbed face feature vector.

[0038] In this scheme, based on the structural characteristics of wavelet coefficients, the scale parameters of shared noise in the sub-bands are determined hierarchically. After calculating the contribution of noise to the privacy budget hierarchically, preliminary optimized scale parameters are obtained through hierarchical optimization of sub-problems. These preliminary optimized scale parameters are then substituted into the overall constraint optimization problem for further optimization, yielding globally optimized scale parameters. Finally, by adding noise hierarchically, a noisy scale parameter is obtained.

[0039] Preferably, step 2 specifically involves perturbing the facial feature vector. The i-th element f i ′(i∈{1,…,M F The first-order Taylor expansion of}) with wavelet coefficient hierarchical structure characteristics is as follows:

[0040]

[0041] Wherein, the Laplace noise of the k-th coefficient of the j-th subband of the l-th layer is Based on the multi-level weighted differential privacy mechanism, the geometric number K is sampled from the geometric distribution Geo(p). In the first level, only the first K-1 weighted coefficients after sorting are perturbed. Then, in the first level, assuming the 2nd, 3rd, and 4th sub-bands are perturbed... One coefficient, and In each sub-band, when At that time, noise The value is 0. Therefore, the weight is defined as 0. f i The first-order Taylor expansion of ′ is

[0042]

[0043] Preferably, step 3 specifically involves f i The noise of ' is ζ i Assuming there are noise classification parameters

[0044]

[0045] According to f i The first-order Taylor expansion of ′ Since noise within the same subband and at the same level shares the same scale parameter, for all noise in the j-th subband of the l-th level, the scale parameter is: That is, for all k∈{1,…,N} (l)}, Using variance calculation,

[0046]

[0047] Assumption Follows a Laplace distribution, f i The sensitivity of ′ is Δ i Based on the fact that the variance of Laplace noise is twice the square of the scale parameter, and the relationship between privacy budget and Laplace noise, from Obtain the weighting coefficients for each level of perturbation on f i 'Contributed privacy budget'.

[0048] Preferably, step 4 specifically involves, similar to step 3, obtaining the data utility loss function after coefficient perturbation at level l through variance calculation.

[0049] Preferably, step 5 specifically involves, according to the Lagrange multiplier method, the Lagrange function of the l-th (l∈{1,…,L}) subproblem is:

[0050]

[0051] Where, λ l It is the Lagrange factor. Solve from the Lth order to the 1st order. The privacy budget ε, calculated for the initial optimized scale parameters of each layer with respect to the wavelet coefficients of the perturbation at that layer, is also calculated. (l) expression and data utility function with respect to ε (l) The expression f loss (l) (ε (l) ).

[0052] Preferably, step 6 specifically involves constructing a Lagrange function based on the Lagrange multiplier method, where λ is the Lagrange factor of the overall constraint optimization problem.

[0053]

[0054] Solve ε was calculated (l) (ε0),

[0055]

[0056] Substitute it The optimized scale parameters were calculated. An analytical expression for a given privacy budget ε0.

[0057] The beneficial effects of this invention are: this invention not only effectively prevents adversaries from obtaining sensitive facial features through a multi-level weighted differential privacy machine and retains higher data utility, but also decomposes the optimization problem by combining a hierarchical structure of multi-level wavelet coefficients, effectively reducing the parameters to be solved and lowering the complexity of the scale parameters for solving the optimization. Attached Figure Description

[0058] Figure 1 This is a detailed flowchart of the present invention;

[0059] Figure 2This is a comparison chart of the present invention (ML-WDP), the privacy-preserving face recognition method based on frequency domain discrete cosine transform differential privacy (DCT-DP), the DP method that adds noise to all pixels of a face image (Pixel-DP), and the peak signal-to-noise ratio (PSNR) of weighted wavelet differential privacy (WWDP) based on publicly available LFW face data.

[0060] Figure 3 This is a comparison chart of the peak signal-to-noise ratio (PSNR) of the present invention (ML-WDP), the privacy-preserving face recognition method based on frequency domain discrete cosine transform differential privacy (DCT-DP), the DP method that adds noise to all pixels of a face image (Pixel-DP), and the weighted wavelet differential privacy (WWDP) based on the publicly available PubFig83 face image dataset. Detailed Implementation

[0061] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0062] The server used in this invention experiment was equipped with a 13th generation Intel(R) Core(TM) i7-13700KF CPU. The operating system was Windows 10. The graphics processing unit was an NVIDIA GeForce RTX 4080. The parallel computing platform was CUDA 12.2. The programming language was Python 3.10. The deep learning framework was PyTorch 1.8.1cu111.

[0063] Multi-level weighted differential privacy protection method based on wavelet transform, such as Figure 1 As shown, the specific steps are as follows:

[0064] Let p (0 < p < 1) be the probability parameter of the sampling geometry K in the geometric superposition mechanism, K ~ Geo(p). ε0 is a constant, representing the total privacy budget for a given face feature. The pixel matrix of the face image is P, with... The facial feature vector of a face image consists of [number] elements. have There are several elements. `i` is the index of an element in the face feature vector, and `f` is the index of that element. i Δ represents the i-th element of the face feature vector. i It is f i Sensitivity Let the constant term C p ,

[0065]

[0066] Step 1: The face image undergoes an L-level wavelet transform (Haar wavelet transform, HWT) to obtain wavelet coefficients representing facial features. The coefficients are grouped by level and sub-band. l represents the level index of the wavelet coefficients, l∈{1,…,L}. Each block of level l has… One coefficient, j represents the sub-band index of the coefficient. When the coefficient belongs to level l (l∈{1,…,L-1}), j∈{2,3,4}; when the coefficient belongs to level L (l=L), j∈{1,2,3,4}. k is the sequential index of the coefficient within the sub-band. This represents the k-th coefficient of the j-th subband of the l-th layer.

[0067] Step 2, Perturbed facial feature vector The i-th element f i ′(i∈{1,…,M F The first-order Taylor expansion of}) with wavelet coefficient hierarchical structure characteristics is as follows:

[0068]

[0069] Wherein, the Laplace noise of the k-th coefficient of the j-th subband of the l-th layer is Geometric number K is sampled from geometric distribution Geo(p). In level 1, we assume perturbations in the 2nd, 3rd, and 4th subbands. One coefficient, and Define weights f i The first-order Taylor expansion of ′ is

[0070]

[0071] Calculate the weights using partial derivative operations.

[0072] Step 3, f i The noise of ' is ζ i Assuming there are noise classification parameters

[0073]

[0074] According to f i The first-order Taylor expansion of ′ Since noise within the same subband and at the same level shares the same scale parameter, for all noise in the j-th subband of the l-th level, the scale parameter is: That is, for all k∈{1,…,N} (l)}, Using variance calculation,

[0075]

[0076] Assumption Follows a Laplace distribution, f i The sensitivity of ′ is Δ i Calculate the weighted wavelet coefficients of the perturbations from level L to level 1 for the perturbation of the face feature vector. The i-th element f i Privacy budget contribution

[0077] 1) The scaling parameter of the Laplace noise added to the wavelet coefficients of the j-th subband (j∈{1,2,3,4}) of level L is: The weighting coefficients of this level of disturbance affect f i The privacy budget contributed is

[0078]

[0079] 2) The scaling parameter of the Laplace noise added to the wavelet coefficients of the j-th subband (j∈{2,3,4}) of the l-th level (l∈{2,3,…,L-1}) in the intermediate layer is: The weighting coefficients of this level of disturbance affect f i The privacy budget contributed is

[0080]

[0081] 3) The scaling parameter of the Laplace noise added to the wavelet coefficients of the j-th subband (j∈{2,3,4}) of level 1 is: Based on the properties of the geometric superposition mechanism, the weighting coefficients of this level of perturbation affect f. i The privacy budget contributed is

[0082]

[0083] Step 4, assuming that in the wavelet coefficients of the L-th level perturbation, the privacy budget contribution of the coefficients of the l-th level to ε0 is ε (l) Let l ∈ {1,…,L}. Based on the structural decomposition of the L-th level wavelet coefficients, L constrained optimization subproblems are obtained. The l-th subproblem is to determine the coefficients of the l-th level perturbation relative to the perturbed face feature vector. The sum of the privacy budgets contributed by each element equals ε. (l) Under the given conditions, solve the data utility loss function for the l-th level. Minimum Right now,

[0084]

[0085] Similar to step 3, variance calculation is used to calculate the data utility loss function for each level.

[0086] 1) When l = L,

[0087] 2) When l∈{2,…,L-1},

[0088] 3) When l = 1,

[0089] Step 5: Construct the constant term f of the l-th level (l∈{1,…,L}). c (l,p),

[0090]

[0091] and the constant term C w ,

[0092]

[0093] According to the Lagrange multiplier method, the Lagrange function of the l-th (l∈{1,…,L}) subproblem is:

[0094]

[0095] Where, λ l It is the Lagrange factor. Solve from the Lth order to the 1st order. The privacy budget ε, calculated for the initial optimized scale parameters of each layer with respect to the wavelet coefficients of the perturbation at that layer, is also calculated. (l) expression

[0096]

[0097] Further calculations yielded the data utility function with respect to ε. (l) The expression f loss (l) (ε (l) ):

[0098]

[0099] Step 6: The global constraint is that the sum of the privacy budget contributions of the wavelet coefficients at each level of perturbation equals the given total privacy budget ε0, i.e., The global data utility loss function is the sum of the data utility loss functions at each level, that is, Construct a globally constrained optimization problem: Under the constraints, solve for the loss function f. loss (ε (l) ) Minimum ε(l) According to the Lagrange multiplier method, λ is the Lagrange factor of the overall constrained optimization problem. We construct the Lagrange function.

[0100]

[0101] Solve ε was calculated (l) (ε0),

[0102]

[0103] Substitute it Optimized scale parameters are obtained

[0104]

[0105] Step 7: Add noise to the face image: First, generate a Laplacian noise set by hierarchical subbands based on the optimized scale parameter, adding noise to all weighted wavelet coefficients level by level and subband from level L to level 2; then, sample the geometric number K from the geometric distribution Geo(p), adding noise to the first K-1 sorted weighted coefficients; finally, perform inverse wavelet transform to obtain the perturbed face image pixel matrix P′, and obtain the perturbed face feature vector.

[0106] The design of this solution not only effectively prevents adversaries from obtaining sensitive facial features, but also preserves better visual effects within a given privacy budget.

Claims

1. A multi-level weighted differential privacy protection method for face images, characterized in that, Based on the hierarchical weighted wavelet coefficient structure obtained from multi-level wavelet transform, a scaling parameter for sharing differential privacy Laplacian noise among wavelet coefficients in the same level and subband is designed. Let p (0 < p < 1) be the probability parameter of the sampling geometry K in the geometric superposition mechanism, K ~ Geo(p), and ε0 be a constant, serving as the total privacy budget for a given face feature. The pixel matrix of the face image is P, with... The face feature vector of the face image has elements. have There are elements, where i is the index of an element in the face feature vector, and f... i Δ represents the i-th element of the face feature vector. i It is f i Sensitivity Let the constant term C p , The specific steps are as follows: Step 1: The face image undergoes an L-level wavelet transform (Haar wavelet transform, HWT) to obtain wavelet coefficients representing facial features. The coefficients are grouped by level and sub-band, where l represents the level index of the wavelet coefficients, l∈{1,…,L}. Each block of the l-th level has… One coefficient, j represents the sub-band index of the coefficient. When the coefficient belongs to level l (l∈{1,…,L-1}), j∈{2,3,4}; when the coefficient belongs to level L (l=L), j∈{1,2,3,4}. k is the sequential index of the coefficient within the sub-band. This represents the k-th coefficient of the j-th subband of the l-th layer. Step 2: Calculate the weights using partial derivative operations. It is the k-th coefficient of the j-th subband of the l-th layer, which is Laplacian noise with a mean of 0 and a scale parameter of arbitrary value. i ′ is the perturbed facial feature vector The i-th element; Step 3: Using variance calculation, calculate the weighted wavelet coefficients of the perturbation from level L to level 1 on the perturbed face feature vector. The i-th element f i Privacy budget contribution 1) The scaling parameter of the Laplace noise added to the wavelet coefficients of the j-th subband (j∈{1,2,3,4}) of level L is: The weighting coefficients of this level of disturbance affect f i The privacy budget contributed is 2) The scaling parameter of the Laplace noise added to the wavelet coefficients of the j-th subband (j∈{2,3,4}) of the l-th level (l∈{2,3,…,L-1}) in the intermediate layer is: The weighting coefficients of this level of disturbance affect f i The privacy budget contributed is 3) The scaling parameter of the Laplace noise added to the wavelet coefficients of the j-th subband (j∈{2,3,4}) of level 1 is: Based on the properties of the geometric superposition mechanism, the weighting coefficients of this level of perturbation affect f. i The privacy budget contributed is Step 4, assuming that in the wavelet coefficients of the L-th level perturbation, the privacy budget contribution of the coefficients of the l-th level to ε0 is ε (l) Let l ∈ {1,…,L}. Based on the structural decomposition of the L-th wavelet coefficients, L constrained optimization subproblems are obtained. The l-th subproblem is to find the coefficients of the perturbation at the l-th level in relation to the perturbed face feature vector. The sum of the privacy budgets contributed by each element equals ε. (l) Under the given conditions, solve the data utility loss function for the l-th level. Minimum Right now, Using variance calculations, calculate the data utility loss function for each level. 1) When l = L, 2) When l∈{2,…,L-1}, 3) When l = 1, Step 5: Construct the constant term f of the l-th level (l∈{1,…,L}). c (l,p), and the constant term C w , Solve the L subproblems using the Lagrange multiplier method to calculate the privacy budget ε of the wavelet coefficient contribution of the scale parameter of each preliminary optimization layer with respect to the perturbation of that layer. (l) expression Further calculations: Obtain the data utility function with respect to ε (l) The expression f loss (l) (ε (l) ); Step 6: The global constraint is that the sum of the privacy budget contributions of the wavelet coefficients at each level of perturbation equals the given total privacy budget ε0, i.e., The global data utility loss function is the sum of the data utility loss functions at each level, that is, Construct a globally constrained optimization problem: Under the constraints, solve for the loss function f. loss (ε (l) ) Minimum ε (l) Calculate ε using the Lagrange multiplier method. (l) (ε0), and substitute it into Optimized scale parameters are obtained The parsing expression; Step 7: Add noise to the face image: First, generate a Laplacian noise set by hierarchical subbands based on the optimized scale parameter, adding noise to all weighted wavelet coefficients level by level and subband from level L to level 2; then, sample the geometric number K from the geometric distribution Geo(p), adding noise to the first K-1 sorted weighted coefficients; finally, perform inverse wavelet transform to obtain the perturbed face image pixel matrix P′, and obtain the perturbed face feature vector.

2. The multi-level weighted differential privacy protection method for face images according to claim 1, characterized in that, Specifically, step 3 is f i The noise of ' is ζ i Assuming there are noise classification parameters According to f i The first-order Taylor expansion of ′ Since noise within the same subband and at the same level shares the same scale parameter, for all noise in the j-th subband of the l-th level, the scale parameter is: That is, for all k∈{1,…,N} (l) }, Using variance calculation, Assumption Follows a Laplace distribution, f i The sensitivity of ′ is Δ i Based on the fact that the variance of Laplace noise is twice the square of the scale parameter, and the relationship between privacy budget and Laplace noise, from Obtain the weighting coefficients for each level of perturbation on f i 'Contributed privacy budget'.

3. The multi-level weighted differential privacy protection method for face images according to claim 1, characterized in that, Step 5 specifically involves assuming that in the wavelet coefficients of the L-th level perturbation, the privacy budget contribution of the coefficients of the l-th level to ε0 is ε. (l) Let l∈{1,…,L}. The structural decomposition of the L-th wavelet coefficients yields L constrained optimization subproblems. According to the Lagrange multiplier method, the Lagrangian function of the l-th (l∈{1,…,L}) subproblem is: Where, λ l It is the Lagrange factor, from the Lth to the 1st order, solve for... The privacy budget ε, calculated for the initial optimized scale parameters of each layer with respect to the wavelet coefficients of the perturbation at that layer, is also calculated. (l) expression and data utility function with respect to ε (l) The expression f loss (l) (ε (l) ).

Images