Differential privacy face image protection method based on feature mapping

Abstract

A feature mapping-based differential privacy face image protection method is disclosed.The target sample data set is clustered using a decision tree clustering algorithm, and then stratified sampling is performed to obtain a small sample target set.The average face feature vector is calculated by weighted aggregation of the small sample target set.The distance between the feature vector of the face region in each target sample data set image and the feature vector of the average face is used as the privacy budget weight distribution standard, which aims to reduce the noise added to the image and improve the image usability.After the small sample target set is reduced by PCA, the face feature vectors of different lengths k are intercepted, the structural similarity of the face feature vectors after adding noise and the original image is calculated, and the model recognition rate after being put into the CNN classification model is calculated.The k value when the two curves are closest is the face feature vector length of the image protection method, which balances the image usability and privacy, and does not need to manually set parameters, and completes the noise addition task of the face part in the image.

Description

Technical Field

[0001] This invention relates to a differential privacy face image protection method based on feature mapping, belonging to the field of information security technology. Background Technology

[0002] Privacy-Preserving Data Publishing (PPDP) is a promising method of information sharing that can simultaneously protect personal privacy and sensitive information. Sectors such as finance, healthcare, and e-commerce rely heavily on large-scale data publishing. In everyday life, the use of Multidimensional Sensitive Data (MSA) is more common than Single-Dimensional Sensitive Data (SSA). With the development of smartphones and high-resolution cameras, sharing photos has become much easier. For example, users on social media platforms like WeChat, Weibo, and QQ can publish photos taken with their phones or cameras in real time; major cloud service providers such as Google, Baidu, and Microsoft offer free image-based services, allowing users to save and manage their photos and download them to their phones or computers at any time. Collecting these photos and processing and analyzing large amounts of images using biometric technology could provide more personalized services for businesses and individuals. However, because images often contain sensitive facial information, simply sharing or publishing image data can lead to privacy breaches if attackers infer social relationships from facial images. Therefore, designing privacy-preserving publishing methods for facial images is essential.

[0003] However, the sharing and use of data has also brought about various privacy leaks, adding many security risks to people's daily lives. Currently, image privacy protection methods can be roughly divided into traditional anonymization methods and standard image anonymization methods based on deep learning. However, images anonymized using traditional anonymization methods have poor visual effects and lose the original image data, while standard image anonymization methods based on deep learning do not provide guaranteed privacy proofs.

[0004] Differential privacy was first proposed by Dwork et al. in 2006. Differential privacy is a relatively strict definition of privacy, offering different levels of privacy protection based on the privacy budget ε. For differential privacy, the presence or absence of a record in the dataset has little impact on the overall published results. Therefore, the risk of privacy leakage caused by a single record can be controlled within an acceptable range, largely eliminating individual hesitation and concerns about privacy breaches. Thus, differential privacy is a privacy protection method built upon a vast amount of background knowledge. Differential privacy methods distort sensitive data by adding perturbation noise, and the amount of added noise is independent of the data size, thus achieving a high level of privacy protection even for large datasets.

[0005] Among existing deep learning-based face image privacy publishing algorithms, differential privacy-based algorithms provide strict privacy guarantees. Furthermore, due to the characteristics of differential privacy, a balance can be struck between privacy protection and data availability by setting an appropriate privacy budget, meeting different privacy protection requirements. However, the ability of differential privacy-based deep learning face image privacy publishing algorithms to balance image availability and privacy needs improvement. Some researchers have proposed image privacy publishing algorithms based on PCA dimensionality reduction, which reduce the dimensionality of sensitive ROI regions using PCA and then add Laplacian noise to the resulting feature vectors. However, when adding noise to different sensitive regions within the same image, the algorithm does not consider the image properties and applies weighted noise, and the dimensionality reduction parameter k needs to be manually selected during dimensionality reduction. Therefore, the algorithm's ability to balance image availability and privacy is insufficient. Summary of the Invention

[0006] To effectively address the privacy leakage issue when images are posted on social media platforms, this invention proposes a differential privacy-preserving face image protection method based on feature mapping. The scheme first uses a decision tree clustering algorithm to cluster the target sample dataset, then performs stratified sampling on the clustered image clusters to obtain a small sample target set. Next, a weighted aggregation of the small sample target set is performed to calculate the average face feature vector. The distance between the face feature vector in each target sample image and this average face feature vector is used as the privacy budget weight allocation standard, aiming to reduce added noise and improve image usability. Finally, the small sample target set is dimensionality-reduced using PCA, and face feature vectors of different lengths k are extracted. The structural similarity (SSIM) of the noisy face feature vectors compared to the original image and the model recognition rate after being fed into a CNN classification model are calculated. The k value where the curves of the two are closest is the face feature vector length of this image protection method, balancing image usability and privacy, and completing the task of adding noise to the face portion of the image without manual parameter setting.

[0007] The technical solution of the present invention:

[0008] A differential privacy face image protection method based on feature mapping, comprising the following steps:

[0009] Define variables:

[0010] Table 1 Commonly Used Variables and Explanations

[0011]

[0012] (1) Cluster the images in the target sample dataset using the decision tree clustering algorithm, and then perform stratified sampling to obtain a small sample target set;

[0013] (1.1) Cluster the target sample dataset using the decision tree clustering algorithm;

[0014] Calculate the second moment, entropy, and contrast of the gray-level co-occurrence matrix corresponding to each image in the target sample dataset, which represent the uniformity of gray-level distribution, image complexity, and sharpness of the image, respectively.

[0015] The gray-level co-occurrence matrix features of second-order angular moments, entropy, and contrast are used as inputs to a decision tree clustering algorithm to cluster images in the target sample dataset, resulting in clustered image clusters.

[0016] (1.2) For each image cluster, during the stratified sampling process, the number of images sampled is set to half the number of each image cluster. Then, the corresponding number of images are randomly selected from the current image cluster and added to the small sample target set.

[0017] (2) The average face feature vector is calculated by weighted aggregation of the small sample target set. The Euclidean distance between the face region feature vector and the average face feature vector is set as the privacy budget weight allocation standard.

[0018] The process for setting privacy budget weighting criteria is as follows:

[0019] (2.1) For a small sample target set, the MTCNN face recognition model is used to determine the face region, and the PCA method is used to reduce the dimension and extract the face region feature vector of length K=16. The average face feature vector is obtained by calculating the average value of all face region feature vectors.

[0020] (2.2) For each image in the small sample target set, if the image contains more than one face, calculate the Euclidean distance between the feature vector of the face region in the image and the feature vector of the average face. The noise weight of each face region is allocated according to the Euclidean distance with the feature vector of the average face. The larger the distance, the more noise weights are allocated.

[0021] (3) After reducing the dimension of the small sample target set using the PCA method, extract the face region feature vectors of different lengths k. Calculate the structural similarity curve between the face region feature vectors after adding noise and the small sample target set, as well as the model recognition rate curve after being put into the CNN classification model. The k value when the two curves are closest is the length of the face region feature vector in this image protection method.

[0022] The specific process for calculating the value of k is as follows:

[0023] (3.1) After reducing the dimensionality of the small sample target set using the PCA method, extract the face region feature vectors of different lengths k, where k ranges from 2 to 16;

[0024] (3.2) Add noise to the face region feature vectors of different lengths k to satisfy the privacy measurement;

[0025] Specifically, for face region feature vectors of different lengths k, a k-dimensional space is mapped, where k is the dimension of the face region feature vector. In the k-dimensional space, x0 represents the input face region feature vector, which contains real singular values. The mechanism for sampling the output vector x according to the following probability distribution satisfies ε·d. k -privacy:

[0026]

[0027]

[0028] In the formula, d k Denotes the k-dimensional Euclidean distance;

[0029] (3.3) Calculate the structural similarity of the face region feature vector after adding noise and compare it with the small sample target set, and the model recognition rate after being put into the CNN classification model. Plot the SSIM curve and recognition rate curve when the face region feature vectors of different lengths k are truncated. The k value when the two curves are closest is the face region feature vector length of this image protection method.

[0030] (3.4) After obtaining the value of k, the face regions of the target sample dataset are compressed into face region feature vectors of length k using the PCA method. If the image contains more than one face, the Euclidean distance between each face region feature vector and the average face feature vector is calculated. The noise weights for each face region are allocated according to the Euclidean distance between them and the average face feature vector. The image is then denoised to satisfy the privacy measurement. Finally, the denoised face region feature vectors are restored to the image to generate a denoised image.

[0031] The beneficial effects of this invention: The publication of image data can provide a wealth of user information to facilitate data analysis and mining. However, while bringing convenience, it also brings a series of security issues. Improper publication can pose a series of threats to users' privacy information. Therefore, this invention proposes a differential privacy face image protection method based on feature mapping. By combining PCA dimensionality reduction with the Laplacian mechanism of differential privacy, noise is added to the feature vectors of sensitive parts in the image, thereby protecting sensitive information in the image and ensuring high image usability. Attached Figure Description

[0032] Figure 1 This is a structural diagram of the differential privacy face image protection strategy based on feature mapping described in this invention.

[0033] Figure 2This is a flowchart of the image availability adjustment described in this invention.

[0034] Figure 3 This is a flowchart illustrating the trade-off between image availability and privacy as described in this invention. Detailed Implementation

[0035] To more clearly illustrate the objectives, technical solutions, and advantages of this invention, the following detailed description will be provided through embodiments and accompanying drawings. A differential privacy face image protection method based on feature mapping comprises the following steps:

[0036] This method includes how to adjust image availability and how to balance image availability and privacy.

[0037] Reference Figure 2 The specific process for adjusting image availability is as follows:

[0038] Step 1: Calculate the gray-level co-occurrence matrix for each image.

[0039] Step 2: Calculate the second moment of the gray-level co-occurrence matrix, its entropy, and its contrast.

[0040] Step 3: Cluster the images using the decision tree clustering algorithm.

[0041] Step 4: Stratified sampling is used to obtain a small sample target set.

[0042] Step 5: Use the MTCNN face recognition model to determine the face regions in a small sample target set.

[0043] Step 6: Convert the face region into a face feature vector using the PCA method.

[0044] Step 7: Calculate the average face feature vector to obtain the average face feature vector.

[0045] Reference Figure 3 The specific process of balancing image usability and privacy is as follows:

[0046] Step 8: Perform PCA dimensionality reduction on the small sample target set.

[0047] Step 9: Extract facial feature vectors of different lengths k.

[0048] Step 10: After adding noise, feed the data into the CNN classification model and plot the model recognition rate curves for different face feature vector lengths k.

[0049] Step 11: Calculate the SSIM value curves of the image after adding noise and the original image for different face feature vector lengths k.

[0050] Step 12: Find the length of the face feature vector when the two curves are closest, and determine the optimal length of the face feature vector.

[0051] Step 13: Perform PCA dimensionality reduction on the face regions of all images.

[0052] Step 14: Add weighted noise to the face feature vector, and assign weights based on the distance from the average face feature vector.

[0053] Step 15: Restore the noisy facial feature vectors to the image.

Claims

1. A differential privacy face image protection method based on feature mapping, characterized in that, The steps are as follows: (1) Cluster the images in the target sample dataset using the decision tree clustering algorithm, and then perform stratified sampling to obtain a small sample target set; (1.1) Cluster the target sample dataset using the decision tree clustering algorithm; Calculate the second moment, entropy, and contrast of the gray-level co-occurrence matrix corresponding to each image in the target sample dataset, which represent the uniformity of gray-level distribution, image complexity, and sharpness of the image, respectively. The gray-level co-occurrence matrix features of second-order angular moments, entropy, and contrast are used as inputs to a decision tree clustering algorithm to cluster images in the target sample dataset, resulting in clustered image clusters. (1.2) For each image cluster, during the stratified sampling process, the number of images sampled is set to half the number of each image cluster. Then, the corresponding number of images are randomly selected from the current image cluster and added to the small sample target set. (2) The average face feature vector is calculated by weighted aggregation of the small sample target set. The Euclidean distance between the face region feature vector and the average face feature vector is set as the privacy budget weight allocation standard. The process for setting privacy budget weighting criteria is as follows: (2.1) For a small sample target set, the MTCNN face recognition model is used to determine the face region, and the PCA method is used to reduce the dimension and extract the face region feature vector of length K=16. The average face feature vector is obtained by calculating the average value of all face region feature vectors. (2.2) For each image in the small sample target set, if the image contains more than one face, calculate the Euclidean distance between the feature vector of the face region in the image and the feature vector of the average face. The noise weight of each face region is allocated according to the Euclidean distance with the feature vector of the average face. The larger the distance, the more noise weights are allocated. (3) After reducing the dimension of the small sample target set using the PCA method, extract the face region feature vectors of different lengths k. Calculate the structural similarity curve between the face region feature vectors after adding noise and the small sample target set, as well as the model recognition rate curve after being put into the CNN classification model. The k value when the two curves are closest is the length of the face region feature vector in this image protection method. The specific process for calculating the value of k is as follows: (3.1) After reducing the dimensionality of the small sample target set using the PCA method, extract the face region feature vectors of different lengths k, where k ranges from 2 to 16; (3.2) Add noise to the face region feature vectors of different lengths k to satisfy the privacy measurement; Specifically, for face region feature vectors of different lengths k, a k-dimensional space is mapped, where k is the dimension of the face region feature vector. In the k-dimensional space, x0 represents the input face region feature vector, which contains real singular values. The mechanism for sampling the output vector x according to the following probability distribution satisfies ε·d. k -privacy: In the formula, d k Represents the k-dimensional Euclidean distance; (3.3) Calculate the structural similarity of the face region feature vector after adding noise and compare it with the small sample target set, and the model recognition rate after being put into the CNN classification model. Plot the SSIM curve and recognition rate curve when the face region feature vectors of different lengths k are truncated. The k value when the two curves are closest is the face region feature vector length of this image protection method. (3.4) After obtaining the value of k, the face regions of the target sample dataset are compressed into face region feature vectors of length k using the PCA method. If the image contains more than one face, the Euclidean distance between each face region feature vector and the average face feature vector is calculated. The noise weights for each face region are allocated according to the Euclidean distance between them and the average face feature vector. The image is then denoised to satisfy the privacy measurement. Finally, the denoised face region feature vectors are restored to the image to generate a denoised image.

Images