A privacy protection method and system for image data

Abstract

The application discloses a kind of privacy protection method and system of image data.The method is: obtaining the original image data set to be protected and its corresponding label;Hybrid agent module is constructed, including fixed parameter model, updatable parameter model, class center aggregation module;A batch of images is collected from the original image data set and is trained after adding noise to it updatable parameter model;Final noise is obtained by noise iterative update, and the method of each update is: a batch of images is collected and is input into fixed parameter model, updatable parameter model after adding noise to it, then calculate classification loss according to predicted output result and corresponding label, and the migration loss of class center aggregation module is calculated according to the deep feature output by updatable parameter model;The joint loss is calculated according to the migration loss and the classification loss of two models to update noise;The image in original image data set is protected using final noise.The image protected by the application still has good visual quality.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and data security technology, specifically relating to a method for protecting the privacy of image data, an electronic device and a storage medium, which is particularly suitable for preventing unauthorized data from being effectively used by deep learning models in scenarios such as image sharing and model training. Background Technology

[0002] With the rapid development of deep learning technology, massive image data has become the key fuel driving breakthroughs in computer vision, image recognition, and multimodal large-scale models. From the classic ImageNet to today's large-scale open-source and private datasets, high-quality training data directly determines the performance ceiling of a model. Deep neural networks, represented by ResNet and Transformer, as well as large-scale models such as ChatGPT and DeepSeek-V2, all rely on learning statistical patterns from massive amounts of data for their outstanding understanding and generation capabilities.

[0003] However, this deep reliance on and widespread collection of data has also brought increasingly serious risks of privacy breaches. Sensitive images such as personal photos, medical images, and business design drawings may be used to train business models without explicit authorization. This not only infringes on the privacy and ownership rights of the data subject, but may also cause the model to memorize and leak sensitive information. Therefore, users, businesses, and research institutions are increasingly emphasizing the protection of their data assets and opposing the arbitrary scraping and unauthorized use of data.

[0004] To address this contradiction, researchers proposed the "non-learnable examples" technique. The core idea of ​​this technique is to apply a carefully designed, imperceptible, subtle perturbation (often called "noise" or "adversarial perturbation") to the data before it is made public. The processed data is visually almost indistinguishable from the original data and can be used normally for display, sharing, and other purposes. However, when this data is used to train deep neural networks, its inherent noise interferes with and disrupts the model's learning process, preventing the model from extracting effective features and patterns from the data. Ultimately, this results in a significant drop in model performance (such as classification accuracy) when evaluated on clean, unperturbed test data, thus achieving the protective goal of "data usable but model unlearnable."

[0005] While this paradigm offers a novel approach to data privacy protection, existing unlearnable methods still face significant challenges in practical deployment, primarily in the following two aspects: 1) Insufficient robustness: The effectiveness of unlearnable examples heavily relies on the assumption that the attacker (model trainer) will use a standard, undefended training process. However, in real-world scenarios, trainers commonly employ various defensive training techniques to improve model generalization and robustness. For example, actively injecting adversarial examples during training can make the model insensitive to minor perturbations. This can potentially "immunize" against unlearnable noise interference, allowing model performance to recover. Methods such as RandomAugment, MixUp, and CutMix, through complex mixing and transformation of training data, can potentially disrupt or dilute pre-added specific noise patterns, weakening their interference effect.

[0006] 2) Limited portability: Existing methods typically optimize for a specific "proxy model" when generating noise. This design means that strong interfering noise generated on one model (such as ResNet) may have little effect on another model with a very different architecture (such as Vision Transformer or DenseNet). In reality, attackers may use models with any unknown architecture.

[0007] In summary, the vulnerabilities of existing technologies to real-world training defenses and their rigidity in handling unknown application scenarios severely hinder the transition of "non-learnable example" technology from theory to widespread practical application. Therefore, the industry urgently needs an innovative image privacy protection technology that can generate images that are not only imperceptible to the human eye but also possess strong robustness to resist various training defenses and high transferability to cover perturbations from unknown models and data distributions. Only such a technology can provide truly reliable and practical protection for the secure sharing of image data in open network environments. Summary of the Invention

[0008] The present invention aims to overcome the shortcomings of the prior art and provide an image privacy protection method and system to solve the technical problems of poor robustness and weak transferability between different models when facing defense measures in existing non-learnable example techniques.

[0009] To achieve the above objectives, the present invention adopts the following technical solution: A method for protecting the privacy of image data, comprising the following steps: Obtain the original image dataset to be protected and its corresponding labels; Construct a hybrid proxy module, including a fixed parameter model, an updatable parameter model, and a class-centric aggregation module; A batch of images are collected from the original image dataset and noise is added to them to train the updatable parameter model; The noise is iteratively updated to obtain the final noise. Each update involves: collecting a batch of images from the original image dataset, adding noise to them, and inputting them into a fixed-parameter model and a trained, updatable-parameter model, respectively. Then, based on the predicted output and corresponding labels, the classification loss generated by the fixed-parameter model and the classification loss generated by the trained, updatable-parameter model are calculated. The class center aggregation module calculates the transferability loss based on the deep features output by the trained, updatable-parameter model. The noise is then updated based on the joint loss calculated from the transferability loss and the two classification losses. The images in the original image dataset are protected using the final noise.

[0010] Preferably, the migration loss is ;in, For a set of categories, It is a category The number of samples, It is a category The feature centers of the noisy samples For the updatable parameter model DSM, the input image ( The deep features output during processing. For the i-th sample image of category c, for Noise added; It is the deep feature output by the updatable parameter model DSM when processing the j-th noisy sample image.

[0011] Preferably, the deep features are the features output from the penultimate layer when the Update Parameter Model (DSM) processes the input image.

[0012] Preferably, the classification loss is the standard cross-entropy loss; the joint loss is calculated by weighted summation of the migration loss and the binary classification loss.

[0013] Preferably, the gradient of the joint loss with respect to the noise is calculated, and the noise is updated using the projected gradient descent method.

[0014] Preferably, the fixed parameter model is a deep neural network trained adversarially; the updatable parameter model is a deep neural network with a different architecture or initialization than the fixed parameter model.

[0015] Preferably, the final noise is added element-wise to the images in the original image dataset to obtain the protected image dataset.

[0016] A privacy protection system for image data, characterized in that it includes a data acquisition module, a hybrid proxy module, a joint loss optimization module, and a protection module; The data acquisition module is used to acquire the original image dataset to be protected and its corresponding labels; The hybrid proxy module includes a fixed parameter model, an updatable parameter model, and a class-center aggregation module; The joint loss optimization module is used to collect a batch of images from the original image dataset, add noise to them, train the updatable parameter model, and iteratively update the noise to obtain the final noise. Each update method is as follows: a batch of images is collected from the original image dataset, noise is added to them, and they are input into the fixed parameter model and the trained updatable parameter model respectively. Then, the classification loss generated by the fixed parameter model and the classification loss generated by the trained updatable parameter model are calculated based on the predicted output results and corresponding labels. The class center aggregation module calculates the transferability loss based on the deep features output by the trained updatable parameter model. The noise is updated based on the joint loss calculated from the transferability loss and the two classification losses. The protection module is used to protect the images in the original image dataset using the final noise.

[0017] A computing device, characterized in that it comprises: a processor and a memory storing a computer program, wherein the computer program, when run by the processor, executes the method described above.

[0018] A computer-readable storage medium, characterized in that it stores instructions that, when executed on a computer, cause the computer to perform the above-described method.

[0019] The beneficial effects achieved by this invention are as follows: 1. By using a hybrid proxy model framework, especially by introducing a fixed-parameter model trained against adversarial forces, the generated noise can effectively resist various mainstream defense methods such as adversarial training and data augmentation (such as CutMix and Mixup), and the protection effect is long-lasting and stable.

[0020] 2. By explicitly constraining the structure of noise in the feature space through the class center aggregation module, the generated noise exhibits good generalization ability across models and datasets. Noise generated on one model / dataset can still maintain high effectiveness on other unknown models / datasets.

[0021] 3. During the optimization process, the visual impact of noise can be strictly controlled through the noise infinity norm constraint to ensure that the protected image still has good visual quality and does not affect normal browsing. Attached Figure Description

[0022] Figure 1 This is a flowchart of the method of the present invention.

[0023] Figure 2 This is a schematic diagram of the shared image data privacy protection method of the present invention.

[0024] Figure 3 This is a flowchart of the shared image data privacy protection method of the present invention.

[0025] Figure 4 This is a schematic diagram illustrating the principle of iteratively updating model parameters and noise in a method for protecting privacy of shared image data.

[0026] Figure 5 This is a system diagram of the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described in detail below with reference to the accompanying drawings and specific experimental data.

[0028] Firstly, the present invention provides an image privacy protection method, the detailed process and mechanism of which are as follows: S1. Obtain the original image dataset to be protected. The input for this step is a series of original images to be protected. Its corresponding tags This includes initial noise. The data can be a collection of images in any format. Typically, images undergo preprocessing operations such as standardization.

[0029] S2. Construct a hybrid proxy module. This step builds a joint optimization environment consisting of three complementary functionalities: a fixed-parameter model, an updatable-parameter model, and a class-centric aggregation module. Static Surrogate Model (SSM): This model uses a deep neural network trained adversarially (e.g., a ResNet-18 pre-trained on the target dataset using PGD). The weights of this model are used throughout the subsequent noise optimization process. Completely frozen, its high-dimensional feature space provides a robust representation of the semantic content of the image.

[0030] Dynamic Surrogate Model (DSM): This model can be a deep neural network with a different architecture or initialization than the fixed-parameter model (e.g., a ResNet-18 trained without adversarial training or a VGG network). Its weights are updated during the noise optimization process. It will be continuously updated. The model weight updates reflect the model's adaptation to noise during training.

[0031] Class-Center Aggregation Module (CCAM): This is a computational module with no independent parameters. It receives the penultimate layer feature output from the updatable parametric model. The core function of this module is to calculate and maintain a dynamic class prototype feature vector, and to calculate the transferability loss based on this vector. .

[0032] S3. Design a joint optimization objective function and construct a joint loss optimization module. The core innovation of this invention lies in designing a multi-task joint loss function to guide the noise generation process: in, , These are the weighting coefficients.

[0033] DSM Loss Items This is the standard cross-entropy loss, calculated on an updatable parameter model for a noisy image. The prediction and its actual label The error between them.

[0034] Here, CE stands for Cross-Entropy Loss, a common classification loss function. Its optimization direction is to minimize the model's classification loss, causing the model to prioritize fitting noise rather than real image features during training.

[0035] SSM loss item This aims to improve resistance to noise defenses. It achieves this by utilizing information from a pre-trained fixed-parameter surrogate model (SSM). It is a fixed-parameter model for adding noise to the input image. The predicted labels obtained after processing are compared with the correct labels to calculate the cross-entropy loss (CE). This loss term widens the distance between the noisy image and the original image in the robust model (SSM) feature space, forcing the noise to have a semantic-level shift. This feature-level shift makes the noise more difficult to eliminate with standard or adversarial training.

[0036] Migratory loss term This is calculated by the class center aggregation module and is designed to shape the generalization structure of the noise: in, For a set of categories, It is a category The number of samples, It is a category The feature centers of the noisy samples (dynamically calculated). This represents the updatable parametric model (DSM) for the input image ( The features output from the penultimate layer during processing, For the i-th sample image of category c, for Noise added; This is the second-to-last layer output feature of the updatable parametric model DSM when processing the j-th noisy sample image. The first term minimizes the intra-class distance, making the noise patterns of samples of the same class converge; the second term maximizes the inter-class center distance, separating the noise patterns of different classes. This "cohesive-outer-sparse" structure gives the noise patterns clear class discriminativeness, making it easier to transfer to other models that work based on similar discriminative features.

[0037] S4. Generate the final noise through alternating optimization. This step uses an iterative optimization strategy to generate noise. : 1) Model update: Fix the current noise Sample a batch from the dataset and calculate The weights of the Distributed Updatable Parameter Model (DSM) are updated via backpropagation. Several steps (e.g., 5-10 steps). This allows the DSM to quickly adapt to the noise, thus providing a more accurate gradient to update the noise itself in the next step.

[0038] 2) Noise Update: Fix the weight parameters of all proxy models and calculate the joint loss. Regarding noise gradient The noise is updated using the projected gradient descent method: in, Step size, This indicates that noise is projected onto a predefined constraint space. Inside, for example - Norm: ,in It is a small value (such as 8 / 255) to ensure that the noise is imperceptible.

[0039] 3) Repeat the above iterative optimization strategy until the noise converges or the preset number of iterations is reached, to obtain the final optimized unlearnable noise. .

[0040] S5. Generate and publish the protected image. Remove the final noise. Compared with the original image dataset Adding elements together yields the protected image dataset. This dataset can be securely published or shared.

[0041] In a second aspect, the present invention provides an electronic device for implementing the above-described method, comprising at least one processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement all the steps of the above-described method.

[0042] Thirdly, the present invention provides a computer-readable storage medium (such as a USB flash drive, hard disk, optical disk, or server storage space) storing a computer program thereon, which, when executed by a processor, implements all the steps of the above-described method.

[0043] like Figure 1 As shown, the present invention provides a method for protecting the privacy of image data, the steps of which include: Obtain the original image dataset to be protected and its corresponding labels; Construct a hybrid proxy module, including a fixed parameter model, an updatable parameter model, and a class-centric aggregation module; A batch of images are collected from the original image dataset and noise is added to them to train the updatable parameter model; The noise is iteratively updated to obtain the final noise. Each update involves: collecting a batch of images from the original image dataset, adding noise to them, and inputting them into a fixed-parameter model and a trained, updatable-parameter model, respectively. Then, based on the predicted output and corresponding labels, the classification loss generated by the fixed-parameter model and the classification loss generated by the trained, updatable-parameter model are calculated. The class center aggregation module calculates the transferability loss based on the deep features output by the trained, updatable-parameter model. The noise is then updated based on the joint loss calculated from the transferability loss and the two classification losses. The images in the original image dataset are protected using the final noise.

[0044] An optional embodiment of the present invention specifically discloses a privacy protection method for sharing image data, the principle and process of which are as follows: Figure 2 and Figure 3 As shown.

[0045] This embodiment uses the CIFAR-10 dataset as an example to explain in detail the implementation process of this method.

[0046] 1. Model Building: Static Surrogate Model (SSM): ResNet-18 was chosen as the basic architecture. First, the model was pre-trained on a clean CIFAR-10 training set using the projective gradient descent (PGD) adversarial training method. The key parameter was set as follows: perturbation radius... Step length Number of iterations After training, a model robust to adversarial perturbations is obtained. In the subsequent optimization process involving the generation of unlearnable noise, all parameters of this model are then used. Freezed, no longer updating.

[0047] Dynamic Surrogate Model (DSM): To simulate the diversity of attacker models, this embodiment selects a model different from the SSM architecture—the VGG-11 network—as the DSM. Its weights... Random initialization is performed using a standard normal distribution to ensure a difference from the initial feature space of the SSM. During noise optimization, the parameters of the DSM are dynamically updated.

[0048] The Class-Center Aggregation Module (CCAM) operates on the feature layer (second to last) before the global average pooling layer of the Distributed Modeling Model (DSM). This layer outputs a 512-dimensional feature vector. For a training batch, this module dynamically calculates the mean of the feature vectors corresponding to all samples of each class in the batch, which serves as the temporary prototype center for that class. .

[0049] 2. Definition of Joint Optimization Objective The joint optimization objective function used in this embodiment The definition is as follows, which precisely corresponds to the core formula in the claims: The specific calculation methods for each item of loss are as follows: DSM loss ( ): Standard cross-entropy loss is used.

[0050] This loss directly drives noise. Correct classification of interference DSM.

[0051] SSM loss ( This is one of the key innovations of this embodiment, which utilizes the feature information of both SSM and DSM.

[0052] in, This represents the features extracted from the penultimate layer of the SSM. This represents the features extracted from the penultimate layer of the DSM. Hyperparameters Set to 0.1. The first term forces the noisy image to be far away from the original image in the robust SSM feature space, ensuring that the perturbation produces a semantic-level shift; the second term encourages them to be close in the dynamic DSM feature space, making the noise more covert and harder to filter out by the ordinary training process.

[0053] Migratory loss ( ): Calculated using CCAM for structured noise.

[0054] in, The number of categories in the batch. It is a category The number of samples, The balancing factor is set to 0.5. The first term minimizes the intra-class feature variance, making the noise patterns of samples within the same class similar; the second term maximizes the inter-class center distance, making the noise patterns of different classes significantly different.

[0055] Hyperparameter settings: In this embodiment, the tradeoff hyperparameter is initialized as follows: .

[0056] The total loss function is defined as: Where CE is the cross-entropy loss. During initialization, set... .

[0057] 3. Noise generation and alternation optimization: like Figure 4 As shown, an iterative alternating optimization strategy is used to generate noise. and apply constraint( To ensure imperceptibility.

[0058] 1) Initialization: For each image in the CIFAR-10 training set Initialize a random noise ,in .

[0059] 2) Noise Update: Fix the model parameters of SSM and DSM. For a batch of images... and its current noise : Calculate joint loss Regarding noise gradient .

[0060] Update the noise using sign gradient descent: Among them, step size .

[0061] Project the updated noise onto - Within the norm sphere, ensure visual imperceptibility: 3) Model update: Fix the latest noise in the current batch. Using noisy data Performing an updateable parametric model (DSM) The training follows standard stochastic gradient descent (SGD) steps. The learning rate is set to... This step allows the DSM to quickly adapt to the new noise conditions.

[0062] 4) Iteration: Noise updates and model updates are repeated until the preset total number of iterations is reached. At this time, the noise It is considered to have converged to the optimal value.

[0063] 4. Generate a protected image After optimization, the optimal noise set for the CIFAR-10 training set is obtained. For any CIFAR-10 image to be protected Its protected version Generates using simple element-wise addition: .

[0064] like Figure 5 As shown, an optional embodiment of the present invention provides an image data privacy protection system, characterized in that it includes a data acquisition module, a hybrid proxy module, a joint loss optimization module, and a protection module; The data acquisition module is used to acquire the original image dataset to be protected and its corresponding labels; The hybrid proxy module includes a fixed parameter model, an updatable parameter model, and a class-center aggregation module; The joint loss optimization module is used to collect a batch of images from the original image dataset, add noise to them, train the updatable parameter model, and iteratively update the noise to obtain the final noise. Each update method is as follows: a batch of images is collected from the original image dataset, noise is added to them, and they are input into the fixed parameter model and the trained updatable parameter model respectively. Then, the classification loss generated by the fixed parameter model and the classification loss generated by the trained updatable parameter model are calculated based on the predicted output results and corresponding labels. The class center aggregation module calculates the transferability loss based on the deep features output by the trained updatable parameter model. The noise is updated based on the joint loss calculated from the transferability loss and the two classification losses. The protection module is used to protect the images in the original image dataset using the final noise.

[0065] Experimental verification: To verify the effectiveness of the method in this embodiment, the following experiments were conducted: Validation test: Novel VGG-11, ResNet-50, and other models were trained using the generated, unlearnable CIFAR-10 training set. Compared to models trained on clean data, accuracy dropped from approximately 93% to below 15%.

[0066] Robustness test: During training, the attacking model was subjected to PGD adversarial training (ρ=4 / 255, 8 / 255). Experiments show that the data protected by the method of this invention can still suppress the model accuracy to below 60%, while the accuracy of traditional methods can recover to over 80% under adversarial training.

[0067] Transferability testing: Noise generated on CIFAR-10 was directly applied to the SVHN dataset. The model trained using this noisy SVHN dataset showed a test accuracy decrease of over 50% compared to using clean data, demonstrating good cross-dataset transferability.

[0068] The above descriptions are merely embodiments of the present invention and do not limit the scope of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention. Similarly, any equivalent structural or procedural transformations made using the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are also included within the scope of protection claimed by the present invention.

Claims

1. A method for protecting the privacy of image data, comprising the following steps: Obtain the original image dataset to be protected and its corresponding labels; Construct a hybrid proxy module, including a fixed parameter model, an updatable parameter model, and a class-centric aggregation module; A batch of images are collected from the original image dataset and noise is added to them to train the updatable parameter model; The noise is iteratively updated to obtain the final noise. Each update involves: collecting a batch of images from the original image dataset, adding noise to them, and inputting them into a fixed-parameter model and a trained, updatable-parameter model, respectively. Then, based on the predicted output and corresponding labels, the classification loss generated by the fixed-parameter model and the classification loss generated by the trained, updatable-parameter model are calculated. The class center aggregation module calculates the transferability loss based on the deep features output by the trained, updatable-parameter model. The noise is then updated based on the joint loss calculated from the transferability loss and the two classification losses. The images in the original image dataset are protected using the final noise.

2. The method according to claim 1, characterized in that, The migration loss is ;in, For a set of categories, It is a category The number of samples, It is a category The feature centers of the noisy samples For the updatable parameter model DSM, the input image ( The deep features output during processing. For the i-th sample image of category c, for Noise added; It is the deep feature output by the updatable parameter model DSM when processing the j-th noisy sample image.

3. The method according to claim 1 or 2, characterized in that, The deep features are the features output from the penultimate layer when the Update Parametric Model (DSM) processes the input image.

4. The method according to claim 3, characterized in that, The classification loss is the standard cross-entropy loss; the joint loss is calculated by weighted summation of the migration loss and the two-classification loss.

5. The method according to claim 1, characterized in that, Calculate the gradient of the joint loss with respect to the noise, and update the noise using the projective gradient descent method.

6. The method according to claim 1, characterized in that, The fixed-parameter model is a deep neural network trained adversarially; the updatable-parameter model is a deep neural network with a different architecture or initialization than the fixed-parameter model.

7. The method according to claim 1, characterized in that, The final noise is added element-wise to the images in the original image dataset to obtain the protected image dataset.

8. A privacy protection system for image data, characterized in that, It includes a data acquisition module, a hybrid agent module, a joint loss optimization module, and a protection module; The data acquisition module is used to acquire the original image dataset to be protected and its corresponding labels; The hybrid proxy module includes a fixed parameter model, an updatable parameter model, and a class-center aggregation module; The joint loss optimization module is used to collect a batch of images from the original image dataset, add noise to them, train the updatable parameter model, and iteratively update the noise to obtain the final noise. Each update method is as follows: a batch of images is collected from the original image dataset, noise is added to them, and they are input into the fixed parameter model and the trained updatable parameter model respectively. Then, the classification loss generated by the fixed parameter model and the classification loss generated by the trained updatable parameter model are calculated based on the predicted output results and corresponding labels. The class center aggregation module calculates the transferability loss based on the deep features output by the trained updatable parameter model. The noise is updated based on the joint loss calculated from the transferability loss and the two classification losses. The protection module is used to protect the images in the original image dataset using the final noise.

9. A computing device, characterized in that, include: A processor, a memory storing a computer program, wherein the computer program, when executed by the processor, performs the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, A storage instruction that, when executed on a computer, causes the computer to perform the method as described in any one of claims 1 to 7.

Images