Image privacy protection method and system for resisting federated gradient transpose attack, and medium
Through shadow model confrontation technology and noise processing, the problems of image area privacy leakage differences and defense imbalance in the existing technology are solved, effective protection of client image data and balanced main task performance, and calculation overhead and concurrency are optimized. sex.
Patent Information
- Application Number
- CN202411025977.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-29
- Publication Date
- 2024-11-15
AI Technical Summary
The existing methods to combat federal gradient transpose attacks fail to effectively consider the privacy leakage differences in image areas, resulting in uneven defense levels, affecting the performance of the main task, and failing to effectively protect the privacy of customer image data.
Using shadow model adversarial technology, shadow model is constructed by generating adversarial networks, pre-training the weights and hidden variables of the shadow model, simulate potential attackers, add relative noise at the sample level, and use histogram equalization and gradient activation map to weaken the noise in the foreground area , obtain absolute noise, used to train the initial task model and achieve effective protection of image data.
It realizes effective protection of client image data, reduces the impact of the noise-added process on the main task, achieves better performance and privacy protection balance, defends against potential strong opponents, optimizes computing overhead, and improves federal training Concurrency.
Smart Images

Figure CN118968217A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of federated learning technology, and in particular to an image privacy protection method, system and medium for counteracting federated gradient transposition attacks. Background Art
[0002] Federated learning is a training paradigm that protects customer data privacy. Compared with traditional centralized deep learning, it does not require the transmission of original data, but is based on the model update information uploaded to the server by each customer, namely gradients and auxiliary information. The updated global model obtained by federated aggregation has similar task prediction capabilities to the model obtained by centralized deep learning training.
[0003] In the context of federated learning, gradient transposition attacks can reconstruct the client's local original data based on the gradient and auxiliary information uploaded by the client to the server, thus causing serious privacy leakage, which violates the original intention of federated learning to protect local data. In existing research on defense against gradient transposition attacks, the focus is mainly on processing the gradient. In order to protect the original data from being reconstructed by gradient transposition attacks, the gradient is generally subjected to operations such as noise addition, truncation, and sparseness.
[0004] However, existing methods do not take into account the differences in privacy leakage in different image regions when defending against gradients, resulting in the areas that are susceptible to reconstruction attacks being too weak in defense, while the content that is not easy to reconstruct is over-protected, which impairs the performance of the main task and fails to achieve effective protection of customer image data. Summary of the invention
[0005] The purpose of this application is to provide an image privacy protection method, system and medium against federated gradient transposition attacks, thereby achieving effective protection of client image data.
[0006] To achieve the above objectives, this application provides the following solutions:
[0007] In a first aspect, the present application provides an image privacy protection method against a federated gradient transposition attack, the image privacy protection method against a federated gradient transposition attack comprising:
[0008] Determine the initial task model for federated learning;
[0009] The initial task model is pre-trained using a local data set of the client to obtain a pseudo-update task model; the local data set includes a plurality of original images to be protected;
[0010] Pre-training a shadow model constructed based on a generative adversarial network using a public data set to obtain a pre-trained shadow model; the public data set and the local data set have the same data type;
[0011] Randomly initialize a latent variable for each original image in the local data set based on a normal distribution, and use the pre-trained shadow model to update each initialized latent variable to obtain a plurality of updated latent variables;
[0012] The weights of the pre-trained shadow model mapping layer and all updated hidden variables are fixed, and the weights of the pre-trained shadow model synthesis layer are adjusted with the gradient and statistical information of the pseudo-update task model as the target to obtain the updated shadow model; the statistical information includes at least: the mean and variance of the batch normalization layer;
[0013] Inputting the updated latent variables corresponding to the original image in the local data set into the updated shadow model to obtain a reconstructed image, and determining the relative noise based on the original image and the reconstructed image;
[0014] After performing histogram equalization on the relative noise and reducing the foreground area noise in sequence, the absolute noise is obtained;
[0015] The initial task model is trained using the local data set to which the absolute noise is added to obtain a real update task model, and the gradient and statistical information of the real update task model are uploaded to a server; the gradient and statistical information of the real update task model are used for subsequent federated aggregation.
[0016] Optionally, after performing histogram equalization on the relative noise and reducing the foreground area noise in sequence, obtaining the absolute noise specifically includes:
[0017] Performing histogram equalization on the relative noise to obtain balanced noise;
[0018] Determine a gradient activation map according to the pseudo-update task model, and use the gradient activation map to weaken the influence of the equalization noise on the foreground area to obtain gradient activation noise;
[0019] The gradient activation noise is reduced based on the current training state of the initial task model in the federated learning to obtain the absolute noise.
[0020] Optionally, the calculation formula of the relative noise is:
[0021]
[0022]
[0023] Where N 1 is the relative noise, σ(·) is the SoftMax function, F MSE represents the difference between the reconstructed image and the original image, T is the temperature coefficient, x i is the i-th pixel in the original image, is the i-th pixel in the reconstructed image, H and W are the height and width of the image respectively, x j is the jth pixel in the original image.
[0024] Optionally, the calculation formula of the equalization noise is:
[0025] N 2 =σ(G(N 1 ));
[0026]
[0027] Where N 2 is the balanced noise, σ(·) is the SoftMax function, G(·) represents histogram equalization, N 1 is the relative noise, L is the total number of gray levels of the image, p(r) is the probability density function of the gray level r, and a and b represent the gray range of the image.
[0028] Optionally, the calculation formula of the gradient activation noise is:
[0029] N 3 =N 2 -α CAM sign(N 2 )·F CAM ;
[0030] Where N 3 is the gradient activation noise, N 2 is the balanced noise, sign(·) is the sign function, α CAM is the coefficient of the sign function, F CAM is the gradient activation map.
[0031] Optionally, the calculation formula of the absolute noise is:
[0032]
[0033] Where N is the absolute noise, N 3 is the gradient activation noise, x is the original image, α N is a hyperparameter that controls the absolute amount of noise, r is the total number of training rounds of the initial task function, and R is the current number of training rounds of the initial task function.
[0034] Optionally, after training the initial task model using the local data set to which the absolute noise is added to obtain a true update task model, and uploading the gradient and statistical information of the true update task model to a server, the method further includes:
[0035] Obtain the gradient and statistical information of the real update task model uploaded to the server by each client;
[0036] The global model is aggregated based on the gradients and statistical information of all real update task models and sent to each client.
[0037] In a second aspect, the present application also provides a computer system, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the image privacy protection method against federated gradient transposition attacks.
[0038] In a third aspect, the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image privacy protection method against federated gradient transposition attacks.
[0039] According to the specific embodiments provided in this application, this application discloses the following technical effects:
[0040] This application mainly defends against potential powerful adversaries by means of shadow model confrontation. In order to efficiently simulate the reconstruction of low-frequency information of the data by the adversary during training, the weights and hidden variable inputs of the shadow model are pre-trained based on the public data set before the start of federated training. During federated training, the adversary's attack on high-frequency information is further simulated by updating the shadow model, and based on this attack result, sample-level relative noise is explainedly added, and histogram equalization technology is used to avoid attack shortcuts provided by excessive noise intensity in different areas. In order to reduce the impact of the noise addition process on the main task and achieve a better balance between performance and privacy protection, the absolute noise is obtained by reducing the noise in the foreground area, and the absolute noise is added to the local data set, focusing on protecting areas in the image that are more vulnerable to reconstruction attacks, thereby achieving effective protection of client image data. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0042] Figure 1 A flowchart of an image privacy protection method against a federated gradient transposition attack provided in an embodiment of the present application;
[0043] Figure 2 A practical application flow chart of image privacy protection provided by an embodiment of the present application;
[0044] Figure 3 This is a diagram of the internal structure of a computer system provided in an embodiment of the present application. DETAILED DESCRIPTION
[0045] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0046] The purpose of this application is to provide an image privacy protection method, system and medium against federated gradient transposition attacks, thereby achieving effective protection of client image data.
[0047] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.
[0048] Example 1
[0049] like Figure 1 As shown, this embodiment provides an image privacy protection method against a federated gradient transposition attack, specifically comprising:
[0050] Step S1: Determine the initial task model for federated learning.
[0051] In this embodiment, each client needs to obtain the weight of the task model used for federated learning from the server in advance, and define the task model obtained by each client as the initial task model. Among them, the task model is mainly used to perform the main task (the main task in this embodiment refers to image recognition and classification).
[0052] Step S2: Pre-training the initial task model using the local data set of the client to obtain a pseudo-update task model; the local data set includes a plurality of original images to be protected.
[0053] In this embodiment, the local data set refers to the image data that the client needs to protect. The initial task model is first pre-trained using these real image data. The model gradient and statistical information obtained after pre-training (mainly the mean and variance of the model batch normalization layer) will not be uploaded to the server. Therefore, the pre-trained initial task model is also called a pseudo-update task model.
[0054] Step S3: Use a public data set to pre-train the shadow model constructed based on the generative adversarial network to obtain a pre-trained shadow model; the data type of the public data set is the same as that of the local data set.
[0055] In this embodiment, the difference between the public dataset and the local dataset is that the image data actually stored in the two is different, but the data types of the two are similar. Only in this way can the effectiveness of shadow model pre-training be guaranteed. In addition, pre-training some network components before federated training can avoid the problems of low computational efficiency and low parallelism during federated training, and pre-training of the shadow model can provide faster convergence initialization weights for the process of updating the shadow model.
[0056] Step S4: randomly initialize a latent variable for each original image in the local data set based on the normal distribution, and use the pre-trained shadow model to update each initialized latent variable to obtain multiple updated latent variables.
[0057] In this embodiment, the pre-trained shadow model includes a mapping layer and a synthesis layer; the input of the mapping layer is the hidden variable corresponding to each original image, and the mapping layer is used to map the hidden variable to a semantic feature; the input of the synthesis layer is the semantic feature, and the output is a reconstructed image corresponding to the original image. The updated hidden variable is used as the input of the pre-trained shadow model, which can enhance the shadow simulation effect for low-frequency information and reduce the computational cost of adjusting the pre-trained shadow model during federated learning.
[0058] Specifically, the update process of the above hidden variables can be described as:
[0059]
[0060] Where x is the original image in the local dataset, is the reconstructed image output by the pre-trained shadow model, z is a hidden variable, where means “where”, and f s (·) is the pre-trained shadow model, w s are the weights of the pre-trained shadow model.
[0061] Step S5: Fix the weights of the mapping layer of the pre-trained shadow model and all updated latent variables, take the gradient and statistical information of the pseudo-update task model as the target, adjust the weights of the synthesis layer of the pre-trained shadow model, and obtain the updated shadow model.
[0062] In this embodiment, the above process is mainly to improve the ability of the pre-trained shadow model to reconstruct local data, that is, to enhance the simulation attack effect on high-frequency details. The calculation process of the loss function in the pre-trained shadow model adjustment process is as follows:
[0063]
[0064] In the formula, The weights of the pre-trained shadow model synthesis layer, w s' is to update the weight of the shadow model, α and β are the hyperparameters of the loss function in the process of adjusting the pre-trained shadow model, L Grad is the gradient difference between the pseudo-update task model of the reconstructed image input and the pseudo-update task model of the original image input, L BN is the difference between the statistics of the pseudo-update task model for reconstructing the image input and the statistics of the batch normalization layer of the pseudo-update task model, L t (·) is the loss function of the pseudo-update task model (in this embodiment, it is the cross entropy loss function), w t is the weight of the pseudo-update task model, f t (·) is the pseudo-update task model, y is the label of the original image, l is the layer index of the network, μ l (·), Respectively represent the mean and variance obtained by updating the batch normalization layer with reconstructed data, BN l (·) represents batch normalization, mean and variance represent the mean and variance of the batch normalization layer of the pseudo-update task model, respectively.
[0065] Step S6: input the updated latent variables corresponding to the original image in the local data set into the updated shadow model to obtain a reconstructed image, and determine the relative noise based on the original image and the reconstructed image.
[0066] In this embodiment, by calculating the difference between the reconstructed image and the original image, the customer is informed of the area and relative degree of privacy leakage of local data, thereby determining the relative noise and achieving privacy protection at the sample level. The calculation formula of relative noise is:
[0067]
[0068] Where N 1 is the relative noise, σ(·) is the SoftMax function used for normalization, F MSE is the difference between the reconstructed image and the original image (also known as the mean square error reconstruction map), T is the temperature coefficient, x i is the i-th pixel in the original image, is the i-th pixel in the reconstructed image, H and W are the height and width of the image respectively, x j is the jth pixel in the original image.
[0069] Step S7: After performing histogram equalization on the relative noise and reducing the foreground area noise in sequence, the absolute noise is obtained.
[0070] In this embodiment, step S7 specifically includes:
[0071] Step S71: performing histogram equalization on the relative noise to obtain balanced noise.
[0072] Specifically, in order to avoid the weak visual robustness of the noise map, that is, when the noise amplitude difference in different regions is too large, it provides a shortcut related to sensitive information for potential adversaries, it is necessary to 1 Use histogram equalization, that is:
[0073] N 2 =σ(G(N 1 ));
[0074]
[0075] Where N 2 To equalize the noise, G(·) represents histogram equalization, L is the total number of gray levels of the image (i.e., 256), p(r) represents the probability density function with respect to r, and a and b represent the gray range of the image (corresponding to 0 and 255, respectively).
[0076] Step S72: Determine a gradient activation map according to the pseudo-update task model, and use the gradient activation map to weaken the influence of the equalization noise on the foreground area to obtain gradient activation noise; wherein the foreground area is the area in the original image that determines the main task prediction.
[0077] Specifically, after determining the relative noise at the image level, it is also necessary to determine the absolute noise that is ultimately added to the original image. In order to reduce the degree of noise addition in the area related to the main task and avoid excessive negative impact on the performance of the main task, Grad-CAM++ is used to obtain the pixel-level saliency map F of the middle layer of the network. CAM (i.e., gradient activation map), which represents the degree of foreground activation related to the main task. After upsampling it to the original image size, select the area with the largest activation value to represent the significant foreground that the model focuses on, which is used to reduce the impact of the added noise on the foreground of the original image. The calculation formula of the gradient activation noise is:
[0078] N 3 =N 2 -α CAM sign(N 2 )·F CAM ;
[0079] Where N 3 is the gradient activation noise, sign(·) is the sign function, α CAM are the coefficients of the sign function.
[0080] Step S73: reducing the gradient activation noise based on the training state of the initial task model in the federated learning to obtain the absolute noise; the training state mainly refers to the number of training rounds of the initial task model.
[0081] Specifically, considering that the change in the gradient of the task model during training will gradually decrease with the iteration rounds, thus hindering the reconstruction effectiveness of the gradient transposition attack, the amplitude of the absolute noise can be further reduced based on the current number of training rounds of the task model, that is:
[0082]
[0083] Where N is the absolute noise, α N is a hyperparameter that controls the absolute amount of noise, r is the total number of training rounds of the initial task function, and R is the current number of training rounds of the initial task function.
[0084] Step S8: Use the local data set with absolute noise added to train the initial task model to obtain the real update task model, and upload the gradient and statistical information of the real update task model to the server; the gradient and statistical information of the real update task model are used for subsequent federated aggregation.
[0085] In this embodiment, the above absolute noise is ultimately applied to the original image as additive noise, and the noise image is used to truly update the task model to achieve a fine-grained and explainable privacy protection effect.
[0086] Furthermore, after executing step S8, it is also necessary to obtain the gradient and statistical information of the real update task model uploaded to the server by each client; complete the aggregation of the global model based on the gradient and statistical information of all real update task models, and send it to each client.
[0087] In summary, the image privacy protection method against the federated gradient transposition attack of the present application is based on shadow adversarial technology, and innovatively uses the shadow model constructed by the generative adversarial network to simulate potential attackers, provide guidance for the customer's local defense strategy, and protect the more fragile privacy information in the image in a targeted manner. At the same time, the pre-training operation before the start of the federation also optimizes the computational overhead required for the defense strategy in subsequent steps, and reduces the impact of updating the shadow model on the concurrency of the federation. In addition, the noise weakening operation based on the gradient activation map can stabilize the feature fitting effect of the main task and achieve a better balance between task performance and privacy protection. The present application has a strong practical application value and strong comprehensive performance.
[0088] Example 2
[0089] like Figure 2 As shown, this embodiment provides a practical application scenario of an image privacy protection method against a federated gradient transposition attack, as follows:
[0090] In the medical field, in order to protect the privacy and security of patient data, federated training can be used. Without sharing data, the model update information obtained from data training can be uploaded to the server to achieve the ultimate goal of multi-party cooperation in training a universal global model. However, existing studies have shown that even if only model update information is communicated, gradient transposition attacks can be used to recover the original data used for training, so corresponding defense strategies need to be explored.
[0091] Assume that ChestXRay14 and ChestXRay are used as the public dataset and local dataset respectively. Both datasets are chest X-ray image data, which are used for binary classification task training of pneumonia or not. It is expected that the shadow model can efficiently provide privacy protection guidance for each local data of the customer.
[0092] First, based on the training paradigm of StyleGAN3, a public dataset is used to pre-train the generative adversarial network to obtain a pre-trained shadow model. After that, each customer optimizes the hidden variables corresponding to each local data based on the pre-trained shadow model and the mean square error loss. After starting the federated training, the customer first inputs each batch of local data into the task model, and trains the task model based on the cross entropy loss to obtain the pseudo-updated gradient and the statistical information of the batch normalization layer. Then, the pre-generated hidden variables are used as the input of the pre-trained shadow model, the mapping layer of the pre-trained shadow model is fixed, and the image synthesis layer of the pre-trained shadow model is fine-tuned based on the pseudo-updated gradient and statistical information matching loss. After the pre-trained shadow model is fine-tuned, the relative noise is calculated by the output reconstructed image and the image data of the batch, and histogram equalization is performed. At the same time, based on the pseudo-updated task model and the gradient activation map, the foreground activation degree related to the main task can be obtained, which is used to determine the absolute noise together with the relative noise. After adding absolute noise to the original data, the task model is trained again, and the gradient and statistical information of the real updated task model obtained after training are uploaded to the server for global aggregation of the federated process, so as to ensure that the privacy of the original data is difficult to be reconstructed by gradient transposition attacks. After the server sends the aggregated global model as the initialization of the local model, the next round of local training is repeated. The following are the specific execution steps:
[0093] The first step is to pre-train a shadow model based on a public dataset with a data type similar to that of the main task (a binary classification task of whether or not pneumonia is present in this embodiment) (chest X-ray images in this embodiment), to provide initialization weights that are easier to converge and improve the efficiency of subsequent fine-tuning.
[0094] The second step is to randomly initialize a corresponding latent variable based on the normal distribution for each piece of local data of the customer as the input of the pre-trained shadow model. The weights obtained by pre-training the shadow model are fixed, and these latent variables are updated one by one. The goal is to make the reconstructed image of the shadow model closer to the local data as a whole.
[0095] In the third step, after the start of federated learning, the client first backs up the initial weights of the task model (in this embodiment, the input is an image and the output is a classification prediction) locally, and then uses the local data set to pseudo-update the task model (it will not be used as model update information uploaded to the server) to obtain the gradient of the pseudo-updated task model and the statistical information of the batch normalization layer.
[0096] In the fourth step, the weights of the mapping layer of the pre-trained shadow model and the latent variables updated in the second step are fixed, and the image synthesis layer of the pre-trained shadow model is fine-tuned to obtain the adjusted weights of the pre-trained shadow model, thereby enhancing the simulated attack effect of the shadow model on high-frequency details.
[0097] In the fifth step, the shadow model is updated and outputs a reconstructed image, which is used to calculate the difference with the real image. This will prompt customers to identify the areas and relative degree of privacy leakage in their local data, thereby determining the relative noise and achieving privacy protection at the sample level.
[0098] In the sixth step, when calculating the relative noise based on the shadow model, the noise amplitude in different areas may be too different, providing a shortcut to sensitive information for potential adversaries. 1 Use histogram equalization to reduce the noise areas with excessive values to avoid this problem.
[0099] In the seventh step, in order to reduce the degree of noise in the area related to the main task, Grad-CAM++ is used to obtain the pixel-level attention map of the middle layer of the network. After upsampling it to the original image size, the partial area with the largest activation value is selected to represent the significant foreground that the model pays attention to, which is used to reduce the impact of the added noise on the foreground of the original image.
[0100] In the eighth step, the effectiveness of the gradient transposition attack usually decreases as the training progresses, and the amplitude of the absolute noise can be further reduced based on the current training state. Finally, the global model is aggregated based on the model gradients and statistical information uploaded by the customer, and sent to the customers participating in the federation for subsequent rounds of local training.
[0101] Example 3
[0102] This embodiment provides a computer system, which may be a server or a terminal, and its internal structure diagram may be as follows: Figure 3As shown. The computer system includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer system is used to provide computing and control capabilities. The memory of the computer system includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer system is used to store video tag processing data. The input / output interface of the computer system is used to exchange information between the processor and an external device. The communication interface of the computer system is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an image privacy protection method against a federated gradient transposition attack is implemented.
[0103] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer system to which the solution of the present application is applied. The specific computer system may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.
[0104] Example 4
[0105] This embodiment provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.
[0106] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).
[0107] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.
[0108] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.
[0109] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.
Claims
1. An image privacy protection method against federated gradient transposition attacks, It is characterized in that The image privacy protection method against the federated gradient transposition attack includes: Determine the initial task model for federated learning; The initial task model is pre-trained using a local data set of the client to obtain a pseudo-update task model; the local data set includes a plurality of original images to be protected; Pre-training a shadow model constructed based on a generative adversarial network using a public data set to obtain a pre-trained shadow model; the public data set and the local data set have the same data type; Randomly initialize a latent variable for each original image in the local data set based on a normal distribution, and use the pre-trained shadow model to update each initialized latent variable to obtain a plurality of updated latent variables; The weights of the pre-trained shadow model mapping layer and all updated hidden variables are fixed, and the weights of the pre-trained shadow model synthesis layer are adjusted with the gradient and statistical information of the pseudo-update task model as the target to obtain the updated shadow model; the statistical information includes at least: the mean and variance of the batch normalization layer; Inputting the updated latent variables corresponding to the original image in the local data set into the updated shadow model to obtain a reconstructed image, and determining the relative noise based on the original image and the reconstructed image; After performing histogram equalization on the relative noise and reducing the foreground area noise in sequence, the absolute noise is obtained; The initial task model is trained using the local data set to which the absolute noise is added to obtain a real update task model, and the gradient and statistical information of the real update task model are uploaded to a server; the gradient and statistical information of the real update task model are used for subsequent federated aggregation.
2. The image privacy protection method against the federated gradient transposition attack according to claim 1, It is characterized in that After performing histogram equalization on the relative noise and reducing the foreground area noise in sequence, the absolute noise is obtained, which specifically includes: Performing histogram equalization on the relative noise to obtain balanced noise; Determine a gradient activation map according to the pseudo-update task model, and use the gradient activation map to weaken the influence of the equalization noise on the foreground area to obtain gradient activation noise; The gradient activation noise is reduced based on the current training state of the initial task model in the federated learning to obtain the absolute noise.
3. The image privacy protection method against the federated gradient transposition attack according to claim 1, It is characterized in that The calculation formula of the relative noise is: Where N 1 is the relative noise, σ(·) is the SoftMax function, F MSE represents the difference between the reconstructed image and the original image, T is the temperature coefficient, x i is the i-th pixel in the original image, is the i-th pixel in the reconstructed image, H and W are the height and width of the image respectively, x j is the jth pixel in the original image.
4. The image privacy protection method against the federated gradient transposition attack according to claim 2, It is characterized in that The calculation formula of the equalization noise is: N 2 =σ(G(N 1 )); Where N 2 is the balanced noise, σ(·) is the SoftMax function, G(·) represents histogram equalization, N 1 is the relative noise, L is the total number of gray levels of the image, p(r) is the probability density function of the gray level r, and a and b represent the gray range of the image.
5. The image privacy protection method against the federated gradient transposition attack according to claim 2, It is characterized in that The calculation formula of the gradient activation noise is: N 3 =N 2 -α CAM sign(N 2 )·F CAM ; Where N 3 is the gradient activation noise, N 2 is the balanced noise, sign(·) is the sign function, α CAM is the coefficient of the sign function, F CAM is the gradient activation map.
6. The image privacy protection method against the federated gradient transposition attack according to claim 2, It is characterized in that The calculation formula of the absolute noise is: Where N is the absolute noise, N 3 is the gradient activation noise, x is the original image, α N is a hyperparameter that controls the absolute amount of noise, r is the total number of training rounds of the initial task function, and R is the current number of training rounds of the initial task function.
7. The image privacy protection method against the federated gradient transposition attack according to claim 1, It is characterized in that After training the initial task model with the local data set to which the absolute noise is added to obtain the real update task model, and uploading the gradient and statistical information of the real update task model to the server, the method further includes: Obtain the gradient and statistical information of the real update task model uploaded by each client to the server; The global model is aggregated based on the gradients and statistical information of all real update task models and sent to each client.
8. A computer system, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the image privacy protection method against federated gradient transposition attacks according to any one of claims 1 to 7.
9. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the image privacy protection method against federated gradient transposition attacks described in any one of claims 1 to 7 is implemented.
Citation Information
Patent Citations
Federated learning anti-reasoning attack privacy protection method based on double perturbation
CN115481431A
Adaptive DPWGAN training method and system based on federated learning
CN117473558A
Privacy protection method based on generative adversarial network and gradient smoothing, program, equipment and storage medium
CN118296650A
