Diffusion model-based gradient inversion attack method and system in federated learning

Abstract

The invention belongs to the technical field of federated learning, and discloses a method and system for carrying out gradient inversion attack on privacy image data of a user in a federated learning system, and the method comprises the steps: firstly, pre-training a diffusion model and a coder-decoder based on a small amount of prior data similar to the user data through a honest and curious federated server; through gradient information uploaded by a user, a server calculates to obtain a user image tag and a high-dimensional feature. The server constructs a diffusion sampling schedule and a decreasing iteration schedule. And in the progressive decrease round, the generation process of diffusion model sampling is disassembled, gradient matching optimization is carried out on an intermediate result generated by sampling of each time step, and a feature search space is limited through high-dimensional feature regularization until all time steps are completed. According to the method, the gradient inversion attack capability on the user image in the federated learning scene is effectively improved, the similarity and visual quality of the reconstructed image are improved, and more user privacy features are revealed.

Description

Technical Field

[0001] The present invention belongs to the technical field of federated learning, and in particular, relates to a gradient inversion attack method and system based on a diffusion model in federated learning. Background Art

[0002] In the era of big data, the further development of the Internet of Things and artificial intelligence requires a large amount of data as a basic support. The traditional method is often to aggregate all data to a central server for unified processing and training. However, in reality, due to restrictions such as data sources and privacy protocols, data is often scattered, small, and fragmented. For example, on many personal devices, users are reluctant to share personal data, while banks, hospitals and other institutions are not allowed to disclose user data due to the particularity of the industry. The resulting "data island" problem also makes the traditional centralized data aggregation processing method increasingly unfeasible. In such an environment, Federated Learning (FL) as a new distributed learning paradigm has attracted widespread attention. Federated Learning adheres to the concept of "data remains unchanged, model parameters move", so that private data is only saved and used locally by users, and users participating in the federated system only need to upload the relevant parameters of the local model to the central server. The central server implements the parameter update of the central model by aggregating the model parameters, and the server then distributes the updated model parameters to each user terminal, and each user uses the new model parameters as the initial parameters for the next round of training. This method not only protects the privacy data security of users, but also realizes high-performance model training in distributed scenarios.

[0003] However, a problem that has been proven is that the model gradients uploaded by users during federated learning can also cause the privacy information of user data to be leaked. In a federated system that performs image classification tasks, an honest and curious federated central server can obtain the gradient information uploaded by users and the training model of the federated system, randomly initialize a virtual data, and send it to the federated model to obtain a virtual reconstruction gradient. By iteratively optimizing the virtual data by distance matching the virtual gradient and the real gradient, the virtual data can be finally optimized to be reconstructed data that is highly similar to the user's private data. However, the recovery effect of the existing gradient attack method in federated learning is not stable. The simple gradient distance matching optimization method is greatly affected by the randomness of the virtual data initialization. Different initialization results may lead to different gradient search directions, which in turn leads to the failure of inversion reconstruction. In addition, when the resolution of the user's private image increases, the optimizer needs to optimize more parameters and the image search space increases, which makes the method that relies only on gradient matching easy to fall into the local optimal solution. Even if it converges successfully, it may reconstruct a meaningless image. The existing technology improves this problem by introducing the Generative Adversarial Networks (GAN) model. The server trains the generator model with a small number of prior images, and in the inversion and reconstruction process, the generator's prior constraints are used to search the optimization space to reconstruct the user's private image. However, the existing technical methods based on the GAN model have problems such as low fidelity of the reconstructed image and poor visual quality. It is impossible to fully reveal the privacy of the user, and the privacy of the obtained data cannot be fully, clearly and with high quality. Therefore, the existing technology introduces a series of more relaxed ideal assumptions, such as the BatchNorm layer parameters of the user in training are known, or the normalization layer of the federated model is removed to increase the privacy information contained in the gradient. These assumptions make the gradient information easier to attack, thereby improving the effect of the gradient inversion reconstruction attack.

[0004] Through the above analysis, the problems and defects of the prior art are as follows:

[0005] (1) Gradient inversion attacks will gradually fail as the resolution of the user's local image data in the federated learning system increases, and reconstruction will become impossible. Existing technologies rely on some idealized and unrealistic assumptions in the gradient inversion and reconstruction process, which does not meet the requirements of real technical scenarios.

[0006] (2) Existing technologies have difficulty in achieving a good balance between the fidelity of reconstructed images and the visual quality of the images themselves. Problems such as low clarity, high noise, and poor quality of reconstructed images make it impossible to accurately and efficiently reveal the image privacy characteristics of federated learning users. Summary of the invention

[0007] In view of the problems existing in the prior art, the present invention provides a gradient inversion attack method and system based on a diffusion model in federated learning.

[0008] The present invention is implemented as follows: a gradient inversion attack method based on a diffusion model in federated learning uses an honest and curious federal server, a diffusion model and an encoder-decoder pre-trained based on a small amount of prior data similar to user data; the diffusion model is introduced into the gradient inversion reconstruction attack task of pixel-level recovery, and a decreasing iterative generation and optimization reconstruction attack architecture is creatively proposed. In the iteration, the internal structure of the diffusion model is disassembled, and the generation result of each sampling process in the middle is optimized by gradient distance matching. At the same time, the high-dimensional features of the original image are obtained by inverting the gradient, and feature matching regularization is proposed to further limit the search space of the reconstructed features. The characteristic that the predicted noise intensity from the diffusion model time step from back to front is from strong to weak, and the starting time step is selected decreasingly to start iterative generation optimization to achieve gradual recovery of the privacy image from contour to detail; the generated reconstructed image is subjected to non-optimization denoising and image enhancement processing to improve the image details and quality; finally, the reconstructed image is compared with the original image to evaluate their similarity and quality.

[0009] Furthermore, the gradient inversion attack method based on the diffusion model in the federated learning specifically includes the following steps:

[0010] In the first step, the server pre-trains the encoder-decoder and diffusion model; the user performs local training and uploads the updated model parameter gradients to the server.

[0011] In the second step, the server obtains the user's shared gradient and federated task model, and performs gradient inversion reconstruction with recursive generation and optimization.

[0012] In the third step, the reconstructed image is post-processed and image quality enhanced without optimization to further improve the image clarity and quality.

[0013] The fourth step is to evaluate the similarity between the reconstructed image and the original image, and compare and verify the reconstruction quality and performance with the benchmark algorithm.

[0014] Further, the first step specifically includes:

[0015] The honest and curious servers have a small amount of prior image data that is approximately identically distributed with the federated users, and use this prior data to pre-train the encoder-decoder and diffusion model.

[0016] The user uses the ResNet-18 network locally to perform image classification tasks and calculates the local model gradient during the training process, which can be expressed as:

[0017]

[0018] Where f is the network model that performs image classification tasks in the federated system, ω is the user's local model parameter, and x and y are the image data and corresponding labels used for local training by the user. The federated user may implement a privacy protection strategy locally, perform transformations such as compression and noise addition on the local model gradient, generate a shared gradient g = T (F (x)), and upload it to the federated server. T (.) represents the defense processing operation that the federated user may use.

[0019] Furthermore, the second step of performing the decremental generation and optimized gradient inversion reconstruction specifically includes:

[0020] Step 1: Analyze and infer user gradients. After obtaining the model gradients shared by users, the server first extracts the fully connected layer gradients, and then identifies the weight gradient of the i-th row of the fully connected layer (FC) in the user shared gradients. Whether it is negative, infer the category to which the real label of the user image belongs.

[0021] Then according to the weight and bias gradient of the fully connected layer in the shared gradient and Perform inverse calculation to approximately obtain the user's fully connected layer input z during model training FC , that is, the high-dimensional image features extracted from the original data through the classification network can be expressed as:

[0022]

[0023] Where i represents the i-th row of the vector, and k represents the row where the true label is located. The loss function for training federated users, y i Output result of row i of the fully connected layer of the federated model, and Represent the weight and bias gradient corresponding to the i-th category respectively.

[0024] Step 2: The server constructs a schedule, including a sampling schedule S_Sample when the diffusion model performs sampling to generate features, and a schedule S_Start for the start time of each round required for performing decremental iterative denoising. In the first step, the diffusion model whose implicit feature space has been determined by pre-training has a total of T time steps {1, 2, ..., T}. The server determines that a total of M decremental iterative rounds need to be performed, and N sampling times are required each time the diffusion model is executed for denoising. It is required that M must be divisible by N and 0<M≤N. Then two schedules S_Start={T, T-(T / / M), ..., T-(M-1)(T / / M)} are constructed respectively, indicating the starting time step when using decremental iterative generation M times, and S_Sample={1+(N-1)(T / / N), 1+(N-2)(T / / N), ..., 1}, indicating the sampling time step followed by the diffusion model when performing denoising in each iteration, where / / indicates that the division result is rounded down to an integer.

[0025] Step 3: Perform decremental iterative generation and optimization in the potential feature space, and reconstruct the image by gradually disassembling and optimizing the intermediate results of the diffusion model. Specifically, randomly initialize a feature image z and start the decremental denoising generation process. Take the start time S_Start from the timetable S_Start in sequence n Together with the feature image z, it is used as the diffusion model input for the current decreasing round, and n represents the list index. The diffusion model accepts the input, refers to the sampling schedule S_Sample, finds the index k, and sets S_Sample k =S_Start n , from S_Sample k Start, and execute the subsequent time steps S_Sample in the S_Sample schedule in sequence k To S_Sample N-1 Sampling is performed, in which the process of noise removal at any time step t can be expressed as:

[0026]

[0027] Where t = S_Sample j , p = S_Sample j+1 , follow the schedule S_Sample for sampling and denoising, z t is the input feature map of the current time step, and ε θ (z t ,t) represents the noise prediction of the Unet network in the diffusion model at time step t. According to the input image and time, the noise that should be subtracted from the current image at this time is predicted. α is the model hyperparameter that retains the image ratio of the previous step. m tis a randomly generated noise that follows a standard normal distribution, which can increase the diversity of generated images, σ t is a hyperparameter that controls the noise weight.

[0028] Then for the current time step t = S_Sample j Generated intermediate results z *Use decoder D to restore the image from the latent feature space to the image space, that is, x * = D(z). The intermediate result x of the current reconstruction * Send it to the federated model f to obtain the reconstruction gradient Where f is the model for performing image classification tasks in the federated system, and ω is the model parameter. Then the l2 distance between the reconstructed gradient g' and the true gradient uploaded by the user is calculated as:

[0029]

[0030] Remove the fully connected layer of the federated model and convert x * Input the federated model f1 without the fully connected layer to obtain the high-dimensional features z of the reconstructed image FC * =f1(x * ), calculate the real image high-dimensional feature z FC and reconstruct the high-dimensional features z of the image FC * The l2 distance between them is:

[0031] d(z FC * ,z FC )=||z FC * -z FC || 2

[0032] Set the current time step t = S_Sample j The input z t As the optimization object of the optimizer, the optimization target of the current time step can be determined as:

[0033] z t =argmin z (d(g',g)+d(z FC * ,z FC ))

[0034] Further expansion can be expressed in detail as follows:

[0035] z t =argmin z (d(F(D(LDM(z *,t))),g)+d(f1(D(LDM(z * ,t))),z FC ))

[0036] After the iterative optimization of Epoch rounds, the optimized result z t is sampled again through diffusion to obtain the output z at the current time step p :

[0037]

[0038] Taking the output z at the current time step p as the input for the next time step, repeat the above steps after entering the next time step until all the time steps in the time schedule S_Sample are denoised to obtain the reconstructed feature map of the current round

[0039] Read the next moment S_Start in the S_Start time schedule n+1 , and again as the input for a new round, traverse to find the index value h such that S_Start n+1 = S_Sample h , start denoising backward from S_Sample h , repeat the steps within the above iteration M times until the traversal of the S_Start time schedule ends to obtain the final reconstructed feature map z r . r The reconstructed feature map z

[0040] Furthermore, the image post-processing and enhancement in the third step specifically include:

[0041] Step 1: Denoise using the unoptimized DDPM diffusion technique, and take the obtained reconstructed image as the initial input x U , U is the specified starting time step for DDPM denoising. Use the unoptimized DDPM to denoise x U U times, and its time schedule is S = {U, U - 1,..., 1}. The denoising at the S j time step (1 < j < U) can be expressed as:

[0042]

[0043] is the input at time step S j , where α is the model hyperparameter that retains the image ratio of the previous step, m tis a randomly generated noise that follows a standard normal distribution, which can increase the diversity of generated images. is a hyperparameter that controls the noise weight. After careful denoising adjustment by the DDPM denoising U round, the reconstructed image x is obtained. r as output of this step.

[0044] Step 2: x r Perform image enhancement, including contrast enhancement, saturation enhancement, and sharpening. Contrast enhancement is achieved by adjusting the value of each pixel in the image, which can be expressed as x r '=β1(x r -mean)+mean, β1 is the hyperparameter contrast enhancement factor, and mean is the average brightness value of the image.

[0045] To implement saturation enhancement, the image is first converted from the RGB color space to the HSV color space. The enhancement of saturation V can be expressed as: V'=β2V, where V is the saturation value of the original image and β2 is the saturation enhancement factor parameter.

[0046] Image sharpening can be expressed as x r '=x r +β3(x r -G(x r )),G(x r ) is the image after Gaussian blur processing, and β3 is the sharpening factor coefficient. After further detail optimization processing, the final output is x r =x r 'This is the complete reconstruction result of this technical method.

[0047] Furthermore, the similarity between the reconstructed image and the original image and the reconstruction performance verification in the fourth step specifically include:

[0048] Read the user's original image x o and reconstruct the image x r , calculate the mean square error value MSE of the two, which can be expressed as Where N represents the total number of pixels in the image and i is the i-th pixel. The peak signal-to-noise ratio is calculated by MSE. Further, the structural similarity index SSIM is calculated, expressed as SSIM(x o ,x r ).

[0049] In addition, the pre-trained deep neural network VGG is used to extract the features of the original image and the reconstructed image, and the Euclidean distance between these features is calculated to represent the image similarity, which can be expressed as φ l is the feature extraction function of the neural network at layer l, and L is the number of feature layers.

[0050] The similarity and image visual quality of the reconstructed image and the user-reconstructed image in the federated system are determined by comprehensively comparing the four indicators of MSE, PSNR, LPIPS and SSIM to evaluate the performance of the gradient inversion reconstruction method.

[0051] Another object of the present invention is to provide a gradient inversion attack method and system based on a diffusion model in federated learning, wherein the gradient inversion attack method based on a diffusion model in federated learning includes:

[0052] Model pre-training module, used by the server to pre-train the encoder-decoder and diffusion model using prior data before reconstruction begins, to limit image generation during the reconstruction process;

[0053] The regressive generation and optimization reconstruction module is used to reconstruct user images and labels based on user shared gradients. A regressive denoising and reconstruction method is proposed to disassemble the generation process of the diffusion model, gradually optimize the gradient matching loss of the intermediate results, and reduce the feature search space under the constraint of feature regularization loss. Finally, the starting time step of the diffusion model is iteratively reduced to gradually achieve image reconstruction from outline to detail.

[0054] Image post-processing and enhancement module, used to further enhance the image details and image quality after optimization and reconstruction, perform non-optimized diffusion model denoising, and enhance image saturation, contrast, and sharpness;

[0055] The similarity and performance evaluation module is used to determine the similarity between the reconstructed image and the original image of the federated user and the performance of the gradient inversion image reconstruction method, and calculate the mean square error, peak signal-to-noise ratio, perceptual similarity, structural similarity index value between the reconstructed image and the original image, as well as the comprehensive similarity index proposed by this method.

[0056] Another object of the present invention is to provide a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the gradient inversion attack method based on the diffusion model in federated learning.

[0057] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to execute the steps of the gradient inversion attack method based on the diffusion model in the federated learning.

[0058] Another object of the present invention is to provide an information data processing terminal, which is used to implement the gradient inversion attack system based on the diffusion model in the federated learning.

[0059] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:

[0060] First, in the scenario of centralized federated learning with honest and curious servers, the present invention can achieve high-quality gradient inversion attacks on the image data of federated users performing image classification tasks. The present invention introduces an implicit diffusion model and its encoder-decoder to compress the gradient inversion reconstruction process from the image space to the feature space for generation and optimization, reducing the amount of parameter calculation. At the same time, the internal structure of the diffusion model is disassembled, and the intermediate results of each sampling step are optimized by gradient distance matching. At the same time, the image search space is further restricted by high-dimensional feature regularization restrictions, and the intermediate results are optimized, so that the reconstruction accuracy and reconstruction quality of larger resolution images in the gradient inversion attack scenario in federated learning are effectively improved.

[0061] The present invention proposes a gradient inversion attack method for decreasing iterative generation optimization. Not only does the diffusion model start with decreasing starting time steps in multiple iterations, but also the diffusion model executes the diffusion sampling schedule decreasingly within each iteration to generate denoising for the feature map. The different characteristics of the prediction noise generated by the diffusion model as the time step moves backward from strong to weak are utilized. By executing decreasing denoising, the gradient inversion attack can follow the step-by-step reconstruction process from the main body contour to the image details, and the problem of phase shift of the image main body position of the reconstructed image is solved, so that the reconstructed image has a clear contour and a clear main body, and the image main body displacement and ghosting problems will not occur.

[0062] The present invention designs a regularization method based on high-dimensional image features. Compared with the past technology, the present invention can achieve approximate recovery from gradients to high-dimensional features of user images by only using the shared gradients of federal users without introducing additional ideal assumptions, and uses this as a standard to calculate the l2 distance loss with the reconstructed image features. In the optimization process of gradient distance matching, the high-dimensional feature matching of the reconstructed image is further restricted, the similarity of the reconstructed image details is enhanced, and the image reconstruction quality is improved.

[0063] The present invention adopts an image post-processing and enhancement module, utilizes the denoising characteristics of the diffusion model, and uses a diffusion model with a small step number to perform non-optimization denoising on the subtractive reconstruction results, further removes noise points in the reconstructed image, optimizes image details, and fine-tunes the image contrast, saturation, and sharpness. It removes noise, optimizes details, and improves the overall visual quality of the reconstructed image without damaging the existing reconstructed image features, thereby achieving a good trade-off between the similarity of the reconstructed image and the visual quality.

[0064] Second, the performance of gradient inversion reconstruction attack is improved: the present invention introduces an implicit diffusion model and its encoder-decoder, and proposes a decreasing iterative generation and optimization reconstruction architecture, gradually generates and optimizes the intermediate results of diffusion model sampling, and restricts the feature search space based on image feature regularization, which can achieve high-fidelity and visual quality reconstruction of user images with larger resolution.

[0065] Stronger applicability and robustness: The present invention takes into account the situation where federal users adopt certain privacy protection measures. Through multiple rounds of diffusion model iterative denoising, it can maximize the use of prior knowledge to generate and optimize the reconstructed image, and reduce the impact of federal users' privacy protection measures on gradient inversion and reconstruction. In addition, due to the existence of the post-processing denoising module, the privacy attack performance is enhanced. Even if federal users adopt defense measures such as gradient noise addition, gradient clipping or gradient compression against shared gradients, the present invention can still achieve image reconstruction of good quality.

[0066] In summary, the present invention proposes a gradient inversion attack method based on a diffusion model in federated learning for centralized federated learning scenarios, which can realize the privacy image reconstruction process from contour to detail, significantly improve the fidelity and visual quality of the reconstructed image, and is also highly applicable and robust to situations where defense strategies are used.

[0067] Third, as auxiliary evidence of the inventiveness of the claims of the present invention, it is also reflected in the following important aspects:

[0068] (1) The expected benefits and commercial value of the technical solution of the present invention after transformation are:

[0069] The technical solution of the present invention provides a gradient inversion solution based on a diffusion model for the gradient inversion attack problem in federated learning. After the technical solution of the present invention is converted into a specific product, the expected benefits and commercial value are mainly reflected in the following aspects:

[0070] Evaluation and testing of privacy and security mechanisms: When developing a new federated learning system, the technical method of the present invention can be used to evaluate the privacy leakage risk of the federated system and the effectiveness of the existing security mechanism to ensure the security and robustness of the federated system and prevent potential data leakage, thereby improving the reliability and market competitiveness of the federated system.

[0071] Promoting the development of defensive countermeasures: The technical solution of the present invention reveals that user privacy in federated learning can be reconstructed with a high degree of accuracy by an honest and curious server, which will further attract the attention of the federated learning system to the protection of federated user privacy and promote the generation of more effective privacy protection and defense mechanisms.

[0072] (2) The technical solution of the present invention fills the technical gap in the industry at home and abroad:

[0073] The technical solution of the present invention provides a solution based on a diffusion model for the problem of gradient inversion attack in a centralized federated learning system, which effectively fills the technical gap in the industry at home and abroad, and is specifically reflected in the following aspects:

[0074] Compressing the generation and reconstruction process into latent feature space: In the generator-assisted gradient inversion reconstruction attack scenario, as the image resolution increases, the number of optimization parameters increases, and it is difficult to find a meaningful optimal solution, which is an important reason for the reconstruction failure. The present invention uses a codec to compress the image into the latent feature space for the generation process. This strategy significantly reduces the number of parameters and computations required for the reconstruction process, and provides a method for gradient inversion attacks in federated learning.

[0075] Gradient inversion reconstruction method with decreasing iterative generation and optimization: In the gradient inversion attack of the federated learning system, there are challenges such as displacement or ghosting of the main body of the reconstructed image, blurred details, and many noise points. The present invention utilizes the characteristics of the diffusion model that the predicted noise intensity from the back to the front time step is from strong to weak. By iteratively executing the diffusion process, decreasing the starting time step in each iteration, and disassembling and optimizing the intermediate results sampled at each time step, the gradual optimization reconstruction from the main contour of the image to the local details is realized, the problem of displacement of the main body of the image is solved, the similarity and clarity of the reconstructed image are improved, the noise is reduced, and the quality of the reconstructed image is further improved, so that the reconstructed image has a clear contour, a clear main body, and complete details. The present invention fills the technical gap in this scenario.

[0076] Regularization constraint of gradient inversion reconstruction based on high-dimensional features: The present invention considers analyzing and calculating the approximate user fully connected layer input from the user shared gradient, that is, the high-dimensional features of the original image processed by the image classification network, and uses the Euclidean distance between the high-dimensional features of the original image and the high-dimensional features of the reconstructed image as the optimization target regularization term, which can help restore the original image features, limit the image search space, improve the similarity between the reconstructed image and the original image, and enhance the performance of the gradient inversion reconstruction attack.

[0077] Non-optimized post-processing denoising and image enhancement strategy: This paper uses the non-optimized DDPM diffusion model to perform small-step denoising on the reconstructed image, and then fine-tunes the image contrast, saturation, and sharpness, which can further enhance the image details without losing the existing features of the reconstructed image, thereby improving the visual quality of the image itself. This non-optimized post-processing strategy makes full use of the denoising characteristics of the diffusion model. There is no such research and application in the prior art that can significantly improve the quality of the reconstructed image without destroying the existing reconstructed features.

[0078] Fourth, the technical problems of the prior art solved by the technical solution of the present invention and the significant technical progress achieved:

[0079] 1) Solving the privacy protection challenge: In federated learning, the privacy protection of user data is an important issue. The gradient inversion attack method based on the diffusion model in federated learning proposed in this paper can effectively invert the user's image and its corresponding label, although these images are not directly exposed. By analyzing and reconstructing the shared gradient, the attacker can obtain the originally protected user data, thus revealing the shortcomings of the existing privacy protection mechanism in the face of complex attacks.

[0080] 2) Improve the accuracy of the attack method: The method in the present invention utilizes the characteristics of the implicit diffusion model for gradient inversion reconstruction. Compared with the traditional gradient inversion attack method, the implicit diffusion model provides stronger reconstruction capabilities and higher image quality. Through detailed steps, including the timetable setting of the diffusion model and the decremental de-noising process, as well as the gradual optimization of the intermediate results of the diffusion model, the user image can be restored more accurately, thereby improving the success rate of the attack and the image reconstruction quality.

[0081] 3) Enhanced image reconstruction effect: During the image reconstruction process, the present invention uses non-optimized post-processing and image quality enhancement techniques. These techniques include DDPM diffusion denoising and image color enhancement, which further improve the clarity and quality of the reconstructed image. This meticulous post-processing method not only improves the image reconstruction effect, but also makes the reconstructed image closer to the real image, enhancing the actual effect of the attack.

[0082] 4) Provide comprehensive evaluation criteria: In order to evaluate the quality of the reconstructed image, the present invention proposes a set of comprehensive evaluation criteria, including mean square error (MSE), peak signal-to-noise ratio (PSNR), structural similarity index (SSIM) and feature similarity (LPIPS). These evaluation indicators can comprehensively reflect the quality and similarity of image reconstruction, ensuring the effectiveness and performance of the attack method. This multi-dimensional evaluation method improves the accurate judgment of image reconstruction quality, reflecting the significant progress of the present invention in the field of image analysis.

[0083] 5) Application effect of optimization algorithm: The present invention improves the image reconstruction process through detailed mathematical models and algorithm steps, such as noise prediction in the diffusion model and optimization objective function. The optimization using noise prediction in the diffusion model makes the reconstructed image more realistic and detailed, while the implementation of the optimization algorithm ensures computational efficiency and reconstruction accuracy.

[0084] In summary, this paper significantly improves the accuracy and effect of gradient inversion reconstruction attacks in federated learning by introducing implicit diffusion models and comprehensive evaluation methods, reveals the potential weaknesses of privacy protection technology, and provides improved data protection suggestions. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] Figure 1 is a flow chart of a gradient inversion attack method based on a diffusion model in federated learning provided by an embodiment of the present invention;

[0086] Figure 2 Schematic diagram of the structure of the gradient inversion attack method based on the diffusion model in federated learning provided by the embodiment of the present invention

[0087] Figure 3 It is a system flow chart of a gradient inversion attack method based on a diffusion model in federated learning provided by an embodiment of the present invention;

[0088] Figure 4 It is a scenario and architecture diagram of a gradient inversion attack method based on a diffusion model in federated learning provided by an embodiment of the present invention;

[0089] Figure 5 It is a comparison chart of experimental effects of the gradient inversion attack method based on the diffusion model in federated learning provided by an embodiment of the present invention and the previous technical methods; DETAILED DESCRIPTION

[0090] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0091] Specific embodiment 1: Medical imaging data protection

[0092] In the healthcare industry, federated learning is used to train medical image analysis models, such as cancer detection and disease diagnosis models. This approach allows hospitals and medical institutions to jointly train models without sharing sensitive patient data.

[0093] 1) Step 1-Pre-trained model preparation: The Medical Data Center pre-trains the encoder-decoder and diffusion model, and uses public medical imaging datasets for preliminary training.

[0094] 2) Step 2 - Local model training: Each hospital uses its own medical imaging data for local training, calculates the gradient parameters of the local model, and uploads it to the central server.

[0095] 3) Step 3 - Inversion Reconstruction Attack: After receiving the model gradients uploaded by each hospital, the central server uses the gradient inversion reconstruction attack method based on the implicit diffusion model to reconstruct the original medical images and their labels of the hospital. This attack can reveal the patient's sensitive information and challenge the hospital's data protection measures.

[0096] 4) Step 4 - Image quality assessment: The reconstructed medical images are post-processed and image quality enhanced to improve the clarity and accuracy of the image, and the quality of the reconstructed image is evaluated by indicators such as mean square error (MSE) and peak signal-to-noise ratio (PSNR).

[0097] 5) Application of results: Identify and resolve privacy leakage issues in medical data protection, improve the hospital's security measures in data sharing and protection, and ensure that patient privacy is not leaked.

[0098] Specific Example 2: Financial Data Analysis

[0099] In the financial sector, federated learning is used to jointly train risk prediction models, such as credit scoring and fraud detection models. This approach can use data from multiple financial institutions to train models while protecting user privacy.

[0100] 1) Step 1-Pre-model training preparation: Financial institutions use simulated financial transaction datasets to pre-train the encoder-decoder and diffusion model to ensure the accuracy of the latent diffusion model.

[0101] 2) Step 2 - Local Data Training: Each financial institution uses its own customer transaction data to train the model, calculate the updated gradient parameters, and upload them to the central server.

[0102] 3) Step 3 - Inversion and reconstruction attack: After receiving the gradients uploaded by each financial institution, the central server applies the gradient inversion and reconstruction attack method based on the implicit diffusion model of decreasing iterative generation and optimization to perform gradient inversion and reconstruct the customer's financial transaction data and related labels. This attack can reveal the transaction behavior of financial customers and challenge the protection measures of financial data.

[0103] 4) Step 4 - Result processing and verification: The reconstructed financial data is post-processed and quality enhanced without optimization. The accuracy and quality of data reconstruction are evaluated using methods such as mean square error (MSE) and peak signal-to-noise ratio (PSNR), and the effects of different attack methods are compared.

[0104] 5) Application of results: By verifying the effectiveness of the attack method, potential risks in financial data protection can be identified, and the data security measures of financial institutions can be improved to ensure that customers' financial information and transaction behaviors are not leaked.

[0105] These two examples demonstrate the application of gradient inversion attacks based on diffusion models in the medical and financial fields, and promote the development of data privacy protection technology by revealing and improving data protection measures.

[0106] like Figure 1 As shown, the gradient inversion attack method based on the diffusion model in federated learning provided by the embodiment of the present invention includes the following steps:

[0107] S101, the server uses prior data to pre-train the diffusion model and encoder-decoder, and the user uses local data for training and sends the local model gradient to the server.

[0108] S102, the server obtains the user shared gradient and the federated model, performs gradient inversion reconstruction based on the decreasing iterative generation and optimization of the implicit diffusion model, and reconstructs the user image and the corresponding label.

[0109] S103, performing non-optimized denoising post-processing and image quality enhancement tasks on the reconstructed image.

[0110] S104, evaluating the similarity between the reconstructed image and the original image, and comparing and verifying the reconstruction quality and performance with the benchmark algorithm.

[0111] The working principle of the gradient inversion attack method based on the diffusion model in federated learning provided by the embodiment of the present invention can be described in detail as follows:

[0112] 1) Model pre-training and gradient upload: In the initial stage, the server pre-trains the diffusion model and encoder-decoder using a representative prior dataset. This step ensures that the model has sufficient capabilities to process and reconstruct image features. Each federated user trains the model locally using its own dataset and calculates the gradient parameters of the local model. This gradient information is then sent to the central server for further processing.

[0113] 2) Gradient inversion and image reconstruction: After receiving the model gradient uploaded by the user, the central server performs the gradient inversion and reconstruction process. By analyzing the gradients shared by the user, the server analyzes the user data labels and uses the diffusion model to infer the user's original image. This process includes inverting the gradients and bias gradients of the fully connected layer, restoring the high-dimensional features of the image, and using the implicit diffusion model to perform decremental generation, disassembling the intermediate results generated, and finally obtaining a reconstruction result close to the real image through gradient distance matching optimization with high-dimensional feature regularization.

[0114] 3) Post-processing and quality enhancement: To improve the quality of the reconstructed image, the server performs non-optimized denoising post-processing on the generated image. This includes denoising the image using DDPM diffusion technology, and applying image quality enhancement techniques such as contrast enhancement, saturation adjustment, and sharpening to improve the clarity and visual effect of the image.

[0115] 4) Quality evaluation and comparative verification: Finally, the server evaluates the similarity between the reconstructed image and the original image. The quality of the reconstructed image is verified by calculating indicators such as mean square error (MSE), peak signal-to-noise ratio (PSNR), and structural similarity index (SSIM), combined with features extracted by deep neural networks. These evaluation results are used to determine the effectiveness of the attack method and compare it with existing benchmark algorithms to evaluate its performance and room for improvement.

[0116] like Figure 2 As shown, the gradient inversion attack method and system based on the diffusion model in federated learning provided by the embodiment of the present invention include:

[0117] Model pre-training module, used for reconstruction before the attack begins. The server uses a small amount of prior data that is approximately the same distribution as the local images of the federated users to pre-train the diffusion model and encoder-decoder required in the attack.

[0118] The decreasing gradient inversion reconstruction module is used to reconstruct user data labels and images. After obtaining the user's shared gradient, the server uses a decreasing iterative generation and optimization attack method, and uses image high-dimensional feature regularization to further limit the feature search space and restore private image data.

[0119] Image post-processing and enhancement module, used to perform detail denoising and quality optimization enhancement on the reconstructed image, so as to further improve the similarity and visual quality of the reconstructed image;

[0120] The similarity and performance evaluation module is used to compare the similarity between the final reconstructed image result and the user's original image and evaluate the image quality to determine the performance of the gradient inversion reconstruction attack method.

[0121] As a preferred embodiment, Figure 3 As shown, the secure federated learning aggregation method and system in a wireless network provided by the embodiment of the present invention specifically include the following steps:

[0122] In step 1, the honest and curious server has a small amount of prior image data that is approximately identically distributed with the federated users, and uses this prior data to pre-train the encoder-decoder and diffusion model.

[0123] The user uses the ResNet-18 network locally to perform image classification tasks and calculates the local model gradient during the training process, which can be expressed as:

[0124]

[0125] Where f is the network model that performs image classification tasks in the federated system, ω is the user's local model parameter, and x and y are the image data and corresponding labels used for local training by the user. The federated user may implement a privacy protection strategy locally, perform transformations such as compression and noise addition on the local model gradient, generate a shared gradient g = T (F (x)), and upload it to the federated server. T (.) represents the defense processing operation that the federated user may use.

[0126] Step 2: The server obtains the user's shared gradient and federated task model, performs gradient inversion reconstruction, and reconstructs the user image and corresponding label;

[0127] Analyze user gradients and infer them. After obtaining the model gradients shared by users, the server first extracts the gradients of the fully connected layer and identifies the weight gradients of the i-th row of the fully connected layer in the user-shared gradients. Whether it is negative, infer the category to which the real label of the user image belongs.

[0128] Then according to the weight and bias gradient of the fully connected layer in the shared gradient and Perform inverse calculation to approximately obtain the user's fully connected layer input z during model training FC , that is, the high-dimensional image features extracted from the original data through the classification network can be expressed as:

[0129]

[0130] Where i represents the i-th row of the vector, and k represents the row where the true label is located. The loss function for training federated users, y i Output result of row i of the fully connected layer of the federated model, and Represent the weight and bias gradient corresponding to the i-th category respectively.

[0131] Construct a schedule, including a sampling schedule S_Sample when the diffusion model performs sampling generation, and a schedule S_Start for the starting time of each round of backward decrement required for the execution of decremental denoising. In the first step of pre-training, the diffusion model of the implicit feature space has been determined to have a total of T time steps {1, 2, ..., T}. The server determines that a total of M decremental iteration rounds need to be executed, and N sampling times are required each time the diffusion model is executed for denoising. It is required that M must be divisible by N and 0<M≤N. Then, two schedules S_Start={T, T-(T / / M), ..., T-(M-1)(T / / M)} are constructed respectively, indicating the starting time step when M decremental iterations are generated, and S_Sample={1+(N-1)(T / / N), 1+(N-2)(T / / N), ..., 1}, indicating the sampling time step followed by the diffusion model when performing denoising, where / / indicates that the division result is rounded down to an integer.

[0132] Perform decremental denoising in the latent feature space to reconstruct the image, perform external and internal double iterations according to S_Start and S_Sample, take the time step from the S_Start schedule as the starting time, and the initial feature image z as input, from S_Sample k =S_Start n Initially, internal iterative sampling generation is performed according to S_Sample, where the process of denoising generation at any time step t can be expressed as:

[0133]

[0134] Where t = S_Sample j , p = S_Sample j+1 , z t is the input feature map of the current time step, and ε θ (z t ,t) represents the noise prediction of the Unet network in the diffusion model at time step t. According to the input image and time, the noise that should be subtracted from the current image at this time is predicted. α is the model hyperparameter that retains the image ratio of the previous step. m t is a randomly generated noise that follows a standard normal distribution, which can increase the diversity of generated images, σ t is a hyperparameter that controls the noise weight.

[0135] For the current time step t = S_Sample j The generated intermediate result z * Use decoder D to generate image x * = D(z), use the federated model f to obtain the high-dimensional feature z of the fully connected layer input FC * And reconstruct the gradient The l2 distance between the reconstructed gradient g' and the true gradient uploaded by the user is calculated as:

[0136]

[0137] At the same time, calculate the real image high-dimensional features z FC and reconstruct the high-dimensional features z of the image FC * The l2 distance between them is:

[0138] d(z FC * ,z FC )=||z FC * -z FC || 2

[0139] Set the current time step t = S_Sample j The input z t As the optimization object of the optimizer, the optimization target of the current time step can be determined as:

[0140] z t =argmin z (d(g',g)+d(z FC * ,z FC ))

[0141] The optimization result zt is sampled again by diffusion to obtain the output z of the current time step p :

[0142]

[0143] Output z at the current time step p As the input of the next time step, repeat the above steps after entering the next time step until all time steps in the time table S_Sample are de-noised, and the reconstructed feature map of the current round is obtained.

[0144] Read the next S_Start in the S_Start schedule n+1 ,and Together as input again, traverse to find index h and let S_Start n+1 =S_Sample h , from S_Sample h Start back-de-noising, repeat the above steps in the iteration, iterate M times, until the S_Start schedule iteration ends, and obtain the final reconstructed feature map z r . The reconstructed feature map zr After decompression by the decoder, a reconstructed image is obtained

[0145] Step 3: Perform post - processing without optimization and image quality enhancement on the reconstructed image to further improve image clarity and quality;

[0146] Use non - optimized DDPM for x U Denoise U times, and its time schedule is S’ = {U, U - 1,..., 1}. The denoising at the j - th time step (1 < j < U) can be expressed as: j where is the input image at the j - th time step. After fine - grained denoising adjustment of DDPM for U rounds, the reconstructed image x

[0147]

[0148] is the input image at time step S j At the time step, the input image, after fine - grained denoising adjustment of DDPM for U rounds, obtains the reconstructed image x r as the output of this step.

[0149] Perform image enhancement on x r including contrast enhancement, saturation enhancement, and sharpening. Contrast enhancement is achieved by adjusting each pixel value of the image and can be expressed as x r ' = β1(x r - mean)+ mean, where β1 is the hyperparameter contrast enhancement factor and mean is the average brightness value of the image.

[0150] To implement saturation enhancement, first convert the image from the RGB color space to the HSV color space. The enhancement of saturation V can be expressed as: V' = β2V, where V is the saturation value of the original image and β2 is the saturation enhancement factor parameter.

[0151] Image sharpening can be implemented as x r ' = x r + β3(x r - G(x r )), where G(x r ) is the image after Gaussian blur processing, and β3 is the sharpening coefficient. After further detailed optimization processing, the final output x r = x r ' is the complete reconstruction result of the method of the present invention.

[0152] Step 4: Evaluate the similarity between the reconstructed image and the original image, and compare and verify the reconstruction quality and performance with the benchmark algorithm;

[0153] Read the user's original image x o and the reconstructed image x r , and calculate the mean square error value MSE between the two, which can be expressed as Where N represents the total number of pixels in the image and i is the i-th pixel. The peak signal-to-noise ratio is calculated by MSE. Further, the structural similarity index SSIM is calculated, expressed as SSIM(x o ,x r ).

[0154] In addition, the pre-trained deep neural network VGG is used to extract the features of the original image and the reconstructed image, and the Euclidean distance between these features is calculated to represent the image similarity, which can be expressed as φ l is the feature extraction function of the neural network at layer l, and L is the number of feature layers.

[0155] The similarity and image visual quality of the reconstructed image and the user-reconstructed image in the federated system are determined by comprehensively comparing the four indicators of MSE, PSNR, LPIPS and SSIM to evaluate the performance of the gradient inversion reconstruction method.

[0156] In order to prove the creativity and technical value of the technical solution of the present invention, this section provides application examples of the claimed technical solution on specific products or related technologies.

[0157] Example 1: Federated Learning System Security Test

[0158] The invention can be used in a federated learning system that implements image classification tasks to test the security of the federated learning system. In addition, according to the reconstructed image similarity and quality evaluation of the method of the present invention, the privacy leakage degree and security reliability of the federated learning system can be approximately evaluated, and the privacy protection security of the system can be determined before the federated system is officially put into use.

[0159] Example 2: Evaluation of a federal privacy protection algorithm

[0160] This invention is applied to the privacy-preserving algorithm in federated learning, and image data subjected to gradient inversion attack in the privacy-preserving federated learning is reconstructed. The performance and deficiencies of the privacy-preserving algorithm are evaluated based on the reconstruction results, so as to promote the development and improvement of the privacy-preserving algorithm in federated learning.

[0161] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. It can be understood by a person of ordinary skill in the art that the above-mentioned devices and methods can be implemented using computer executable instructions and / or contained in a processor control code, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the carrier medium. The device and its modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, and can also be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0162] The embodiments of the present invention have achieved some positive effects during the development or use process, and indeed have great advantages over the prior art. The following content is described in conjunction with data, charts, etc. of the test process.

[0163] 1. Simulation conditions:

[0164] The simulation experiment of the present invention is carried out on the Linux platform, and the main configuration is: the CPU is Intel (R) Xeon (R) Gold5318Y CPU @ 2.10GHz; the GPU is NVIDIAA40, and the operating system is Ubuntu 20.04.3LTS; the simulation software environment is python 3.8, pytorch2.2.2+cu121, and cuda12.2.

[0165] 2. Simulation content and result analysis

[0166] This simulation experiment uses the method of the present invention and the existing gradient inversion attack method in federated learning, mainly using the generative adversarial network (GAN) model as the image generator in the reconstruction attack, including GGL, GIAS and GIFD for comparison. Figure 4The simulation scenario includes multiple user devices participating in the federal system and a central server. Each federal user contains its own local private image data, and the central server has a small number of images that are approximately identically distributed to the user data. According to different simulation parameters, the gradient inversion attack method based on the diffusion model in the present invention is used for simulation. Unless otherwise specified, the simulation process parameters are as follows: the time step T of the pre-trained diffusion model is set to 1000, the number of decrement iterations M is set to 5, the number of diffusion sampling times N is set to 50, the Adam optimizer is used, the initial learning rate is set to 0.1, and the optimization round Epoch is set to 100. In addition, in the simulation experiment, it is considered that federal users use the CelebA face image dataset scaled to a resolution of 64*64 pixels to perform the task of gender classification. The server uses a small number of training set samples for model pre-training, and the user uses the data in the test set to perform the federated learning task. Through numerical analysis and experimental simulation, the following is obtained: Figure 5 result.

[0167] Figure 5 In the gradient inversion image reconstruction result diagram shown, the gradient inversion attack method based on the diffusion model in federated learning used in the present invention is compared with the attack reconstruction results of the prior art method. It can be seen that the image reconstructed by the GGL method only reveals a very small amount of privacy information such as hair color and background color. Although the image clarity is high and the noise is small, the reconstruction similarity is very low; the privacy features revealed by the GIAS and GIFD reconstruction attacks are significantly increased, and the similarity is further improved, but the reconstructed images have more noise and low clarity. Large areas of noise even cover important image privacy features, resulting in low visual quality of the reconstructed image and difficulty in recognizing image content and detail features; the reconstructed image results obtained by the gradient inversion attack method based on the diffusion model used in the present invention not only more completely reveal the detail features of the original image, but also further improve the similarity with the original image, and the reconstructed image is almost noise-free, and the image visual quality is higher.

[0168] Table 1 is a comparison of the quality of reconstructed images between the gradient inversion attack method based on the diffusion model in federated learning provided by the present invention and the prior art method;

[0169]

[0170] From Table 1, it can be seen that under the four indicators of MSE, PSNR, LPIPS and SSIM for evaluating the similarity between the reconstructed image and the original image and the image quality, the technical method implemented by the example of the present invention is superior to the existing methods. MSE stands for mean square error, and PSNR stands for peak signal-to-noise ratio. These two indicators are based on pixel differences and can intuitively reflect the difference in pixel values ​​between the reconstructed image and the original image. It can be seen that the effect of the technical method of the present invention is better than the three existing methods in terms of pixel-level indicators; LPIPS evaluates the perceptual similarity between the reconstructed image and the original image through a neural network, which is closer to the perceptual difference of human vision to the image. It can be seen that the technical method implemented by the example of the present invention is also better than the three existing methods in terms of semantic perception; SSIM considers the brightness, contrast and structural similarity between the reconstructed image and the original image, which is consistent with the visual perception of the human eye. It can be seen that the technical method implemented by the example of the present invention is also better than the three existing methods in terms of structural perception. In summary, the comparison of the attack reconstructed image and the original image implemented by the technical example of the present invention is better than the existing methods in terms of similarity from pixels to semantic perception, successfully exposing the data privacy of more federal users, and displaying the obtained privacy in a complete image with high visual quality.

[0171] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered by the protection scope of the present invention.

Claims

1. A gradient inversion attack method based on diffusion model in federated learning, characterized in that: The gradient inversion attack method based on the diffusion model in the federated learning is initiated by an honest and curious central server. The server pre-trains the diffusion model and encoder-decoder based on a small amount of prior data similar to the user data. The diffusion model is introduced into the gradient inversion reconstruction attack task of pixel-level recovery, and a reconstruction attack architecture of decreasing iterative generation and optimization is creatively proposed. In the iteration, the internal structure of the diffusion model is disassembled, and the gradient distance matching optimization is performed on the generation result of each intermediate sampling process. At the same time, the high-dimensional features of the original image are obtained by inverting the gradient. Feature matching regularization is proposed to further limit the search space of the reconstructed features. The characteristic that the predicted noise intensity from the diffusion model time step from back to front is from strong to weak, and the starting time step is selected in a decreasing manner for iterative generation optimization to achieve step-by-step reconstruction of the privacy image from contour to detail. The generated reconstructed image is subjected to non-optimized denoising and image enhancement processing to improve the details and quality. Finally, the reconstructed image is compared with the original image to evaluate their similarity and quality.

2. The gradient inversion attack method based on diffusion model in federated learning as described in claim 1, characterized in that: The following steps are involved: In the first step, the federated server pre-trains the encoder-decoder and diffusion model, and the user performs local training and uploads the updated model gradient parameters to the server; In the second step, the server obtains the user's shared gradient and federated task model, performs a gradient inversion attack with decreasing iteration generation and optimization, and reconstructs the user image and corresponding label; The third step is to perform non-optimized post-processing and image quality enhancement on the reconstructed image to further improve the image clarity and quality; The fourth step is to evaluate the similarity between the reconstructed image and the original image, and compare and verify the reconstruction quality and performance with the benchmark algorithm.

3. The gradient inversion attack method based on diffusion model in federated learning as claimed in claim 2, characterized in that: The first step specifically includes: The honest and curious server has a small amount of prior image data that is approximately identically distributed to the federated users. This prior data is used to pre-train the encoder-decoder and diffusion model. The user performs the image classification task locally and calculates the local model gradient during the training process, which can be expressed as: Where f is the model that performs image classification tasks in the federated system, ω is the user's local model parameters, x and y are the image data and corresponding labels used for local training by the user; the federated user may implement privacy protection strategies locally, perform transformations such as compression and noise addition on the local model gradient, generate a shared gradient g = T (F (x)), and upload it to the federated server. T (.) represents the defense processing operations that the federated user may use.

4. The gradient inversion attack method based on diffusion model in federated learning as described in claim 2 is characterized in that: The second step of reconstructing the user image and the corresponding label specifically includes: Step 1: Analyze and infer user gradients. After obtaining the model gradients shared by users, the server first extracts the gradients of the fully connected layer and identifies the weight gradient of the i-th row of the fully connected layer in the user shared gradient. Whether it is negative, infer the category to which the real label of the user image belongs; Then according to the weight and bias gradient of the fully connected layer in the shared gradient and Perform inversion calculation to approximately obtain the input Z of the fully connected layer of the federated model FC , that is, the high-dimensional image features extracted from the original data through the classification network can be expressed as: Where i represents the i-th row of the vector, and k represents the row where the true label is located. The loss function for training federated users, y i Output result of row i of the fully connected layer of the federated model, and Represent the weight and bias gradient corresponding to the i-th category respectively; Step 2: The server constructs a schedule, including a sampling schedule S_Sample when the diffusion model executes sampling generation, and a schedule S_Start of the starting time of each round of backward decrement required for the execution of decremental denoising; in the first step, the diffusion model of the implicit feature space has been determined to have a total of T time steps {1, 2, ..., T} for pre-training; the server determines that a total of M decremental iteration rounds are to be executed, and N samplings are required each time the diffusion model is executed for denoising, requiring M to be divisible by N and 0<M≤N, and then constructs two schedules S_Start={T, T-(T / / M), ..., T-(M-1)(T / / M)}, which represents the starting time step when M decremental iterations are generated, and S_Sample={1+(N-1)(T / / N), 1+(N-2)(T / / N), ..., 1}, which represents the sampling time step followed by the diffusion model when executing denoising, where / / represents rounding down the division result to an integer; Step 3: Perform decremental denoising in the potential feature space to reconstruct the image; specifically, randomly initialize a feature image z and start the decremental denoising generation process; take out the start time S_Start from the timetable S_Start in sequence n and feature image z as the diffusion model input for the current decreasing round; the diffusion model accepts the input, refers to the sampling schedule S_Sample, finds k, and sets S_Sample k =S_Start n , from S_Sample k Start, and execute the subsequent time steps S_Sample in the S_Sample schedule in sequence k To S_Sample N-1 Sampling is performed, in which the process of noise removal at any time step t can be expressed as: Where t = S_Sample j , p = S_Sample j+1 , follow the schedule S_Sample for sampling and denoising, z t is the input feature map of the current time step, and ε θ (z t ,t) represents the noise prediction of the Unet network in the diffusion model at time step t. According to the input image and time, the noise that should be subtracted from the current image at this time is predicted. α is the model parameter that retains the image ratio of the previous step. m t is a noise that follows a standard normal distribution and can increase the diversity of generated images. t is a hyperparameter that controls the noise weight; Then for the current time step t = S_Sample j The generated intermediate result z * Use decoder D to restore the image from the latent feature space to the image space, that is, x * = D(z); the intermediate result x of the current reconstruction * Send it to the federated model f to obtain the reconstruction gradient Where f is the model for performing image classification tasks in the federated system, and ω is the model parameter; then the l2 distance between the reconstructed gradient g' and the true gradient uploaded by the user is calculated as: Remove the fully connected layer of the federated model and convert x * Input the federated model f1 without the fully connected layer to obtain the high-dimensional features z of the reconstructed image FC * =f1(x * ), calculate the real image high-dimensional feature z FC and reconstruct the high-dimensional features z of the image FC * The l2 distance between them is: d(z FC * ,z FC )=||z FC * -z FC || 2 Set the current time step t = S_Sample j The input z t As the optimization object of the optimizer, the optimization target of the current time step can be determined as: z t =argmin z (d(g',g)+d(z FC * ,z FC )) The expansion can be expressed in detail as: z t =argmin z (d(F(D(LDM(z * ))),g)+d(f1(D(LDM(z * ))),z FC )) After the Epoch round of iterative optimization, the optimization result z t After diffusion sampling again, the output z of the current time step is obtained p : Output z at the current time step p As the input of the next time step, repeat the above steps after entering the next time step until all time steps in the time table S_Sample are de-noised, and the reconstructed feature map of the current round is obtained. Read the next S_Start in the S_Start schedule n+1 ,and Take them as input again, traverse and find h so that S_Start i+1 =S_Sample h , from S_Sample h Start back-denoising, repeat the above steps in the iteration, iterate M times, until the S_Start schedule iteration ends, and obtain the final reconstructed feature map z r ; The reconstructed feature map z r Decompressed by the decoder to obtain the reconstructed image 5. The gradient inversion attack method based on diffusion model in federated learning as claimed in claim 2, characterized in that: The third step of performing non-optimized post-processing and image quality enhancement on the reconstructed image specifically includes: Step 1: Denoise the reconstructed image obtained by the unoptimized DDPM diffusion technique and use it as the initial input x where U is the specified starting time step for DDPM denoising. Use the unoptimized DDPM to denoise x U U times, with the time schedule S’ = {U, U - 1,..., 1}. The denoising at the j-th time step (1 < j < U) can be expressed as: U j ​​ in is at time step S j The input image, α is the model parameter that retains the image scale of the previous step, m t is a noise that follows a standard normal distribution and can increase the diversity of generated images. is a hyperparameter that controls the noise weight; after careful denoising adjustment by the DDPM denoising U round, the reconstructed image x is obtained. r As output of this step; Step 2: x r Perform image enhancement, including contrast enhancement, saturation enhancement and sharpening; contrast enhancement is achieved by adjusting the value of each pixel in the image, which can be expressed as x r '=β1(x r -mean)+mean, β1 is the hyperparameter contrast enhancement factor, mean is the average brightness value of the image; To implement saturation enhancement, the image is first converted from RGB color space to HSV color space. The enhancement of saturation V can be expressed as: V' = β2V, where V is the saturation value of the original image and β2 is the saturation enhancement factor parameter; Image sharpening can be expressed as x r '=x r +β3(x r -G(x r )),G(x r ) is the image after Gaussian blur processing, β3 is the sharpening coefficient; after further detail optimization processing, the final output is x r =x r 'This is the complete reconstruction result of this technical method.

6. The gradient inversion attack method based on diffusion model in federated learning as claimed in claim 2, characterized in that: The fourth step evaluates the similarity between the reconstructed image and the original image and compares and verifies the reconstruction quality and performance with the benchmark algorithm, specifically including: Read the user's original image x o and reconstruct the image x r , calculate the mean square error value MSE of the two, which can be expressed as Where N represents the total number of pixels in the image, i is the i-th pixel; the peak signal-to-noise ratio is calculated by MSE Further, the structural similarity index SSIM is calculated, expressed as SSIM(x o ,x t ); In addition, the pre-trained deep neural network VGG is used to extract the features of the original image and the reconstructed image, and the Euclidean distance between these features is calculated to represent the image similarity, which can be expressed as φ l is the feature extraction function of the neural network at layer l, and L is the number of feature layers; The similarity and image visual quality of the reconstructed image and the user's reconstructed image in the federated system are determined by comprehensively comparing the four indicators of MSE, PSNR, LPIPS and SSIM to evaluate the performance of the gradient inversion attack method.

7. A gradient inversion attack method based on a diffusion model in federated learning according to any one of claims 1 to 6, characterized in that: The gradient inversion attack system based on the diffusion model in the federated learning includes: Model pre-training module, used by the server to pre-train the encoder-decoder and diffusion model using prior data before reconstruction begins, to limit image generation during the reconstruction process; The decreasing gradient inversion reconstruction module is used to reconstruct user images and labels based on user shared gradients. After obtaining the user shared gradients, the server uses a decreasing iterative generation and optimization attack method to compress the image into the feature space for operation. In each iteration, the generation results of the intermediate steps are gradually optimized by disassembling the diffusion model. The decreasing iteration realizes reconstruction from contour to detail. At the same time, the image feature regularization is used to further limit the feature search space, reveal the user image data features and reconstruct them. Image post-processing and enhancement module, used to further enhance the image details and image quality after optimization and reconstruction, perform non-optimized diffusion model denoising, and enhance image saturation, contrast, and sharpness; The similarity and performance evaluation module is used to determine the similarity between the reconstructed image and the original image of the federated user and the performance of the gradient inversion image reconstruction method, and calculate the mean square error, peak signal-to-noise ratio, perceptual similarity, structural similarity index value between the reconstructed image and the original image, as well as the comprehensive similarity index proposed by this method.

8. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the gradient inversion attack method based on the diffusion model in federated learning as described in any one of claims 1 to 5.

9. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the gradient inversion attack method based on a diffusion model in federated learning as described in any one of claims 1 to 5.

10. An information data processing terminal, characterized in that: The information data processing terminal is used to implement the gradient inversion attack system based on the diffusion model in federated learning as described in claim 6.

Images