Privacy disclosure risk defense method and system oriented to diffusion model

Abstract

The invention relates to a privacy disclosure risk defense method and system oriented to a diffusion model, and belongs to the field of deep learning. The defense method comprises the following steps: dividing an original training data set into k data subsets which are not overlapped with each other; grouping the total time step T and constructing m denoising prediction sub-networks with the same structure and independent parameters; generating a 0-1 coding matrix of k rows and T columns, and controlling whether the i-th training group participates in parameter updating of the sub-network corresponding to the t-th time step; cross-time-step training of same forward noise addition and coding matrix control is adopted; and scheduling the corresponding sub-network to execute noise prediction according to the current time step in the generation stage so as to gradually generate samples. According to the method, the contact range of the denoising prediction sub-network and the training data is limited, and the difference between the training data and the test data in noise estimation is reduced, so that the risk of member privacy leakage caused by overfitting is reduced, and meanwhile, the quality of a generated sample of a diffusion model is maintained.

Description

Technical Field

[0001] The present invention relates to the field of deep learning. More specifically, the present invention relates to a method and system for defending against privacy leakage risks in a diffusion model. Background Art

[0002] With the development of generative artificial intelligence (AI) technologies, diffusion models have been widely used in generative tasks across multiple domains, such as image generation, image completion, audio synthesis, and text-based video. The diffusion model's superior generative performance and wide range of application scenarios have made it an emerging mainstream deep generative model. However, current diffusion models face the risk of member privacy leakage during training, making them vulnerable to membership inference attacks. Membership inference attacks are a typical privacy attack method. By gaining access to the model, an attacker can determine whether a sample belongs to the model's training dataset and thus infer whether it contains specific sensitive information or behavior related to the model. Existing privacy-preserving methods for diffusion models, such as differential privacy, data augmentation, regularization, and knowledge distillation, fail to achieve a comprehensive and optimal balance between defense capabilities, generative utility, and computational overhead. Therefore, there is an urgent need to design new defense schemes for diffusion models to improve their robustness against privacy attacks. Summary of the Invention

[0003] The purpose of this invention is to address the sensitivity of diffusion models to member inference attacks and the fundamental source of member privacy leakage risks. Based on an integrated training and cross-timestep denoising strategy, a privacy leakage risk defense method and system for diffusion models is constructed. By limiting the contact range between the multiple denoising prediction subnetworks that make up the diffusion model and the training data, the diffusion model can separate and isolate the training set information at the training data grouping level and the timestep level, reducing the difference in the diffusion model's performance in noise estimation for training data and test data, thereby reducing the risk of member privacy leakage caused by overfitting behavior of the diffusion model during training.

[0004] In order to achieve the purpose and other advantages of the present invention, a method for defending against privacy leakage risks in a diffusion model is provided, comprising the following steps: Step 1: Set the number of training groups to k and divide the original training data set of the diffusion model into k non-overlapping data subsets, where k is an integer greater than 1; Step 2: Set the number of time step groups to m, and group the total time steps T in the diffusion model denoising process, where the value range of m is [1, T], and T is an integer greater than 1; Step 3: Construct m denoising prediction sub-networks with the same structure. Each denoising prediction sub-network corresponds to a time step group, and the parameters of each denoising prediction sub-network are independent of each other. Step 4: Generate a coding matrix with k rows and T columns based on the above training groups and total time steps. The element value of the coding matrix is ​​0 or 1, which is used to control whether the i-th training group participates in the denoising prediction sub-network parameter update of the j-th time step group corresponding to the t-th time step, where the value range of i is [1, k], the value range of t is [1, T], and the value range of j is [1, m]; Step 5: Perform cross-time step training on each training group: a) Process the data using the same forward denoising process; b) in the reverse denoising process, controlling whether the data of the i-th training group is used to update the denoising prediction sub-network parameters of the j-th time step group corresponding to the t-th time step according to the value of the i-th row and t-th column in the encoding matrix; Step 6: In the diffusion model generation stage, the corresponding denoising prediction subnetwork is called according to the current time step to perform noise prediction, thereby realizing the step-by-step denoising generation of samples.

[0005] Preferably, step 5 b) also includes parameter integration operations and network integration operations: during the training process, the denoising prediction subnetwork corresponding to each time step group will be optimized and updated based on a pre-set encoding matrix using only part of the training group data, and the final network parameters of the denoising prediction subnetwork are reflected as the integrated learning results of all the training group information used; after the training is completed, the denoising prediction subnetworks corresponding to all time steps will be integrated into a complete network sequence in chronological order to constitute the final diffusion model.

[0006] Preferably, the logic of calling the denoising prediction sub-network in step 6 is: selecting the corresponding denoising prediction sub-network according to the time step group number to which the current time step j belongs.

[0007] Preferably, the privacy leakage risk defense method for the diffusion model further includes step seven: using a membership inference attack tool to test the attack success rate of the diffusion model, and calculating the Fréchet distance of the generated samples.

[0008] The present invention also provides a privacy leakage risk defense system for a diffusion model, comprising: A data processing module is used to set the number of training groups to k and divide the original training data set of the diffusion model into k non-overlapping data subsets, where k is an integer greater than 1; The time step processing module is used to set the number of time step groups to m and group the total time steps T in the diffusion model denoising process, where the value range of m is [1, T], and T is an integer greater than 1; The denoising network construction module is used to construct m denoising prediction sub-networks with the same structure. Each denoising prediction sub-network corresponds to a time step group, and the parameters of each denoising prediction sub-network are independent of each other. An encoding matrix control module is used to generate an encoding matrix of k rows and T columns based on the above training groups and the total time steps. The element value of the encoding matrix is ​​0 or 1, and is used to control whether the i-th training group participates in the denoising prediction sub-network parameter update of the j-th time step group corresponding to the t-th time step, where the value range of i is [1, k], the value range of t is [1, T], and the value range of j is [1, m]; A cross-timestep training module, which is used to perform cross-timestep training on each training group: a) Process the data using the same forward denoising process; b) in the reverse denoising process, controlling whether the data of the i-th training group is used to update the denoising prediction sub-network parameters of the j-th time step group corresponding to the t-th time step according to the value of the i-th row and t-th column in the encoding matrix; The sampling application module is used to call the corresponding denoising prediction subnetwork according to the current time step to perform noise prediction in the diffusion model generation stage, thereby realizing the step-by-step denoising generation of samples.

[0009] Preferably, the cross-time-step training module (b) also includes parameter integration and network integration operations. During training, the denoising prediction subnetwork corresponding to each time-step group is optimized and updated based on a pre-set encoding matrix, using only a portion of the training group data. The final network parameters of the denoising prediction subnetwork are the result of integrated learning of all used training group information. After training is complete, the denoising prediction subnetworks corresponding to all time steps are integrated into a complete network sequence in chronological order, forming the final diffusion model.

[0010] Preferably, the logic of calling the denoising prediction subnetwork in the sampling application module is: selecting the corresponding denoising prediction subnetwork according to the time step group number to which the current time step j belongs.

[0011] Preferably, the privacy leakage risk defense system for the diffusion model also includes a performance verification module, which uses a member reasoning attack tool to test the attack success rate of the diffusion model to evaluate the defense capability of the diffusion model against the member privacy leakage risk, and calculates the Fréchet distance of the generated samples to evaluate the generation utility of the diffusion model after the defense is implemented.

[0012] The present invention also provides an electronic device, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the above-mentioned method.

[0013] The present invention also provides a computer-readable medium having a computer program stored thereon, which implements the above method when executed by a processor.

[0014] The present invention has at least the following beneficial effects: First, through integrated training and a cross-time-step denoising mechanism controlled by an encoding matrix, the present invention can significantly reduce the diffusion model's defense performance against member inference attacks, thereby reducing the risk of member privacy leakage. Second, while achieving defense, the present invention can effectively maintain the high generative utility and low computational overhead of the diffusion model, and provides a controllable trade-off between privacy protection and generative utility based on the encoding matrix. Third, the defense framework constructed by the present invention is applicable to mainstream diffusion model architectures and has good versatility and practicality.

[0015] Other advantages, objectives and features of the present invention will be reflected in part from the following description and will be understood by those skilled in the art through study and practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a schematic diagram of the overall process of the privacy leakage risk defense method for the diffusion model of the present invention; Figure 2 This is a graph showing the privacy and security risk results of the DDPM model and the OEDM model on the CIFAR-10 dataset; Figure 3 This is a graph showing the privacy and security risk results of the DDPM model and the OEDM model on the CIFAR-100 dataset; Figure 4 It is the real image of the member dataset and the corresponding generated image when the DDPM model before and after defense performs the generation task; Figure 5 The DDPM model before and after defense retains the real image of the dataset and the corresponding generated image when performing the generation task; Figure 6 It is the real image of the member dataset and the corresponding generated image when the DP-SGD model before and after defense performs the generation task; Figure 7 The DP-SGD model before and after defense retains the real image of the dataset and the corresponding generated image when performing the generation task; Figure 8It is the real image of the member dataset and the corresponding generated image when the OEDM model before and after defense performs the generation task; Figure 9 The OEDM model before and after defense retains the real images of the dataset and the corresponding generated images when performing the generation task. DETAILED DESCRIPTION

[0017] The present invention will be further described in detail below with reference to the embodiments and drawings so that those skilled in the art can implement the invention with reference to the description.

[0018] It should be understood that terms such as “having”, “including” and “comprising” used herein do not preclude the existence or addition of one or more other elements or combinations thereof.

[0019] It should be noted that the experimental methods described in the following embodiments are all conventional methods unless otherwise specified.

[0020] like Figure 1 As shown, the present invention provides a privacy leakage risk defense method for a diffusion model, comprising the following steps: Step 1: Set the number of training groups to k and divide the original training data set of the diffusion model into k non-overlapping data subsets, where k is an integer greater than 1; Step 2: Set the number of time step groups to m, and group the total time steps T in the diffusion model denoising process, where the value range of m is [1, T], and T is an integer greater than 1; Step 3: Construct m denoising prediction sub-networks with the same structure. Each denoising prediction sub-network corresponds to a time step group, and the parameters corresponding to each denoising prediction sub-network are Independent of each other, Represents the parameters of the jth denoising prediction subnetwork, and the value range of j is [1, m]. Based on the value of m, at most an independent denoising network can be used for each time step, that is, training An independent denoising network; when m=1, it degenerates into a traditional diffusion model, that is, only one denoising prediction network is trained and the network is used for denoising at each time step; Step 4: Generate a coding matrix of k rows and T columns based on the above training group and total time steps , the element value of the encoding matrix is ​​0 or 1, which is used to control whether the i-th training group participates in the denoising prediction sub-network parameter update of the j-th time step group corresponding to the t-th time step, where the value range of i is [1, k], the value range of t is [1, T], and the value range of j is [1, m]; Step 5: Perform cross-time step training on each training group: a) Process the data using the same forward denoising process; b) In the reverse denoising process, according to the value of the i-th row and t-th column in the encoding matrix, control whether the data of the i-th training group is used to update the denoising prediction sub-network parameters of the j-th time step group corresponding to the t-th time step; the parameters corresponding to each denoising prediction sub-network Parameter integration will be performed during the denoising training process across time steps, i.e. , Represents the denoising prediction subnetwork parameters of the j-th time step group corresponding to the i-th training group After training is completed, the denoising prediction sub-networks corresponding to all time steps will be integrated into a complete network sequence in chronological order to form the final diffusion model; Step 6: In the diffusion model generation stage, the corresponding denoising prediction subnetwork is called according to the current time step to perform noise prediction, thereby realizing the step-by-step denoising generation of samples.

[0021] In the above method, step 1 reduces the data sharing between the denoising prediction subnetworks in the diffusion model by grouping the training data based on the idea of ​​ensemble learning, thereby enhancing the generalization ability of the model; steps 2 and 3 introduce multiple denoising prediction subnetworks with consistent structures and independent parameters for different time steps, and achieve parameter-level integration through iterative optimization during the training process; the encoding matrix described in step 4 is a 0-1 matrix, which is used to control the visibility of each training group at each time step, providing adjustability between defense strength and model generation performance; in step 5, during the training process, the denoising prediction subnetwork of each time step only accepts optimization updates of part of the training subset data, thereby avoiding the memory and fitting of the network parameters to all the data; in the sampling process of step 6, the corresponding denoising prediction subnetwork is called according to the time step to generate samples, and the generation process is the inverse denoising process of the standard diffusion model.

[0022] Among them, the logic of calling the denoising prediction subnetwork in step 6 is: select the corresponding denoising prediction subnetwork according to the time step group number to which the current time step j belongs.

[0023] The privacy leakage risk defense method for the diffusion model also includes step seven: using a membership inference attack tool to test the attack success rate of the diffusion model and calculating the Fréchet distance of the generated samples.

[0024] In step seven, the defense capability is evaluated by measuring the success rate of member inference attacks on the diffusion model to assess the model's defense performance against member privacy leakage risks. The lower the attack success rate, the higher the defense performance. The generation utility is evaluated by calculating the FID (Fréchet distance) of the diffusion model to assess the model's maintenance level for generation utility. The lower the FID, the higher the model generation utility.

[0025] Furthermore, the present invention provides a privacy leakage risk defense system for a diffusion model, comprising: A data processing module is used to set the number of training groups to k and divide the original training data set of the diffusion model into k non-overlapping data subsets, where k is an integer greater than 1; The time step processing module is used to set the number of time step groups to m and group the total time steps T in the diffusion model denoising process, where the value range of m is [1, T], and T is an integer greater than 1; Denoising network construction module, which is used to construct m denoising prediction sub-networks with the same structure. Each denoising prediction sub-network corresponds to a time step group, and the parameters corresponding to each denoising prediction sub-network are Independent of each other, Represents the parameters of the jth denoising prediction subnetwork, and the value range of j is [1, m]. Based on the value of m, at most an independent denoising network can be used for each time step, that is, training An independent denoising network; when m=1, it degenerates into a traditional diffusion model, that is, only one denoising prediction network is trained and the network is used for denoising at each time step; The encoding matrix control module is used to generate a k-row and T-column encoding matrix based on the above training group and the total time step , the element value of the encoding matrix is ​​0 or 1, which is used to control whether the i-th training group participates in the denoising prediction sub-network parameter update of the j-th time step group corresponding to the t-th time step, where the value range of i is [1, k], the value range of t is [1, T], and the value range of j is [1, m]; A cross-timestep training module, which is used to perform cross-timestep training on each training group: a) Process the data using the same forward denoising process; b) In the reverse denoising process, according to the value of the i-th row and t-th column in the encoding matrix, control whether the data of the i-th training group is used to update the denoising prediction sub-network parameters of the j-th time step group corresponding to the t-th time step; the parameters corresponding to each denoising prediction sub-network Parameter integration will be performed during the denoising training process across time steps, i.e. , Represents the denoising prediction subnetwork parameters of the j-th time step group corresponding to the i-th training group ; The sampling application module is used to call the corresponding denoising prediction subnetwork according to the current time step to perform noise prediction in the diffusion model generation stage, thereby realizing the step-by-step denoising generation of samples.

[0026] Among them, the logic of calling the denoising prediction subnetwork in the sampling application module is: according to the time step group number to which the current time step j belongs, the corresponding denoising prediction subnetwork is selected.

[0027] The privacy leakage risk defense system for the diffusion model also includes a performance verification module, which uses a member reasoning attack tool to test the attack success rate of the diffusion model to evaluate the diffusion model's defense capability against member privacy leakage risks, and calculates the Fréchet distance of the generated samples to evaluate the generation utility of the diffusion model after the defense is implemented.

[0028] Example 1: A privacy leakage risk defense method for a diffusion model includes the following steps: Step 1: In the data processing module, set the number of training groups The training dataset of the diffusion model is partitioned into three non-overlapping data subsets.

[0029] Step 2: In the time step processing module, set the total time step of the denoising process to 1000 and the number of time step groups A value of 5 groups the time steps in the diffusion model denoising process.

[0030] Step 3: In the denoising network construction module, group each time step divided in step 2 and construct 5 denoising prediction sub-networks with the same structure but independent parameters.

[0031] Step 4: In the encoding matrix control module, generate the encoding matrix for the above training group and denoising time step According to the settings in the above steps, . Through the encoding matrix middle Controls the time steps that the training group needs to skip in subsequent denoising training. For the value of a certain position in the encoding matrix , the corresponding training group is group i [1,k], the time step is t [1,T], the corresponding time step grouping is ,in .but Control Whether the samples in the training group participate in the time-step denoising prediction subnetwork Parameters in training Updates.

[0032] Step 5: In the cross-time step training module, keep the forward denoising process consistent for each training group during the training process. In the reverse denoising training, control the denoising prediction subnetwork for each training group based on the encoding matrix for training optimization. Specifically, for the first training groups, time steps and their corresponding denoising prediction subnetworks ,if , then training groups participate in time steps Corresponding denoising prediction subnetwork Parameter update ;if 0, then training groups will not be used for time steps during training Corresponding denoising prediction subnetwork Parameter update . The parameters corresponding to each denoising prediction sub-network Parameter integration will be performed during the denoising training process across time steps, i.e. , Represents the denoising prediction subnetwork parameters of the j-th time step group corresponding to the i-th training group .

[0033] Step 6: In the sampling application module, sample random noise Input the diffusion model trained in steps 1 to 5, and Call the corresponding denoising prediction subnetwork Perform noise prediction to achieve step-by-step denoising of samples and obtain synthetic samples .

[0034] Step 7: In the performance verification module, the diffusion model trained in Steps 1 through 5 is sampled multiple times to obtain a synthetic sample set. FID calculations are performed against the real dataset to evaluate the model's effectiveness after implementing the defense. Furthermore, a member inference attack is constructed based on the diffusion model's training mechanism and generation characteristics to evaluate the diffusion model's ability to defend against member privacy leaks.

[0035] Among them, FID is Fréchet distance (Fréchet Inception Distance), which is extracted by inputting real samples and generated samples into the pre-trained Inception v3 network to extract the features of each image. and , and calculate the mean of the two sets of feature distributions ( ) and covariance ( ), thereby calculating the indicator obtained by Fréchet distance: The lower the FID, the closer the generated image is to the real image in distribution, that is, the better the model generation performance.

[0036] In the defense capability assessment, the following attack methods are selected to build the attack model: (1) Per time Noise Estimation Error Membership Inference Attack (t-NEMIA): The diffusion model restores the original data by learning the prediction network of the noise at each time step. The training goal of the network is to make the prediction noise Approximate real added noise , its essence is to use the denoising prediction network to fit the posterior distribution of the forward process at each time step , thereby minimizing the posterior estimation error of the denoising prediction network for the forward process. For a data sample For the diffusion model, the diffusion model The estimation error can be expressed as: ,in is sampled from the posterior distribution of the true forward process, It is the denoised sample obtained by the model through the distribution denoising prediction of the learned reverse process. Due to the overfitting phenomenon of the denoising prediction network of the diffusion model during the training process, the noise estimation error of the denoising prediction network for the member sample at a certain time step should be smaller than that of the non-member sample, that is, .in, Belongs to the training data set , Belongs to the reserved dataset . Based on the above attack principles, two attack models are implemented and encapsulated: statistical threshold-based attack and neural network-based attack. The former uses the average standard error to calculate the noise estimation error of each sample and searches for the optimal threshold for membership inference through threshold traversal. If it is higher than the threshold, it is inferred to be a member, otherwise it is inferred to be a non-member; the latter uses the absolute value of the pixel-by-pixel difference of the image in the noise estimation as the input of the attack model, and uses a set of error image data with identity labels to train a binary classifier as the attack model, and its output is the member identity label. (2) Per time Gradient Feature Membership Inference Attack (t-GFMIA): Under white-box access rights, the attacker can obtain the gradient vector of the input sample, that is, Since each member sample in the diffusion model will produce a specific gradient response to the model parameters during the training process, the member samples tend to produce smaller or more concentrated gradient updates due to their participation in parameter optimization updates, that is, .in, Belongs to the training data set , Belongs to the reserved dataset Therefore, we use the time-step gradient features as the input of the attack model to infer membership, implementing and encapsulating the third attack model. Based on the encapsulated attack model, we conduct a membership inference attack on the diffusion model trained by the method of the present invention, and evaluate the model's ability to defend against the risk of member privacy leakage using the attack success rate (ASR).

[0037] Example 2: The defense effectiveness and application practicality of the present invention are evaluated from three dimensions: privacy and security protection, model generation utility maintenance, and computational overhead. The experiment uses two image datasets, CIFAR-10 and CIFAR-100, and divides the training dataset of the original dataset into a training dataset of the target diffusion model and a retained dataset in equal proportion. The experiment selects the unprotected DDPM model as the comparison baseline, and uses the same training dataset to compare the DDPM model and the diffusion model trained by the present invention (for ease of description and understanding, it will be referred to as Our Enhanced Diffusion Model, OEDM in the subsequent description), fixing the total time step of the diffusion model. The maximum number of training steps is 1000, and the maximum number of training steps is 400k. Defense capability is evaluated using the attack success rate (ASR) and the area under the receiver operating characteristic curve (AUC) output by the membership inference attack tool after attacking the target model. These are defined as the privacy and security risk of the target diffusion model. The change in the attack success rate of the diffusion model under membership inference attacks before and after the implementation of the defense scheme reflects the defense capability of the defense scheme against the diffusion model. Regarding model generation performance, FID is used as a metric to evaluate the quality of samples generated by the diffusion model. Computational overhead is measured by the time it takes the model to complete the same number of training steps and the time it takes to perform the same number of generation tasks.

[0038] In terms of privacy and security protection, the defense capability of OEDM is evaluated using three encapsulated attack models (t-NEMIA Statistic t-NEMIA NNs , t-GFMIA) implements membership inference attacks on DDPM models and OEDM models. Figure 2The privacy and security risks of the DDPM model and the OEDM model on the CIFAR-10 dataset are demonstrated. Figure 3 The privacy and security risks of the DDPM model and the OEDM model on the CIFAR-100 dataset are shown. The horizontal axis in the figure represents the attack method, and the left and right vertical axes represent the ASR value and AUC value of the membership inference attack respectively. Figure 2 and Figure 3 ,We can see that OEDM is effective for different membership inference attack methods.,It can reduce the ASR of membership inference attack to below 0.57 and AUC to below 0.60 on both CIFAR-10 and CIFAR-100 datasets.

[0039] In terms of computational overhead, the OEDM model and the unprotected DDPM model were compared during model training and generation task execution. Both models were trained on the CIFAR-10 dataset using the same number of training steps (8,000) and the training time was calculated. The same number of generation tasks (2,000 samples) was then executed using both models, and the generation task execution time was calculated in seconds. The results in Table 1 show that the OEDM and DDPM models performed almost identically in generation task execution time, while the OEDM model took longer to train than the DDPM model, with the relative difference increasing by approximately 44%. Current defense solutions based on the privacy distillation framework based on knowledge distillation have been shown to require significant computational overhead during training: these methods require retraining the diffusion model after privacy distillation. Excluding the processing time associated with privacy distillation itself, the training time alone is approximately twice that of the original model. Therefore, compared with the privacy distillation framework, the defense scheme in this chapter significantly saves the computational overhead of model training, verifying that the defense scheme in this chapter has low time cost when protecting the diffusion model, and can provide more efficient privacy and security protection against membership inference attacks on the diffusion model.

[0040] Table 1 Comparison of computational overhead of diffusion model training and generation before and after defense In terms of maintaining model generation utility and balancing privacy and utility, experiments compared the performance of OEDM with existing defense methods in terms of generation utility and defense capabilities. The defense methods used included local occlusion cutout, L2 regularization, and differentially private gradient descent (DP-SGD). The basic training configuration of the control model remained the same throughout the experiment, and the corresponding defense method was only implemented during data processing or training. The results in Table 2 show that OEDM achieves a good balance between generation utility and defense capabilities. Specifically, compared with the data augmentation-based cutout method, OEDM improves the model's generation performance, reducing the FID from 39.635 to 25.360. At the same time, OEDM demonstrates superior defense capabilities, with attack success rates lower than those of the cutout method under all three attacks. Compared with L2 regularization and DP-SGD, although L2 regularization can reduce the ASR of t-NEMIA to near random guessing, and DP-SGD can reduce the ASR of t-NEMIA and t-GFMIA to near random guessing, it brings about extremely high generation performance degradation, with FID reaching 104.120 and 270.242 respectively. The generation ability of the diffusion model is seriously sacrificed in exchange for privacy protection effect, and its practicality is low. OEDM can achieve similar privacy protection effect to the above methods ( ASR=0.032, 0.018, 0.064, significantly improving the authenticity of image generation (FID=25.360, FID=78.76, 244.882), and OEDM is able to maintain the same generation utility as the pre-defense diffusion model ( FID=1.081). Overall, compared to existing defense methods, OEDM can effectively reduce the vulnerability of diffusion models to member inference attacks while ensuring the model's generative utility, achieving privacy protection while maintaining generative utility.

[0041] Table 2 Performance comparison between OEDM and existing defense methods Example 3: Based on the above experimental verification, the effectiveness of OEDM is further verified through case studies. Figure 4 The real and generated images of the member datasets are shown after the DDPM model before and after defense performs the generation task. Figure 5 It shows that the DDPM model before and after defense retains the real images and generated images on the dataset (non-member dataset) after performing the generation task. Figure 6 The real and generated images of the member datasets are shown after the DP-SGD model before and after defense performs the generation task. Figure 7It shows that the DP-SGD model before and after defense retains the real images and generated images on the dataset after performing the generation task. Figure 8 The real and generated images of the member datasets are shown after the OEDM model before and after defense performs the generation task. Figure 9 The OEDM model before and after defense retains the real images and generated images on the dataset after performing the generation task. Table 3 shows the prediction results of the DDPM model, DP-SGD model and OEDM model under membership inference attack. The attack prediction label uses the attack results of the three encapsulated attack models and is judged based on the majority voting mechanism. The attack prediction label output for member samples is "1", and the supply prediction label output for non-member samples is "0".

[0042] Table 3 Prediction results of DDPM model, DP-SGD model and OEDM model under membership inference attack Depend on Figure 4-Figure 9 As shown in Table 3, for the DDPM model, the attacker can accurately distinguish between member samples and non-member samples, indicating that the DDPM model has a significant risk of member privacy leakage when unprotected. The attacker can successfully capture the performance differences between the target model and non-member samples. For the OEDM model, the predicted labels of the membership inference attack for the same member and non-member samples are close to random guesses and the inference accuracy is low. This shows that after adopting the OEDM defense scheme, the attacker has difficulty extracting effective attack features and establishing membership identification edges, verifying the effectiveness of the OEDM defense scheme. At the same time, comparing the generated images of the DDPM model and the OEDM model, it can be found that the authenticity and similarity of the images generated by the models before and after the defense are basically consistent for member samples and non-member samples. The OEDM model can achieve the same degree of generation effect as the DDPM model before the defense. This shows that the OEDM defense scheme has a good ability to maintain the generation utility of the diffusion model itself, that is, it can achieve effective privacy and security protection without sacrificing model generation performance. While models protected with differentially private gradient descent (DP-SGD) achieve good defense, with low accuracy against attacks on member and non-member prediction labels, they also suffer from significant deficiencies in image generation quality. Compared to the generation results of the DDPM and OEDM models, the images generated by the DP-SGD-protected model exhibit severe blurring and structural loss, failing to restore the semantic information of the original image. This demonstrates that while DP-SGD can protect the privacy of the model's training data, it does so at the expense of model generation performance, significantly reducing the usability and quality of the generated images.

[0043] The number of devices and processing scales described here are used to simplify the description of the present invention. Applications, modifications, and variations of the method and system for preventing privacy leakage risks based on a diffusion model of the present invention are obvious to those skilled in the art.

[0044] Although the embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the description and implementation methods. They can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily implemented. Therefore, without departing from the general concept defined by the claims and the scope of equivalents, the present invention is not limited to the specific details and illustrations shown and described herein.

Claims

1. A privacy leakage risk defense method for a diffusion model, characterized by: The following steps are involved: Step 1: Set the number of training groups to k and divide the original training data set of the diffusion model into k non-overlapping data subsets, where k is an integer greater than 1; Step 2: Set the number of time step groups to m, and group the total time steps T in the diffusion model denoising process, where the value range of m is [1, T], and T is an integer greater than 1; Step 3: Construct m denoising prediction sub-networks with the same structure. Each denoising prediction sub-network corresponds to a time step group, and the parameters of each denoising prediction sub-network are independent of each other. Step 4: Generate a coding matrix with k rows and T columns based on the above training groups and total time steps. The element value of the coding matrix is ​​0 or 1, which is used to control whether the i-th training group participates in the denoising prediction sub-network parameter update of the j-th time step group corresponding to the t-th time step, where the value range of i is [1, k], the value range of t is [1, T], and the value range of j is [1, m]; Step 5: Perform cross-time step training on each training group: a) Process the data using the same forward denoising process; b) in the reverse denoising process, controlling whether the data of the i-th training group is used to update the denoising prediction sub-network parameters of the j-th time step group corresponding to the t-th time step according to the value of the i-th row and t-th column in the encoding matrix; Step 6: In the diffusion model generation stage, the corresponding denoising prediction subnetwork is called according to the current time step to perform noise prediction, thereby realizing the step-by-step denoising generation of samples.

2. The privacy leakage risk defense method for the diffusion model according to claim 1, characterized in that: Step 5 (b) also includes parameter integration and network integration operations: During the training process, the denoising prediction subnetwork corresponding to each time step group will be optimized and updated based on the pre-set encoding matrix using only part of the training group data. The final network parameters of the denoising prediction subnetwork are reflected as the integrated learning results of all the training group information used; After training is completed, the denoising prediction sub-networks corresponding to all time steps will be integrated into a complete network sequence in chronological order to form the final diffusion model.

3. The privacy leakage risk defense method for the diffusion model according to claim 1 is characterized in that: The logic of calling the denoising prediction subnetwork in step 6 is: select the corresponding denoising prediction subnetwork according to the time step group number to which the current time step j belongs.

4. The privacy leakage risk defense method for the diffusion model according to claim 1, characterized in that: It also includes step seven: using the membership inference attack tool to test the attack success rate of the diffusion model and calculating the Fréchet distance of the generated samples.

5. A privacy leakage risk defense system for a diffusion model, characterized by: include: A data processing module is used to set the number of training groups to k and divide the original training data set of the diffusion model into k non-overlapping data subsets, where k is an integer greater than 1; The time step processing module is used to set the number of time step groups to m and group the total time steps T in the diffusion model denoising process, where the value range of m is [1, T], and T is an integer greater than 1; The denoising network construction module is used to construct m denoising prediction sub-networks with the same structure. Each denoising prediction sub-network corresponds to a time step group, and the parameters of each denoising prediction sub-network are independent of each other. An encoding matrix control module is used to generate an encoding matrix of k rows and T columns according to the above training groups and the total time steps. The element value of the encoding matrix is ​​0 or 1, which is used to indicate whether the i-th training group participates in the denoising prediction sub-network parameter update of the j-th time step group, where the value range of i is [1, k] and the value range of j is [1, T]; A cross-timestep training module, which is used to perform cross-timestep training on each training group: a) Process the data using the same forward denoising process; b) in the reverse denoising process, controlling whether the data of the i-th training group is used to update the denoising prediction sub-network parameters of the j-th time step group corresponding to the t-th time step according to the value of the i-th row and t-th column in the encoding matrix; The sampling application module is used to call the corresponding denoising prediction subnetwork according to the current time step to perform noise prediction in the diffusion model generation stage, thereby realizing the step-by-step denoising generation of samples.

6. The privacy leakage risk defense system for diffusion model according to claim 5, characterized in that: The cross-timestep training module (b) also includes parameter integration and network integration operations: During the training process, the denoising prediction subnetwork corresponding to each timestep group will be optimized and updated based on the pre-set encoding matrix using only part of the training group data. The final network parameters of the denoising prediction subnetwork are reflected as the integrated learning results of all the training group information used; After training is completed, the denoising prediction sub-networks corresponding to all time steps will be integrated into a complete network sequence in chronological order to form the final diffusion model.

7. The privacy leakage risk defense system for diffusion model according to claim 5, characterized in that: The logic of calling the denoising prediction subnetwork in the sampling application module is: select the corresponding denoising prediction subnetwork according to the time step group number to which the current time step j belongs.

8. The privacy leakage risk defense system for diffusion model according to claim 5, characterized in that: It also includes a performance verification module, which uses the membership inference attack tool to test the attack success rate of the diffusion model to evaluate the diffusion model's defense capabilities against the risk of member privacy leakage, and calculates the Fréchet distance of the generated samples to evaluate the generation utility of the diffusion model after the defense is implemented.

9. An electronic device, characterized in that include: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to cause the at least one processor to perform the method according to any one of claims 1 to 4.

10. A computer-readable medium having a computer program stored thereon, characterized in that When the program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images