Sample purification method, device, and computer equipment

Abstract

The application relates to a sample purification method, device and computer equipment. The method comprises the following steps: obtaining an adversarial sample in an edge device, adding random noise to the adversarial sample to obtain a to-be-de-noised sample; based on the to-be-de-noised sample, performing noise prediction and sample purification processing by using a noise prediction model to obtain a purified clean sample; the noise prediction model adopts a high-level feature map multiplexing mechanism, the high-level feature map multiplexing mechanism is used for multiplexing high-level feature maps sampled in de-noising steps t to de-noising steps t-n, so as to be spliced with shallow layer feature maps generated by a shallow layer network of the noise prediction model, and a spliced feature map is obtained, n is an integer in [1, 2,..., N-1], and N is a preset cache interval. By using the method, the purification efficiency can be improved, and the cost can be saved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a sample purification method, apparatus, and computer equipment. Background Art

[0002] With the development of deep learning technology, the security issues of deep learning models have gradually become apparent. Research has found that deep learning models are vulnerable to adversarial attacks. Attackers add carefully designed small perturbations to the input data to trick the machine learning model into producing incorrect outputs or behaviors. These maliciously designed samples are called adversarial samples. This problem is particularly severe in edge computing scenarios. Because models deployed on edge devices typically interact directly with the physical world and are constrained by computing resources and real-time requirements, their defense capabilities are often weaker than cloud-based models and are vulnerable to adversarial attacks. For example, in smart healthcare, adversarial samples can lead to misdiagnosis; in smart homes, they can trigger illegal intrusions or device loss of control. Therefore, model security in edge scenarios not only affects the accuracy of the output but also directly affects user privacy and system stability.

[0003] Existing adversarial defense methods include adversarial purification. Adversarial purification involves preprocessing input data to remove adversarial perturbations while preserving the data's original characteristics. Among existing adversarial purification methods, those based on diffusion models perform relatively well. Compared to adversarial training, they have stronger generalization capabilities and can better cope with complex and changing attack scenarios. However, the reverse denoising process of diffusion models often requires numerous iterative computations, resulting in high inference overhead and low efficiency. This makes it difficult to meet the actual deployment requirements of edge devices with limited computing resources and strict real-time requirements. Summary of the Invention

[0004] Based on this, it is necessary to provide a sample purification method, device and computer equipment that can improve purification efficiency and save costs in response to the above technical problems.

[0005] A sample purification method, comprising:

[0006] S101: Obtain an adversarial sample from an edge device and add random noise to the adversarial sample to obtain a sample to be denoised;

[0007] S102. Based on the sample to be denoised, a noise prediction model is used to perform noise prediction and sample purification to obtain a purified clean sample; the noise prediction model adopts a high-level feature map reuse mechanism, and the high-level feature map reuse mechanism is used to reuse the high-level feature map sampled by the denoising step number t into the denoising step number tn, so as to splice it with the shallow feature map generated by the shallow network of the noise prediction model to obtain a spliced ​​feature map, n∈[1,2,…,N-1], where N is a preset cache interval.

[0008] In one embodiment, the noise prediction process includes:

[0009] If the noise prediction model performs forward propagation at the denoising step number t, the cache upsampling block upsamples the denoised sample at the denoising step number t to obtain a high-level feature map;

[0010] In the denoising step number tn, a shallow feature map is generated by a shallow network in the noise prediction model; the shallow feature map is generated based on the sample to be denoised in the denoising step number tn;

[0011] Splicing the shallow feature map and the high-level feature map to obtain a spliced ​​feature map;

[0012] The spliced ​​feature map is upsampled using an upsampling block to obtain the predicted noise of the denoising step number tn.

[0013] In one embodiment, the noise prediction and sample purification processing are performed based on the sample to be denoised using a noise prediction model to obtain a purified clean sample, including:

[0014] S201. When the current denoising step number is denoising step number t, use the formula Calculate the first noise level schedule of the denoising step number t ;in, represents the curvature parameter, represents the maximum noise, represents the minimum noise, T is the total number of denoising steps;

[0015] S202: Schedule the first noise level The sample to be denoised with the denoising step number t is input into the noise prediction model to perform noise prediction, and a first predicted noise with the denoising step number t is obtained;

[0016] S203, determining the noise direction of the denoising step number t based on the first predicted noise;

[0017] S204: First noise level scheduling based on the noise direction, the samples to be denoised with the denoising step number t, and the denoising step number t-1. a first noise level schedule of the denoising step t determining a to-be-denoised sample of the denoising step t-1

[0018] S205, updating the current denoising step in step S201 to the denoising step t-1, updating the denoising step t-1 in step S204 to the denoising step t-2, and repeating steps S201, S202, S203 and S204 until the iteration step meets a first preset condition, and outputting a clean sample.

[0019] In one embodiment, the calculation method of the noise direction of the denoising step t in step S203 is:

[0020] ,

[0021] wherein d is the noise direction of the denoising step t, is the to-be-denoised sample of the denoising step t, d t is the first predicted noise of the denoising step t, is the first noise level schedule of the denoising step t.

[0022] In one embodiment, the calculation formula of the to-be-denoised sample of the denoising step t-1 in step S204 is:

[0023] ,

[0024] wherein is the to-be-denoised sample of the denoising step t-1, is the to-be-denoised sample of the denoising step t, is the first noise level schedule of the denoising step t-1, is the first noise level schedule of the denoising step t, and d is the noise direction.

[0025] In one embodiment, the noise prediction and sample purification processing based on the to-be-denoised sample using a noise prediction model to obtain a purified clean sample comprises:

[0026] S301, obtaining a preset noise level schedule sequence wherein represents the second noise level schedule of the denoising step n;

[0027] S302, when the current denoising step is the denoising step t, calculating the proportion of the remaining original information in the to-be-denoised sample of the denoising step t based on the second noise level schedule of the denoising step t, and calculating the proportion of the remaining original information in the to-be-denoised sample of the denoising step t-1 based on the second noise level schedule of the denoising step t-1;

[0028] S303: Input the square root of the ratio and the sample to be denoised with the denoising step number t into a noise prediction model to perform noise prediction, thereby obtaining a second predicted noise with the denoising step number t;

[0029] S304. Use formula , calculate the predicted clean samples predicted at the denoising step number t ;in, is the sample to be denoised with the denoising step number t, is the ratio corresponding to the denoising step number t, is the second predicted noise of the denoising step number t;

[0030] S305: Calculate a first weight of the random noise and a second weight of the second predicted noise based on the proportion of remaining original information in the to-be-denoised samples of the denoising step number t and the denoising step number t-1, respectively;

[0031] S306, Use formula , determine the denoising samples to be denoised with denoising step number t-1 , where c1 represents the first weight, ε represents the random noise, and c2 represents the second weight, represents the second predicted noise of the denoising step number t, is the ratio corresponding to the denoising step number t-1;

[0032] S307. Update the current denoising step number in step S302 to denoising step number t-1, update the denoising step number t-1 in step S302 to denoising step number t-2, and repeat steps S302, S303, S304, S305 and S306 until the iterative step number meets the second preset condition and outputs a clean sample.

[0033] In one embodiment, step S302 includes:

[0034] Using the formula Calculate the proportion of the remaining original information in the denoising sample with denoising step number t ;

[0035] Using the formula Calculate the proportion of the remaining original information in the denoising sample with denoising step number t-1 ;

[0036] in, represents the second noise level schedule of the denoising step number t, represents the second noise level schedule of the denoising step number t-1, is the intermediate parameter corresponding to the denoising step number t, is the intermediate parameter corresponding to the denoising step number t-1.

[0037] In one embodiment, the calculation formula of the first weight c1 is:

[0038] ,

[0039] The calculation formula of the second weight c2 is:

[0040] ,

[0041] in, is the ratio corresponding to the denoising step number t, is the ratio corresponding to the denoising step number t-1.

[0042] A sample purification device, comprising:

[0043] A sample acquisition module is used to obtain adversarial samples from edge devices and add random noise to the adversarial samples to obtain samples to be denoised;

[0044] A sample purification module is used to perform noise prediction and sample purification based on the sample to be denoised using a noise prediction model to obtain a purified clean sample; the noise prediction model adopts a high-level feature map reuse mechanism, and the high-level feature map reuse mechanism is used to reuse the high-level feature map sampled by the denoising step number t into the denoising step number tn, so as to splice it with the shallow feature map generated by the shallow network of the noise prediction model to obtain a spliced ​​feature map, n∈[1,2,…,N-1], where N is the preset cache interval.

[0045] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0046] The above-mentioned sample purification method, device and computer equipment obtain adversarial samples from edge devices and add random noise to the adversarial samples to obtain samples to be denoised. Based on the samples to be denoised, a noise prediction model is used to perform noise prediction and sample purification to obtain a purified clean sample; the noise prediction model adopts a high-level feature map reuse mechanism, and the high-level feature map reuse mechanism is used to reuse the high-level feature map sampled by the denoising step number t into the denoising step number tn, so as to splice it with the shallow feature map generated by the shallow network of the noise prediction model to obtain a spliced ​​feature map. In this way, redundant calculations can be reduced by reusing the high-level feature map, and the shallow feature map can be updated in real time to retain the generation details, thereby accelerating the reasoning without affecting the generation quality and quickly obtaining a clean sample. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 A diagram showing an application environment of a sample purification method in one embodiment;

[0048] Figure 2 Schematic diagram of a sample purification method according to an embodiment;

[0049] Figure 3 Schematic diagram of the structure of a noise prediction model in one embodiment;

[0050] Figure 4 is a schematic diagram of reusing high-level feature maps in one embodiment;

[0051] Figure 5 Schematic diagram of a sample purification process in one embodiment;

[0052] Figure 6 A schematic flow chart of a sample purification step in another embodiment;

[0053] Figure 7 is a schematic flow chart of a sample purification method according to another embodiment;

[0054] Figure 8 is a structural block diagram of a sample purification device in one embodiment;

[0055] Figure 9 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0057] The sample purification method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown, the terminal interacts with the server via a wired / wireless channel. The data storage system can store data that the server needs to process. The server obtains adversarial samples from the edge device and adds random noise to the adversarial samples to obtain samples to be denoised. Based on the samples to be denoised, the server uses a noise prediction model to perform noise prediction and sample purification to obtain clean samples. The noise prediction model uses a high-level feature map reuse mechanism to reuse the high-level feature maps sampled in denoising step t into denoising step tn. The high-level feature map reuse mechanism is used to concatenate the high-level feature maps generated by the shallow network of the noise prediction model to obtain a concatenated feature map, n∈[1,2,…,N-1], where N is a preset cache interval. The terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, etc. The server can be a single server, a server cluster consisting of multiple servers, or a cloud computing center consisting of multiple servers.

[0058] In one embodiment, Figure 2 As shown, a sample purification method is provided, which is applied to Figure 1 The following steps are used as an example to illustrate the server in the example:

[0059] S101: Obtain an adversarial sample from an edge device and add random noise to the adversarial sample to obtain a sample to be denoised.

[0060] Adversarial samples refer to input data that has been deliberately designed and modified in machine learning and related fields, especially for deep learning models.

[0061] In some embodiments, adversarial examples are obtained by obtaining a clean target sample and performing an adversarial attack on the target sample to generate an adversarial sample. Adversarial attack methods include, but are not limited to, PGD_EOT (Projected Gradient Descent with Expectation Over Transformation) and BPDA_EOT (Backward Pass Differentiable Approximation with Expectation Over Transformation).

[0062] Furthermore, PGD_EOT is an adversarial attack method that combines PGD and EOT. The core process is as follows: First, initialize a perturbation vector, usually starting from zero or a small random value. Then, enter the iterative process: in each iteration, a set of predefined transformations are applied to the current perturbation, such as rotation, scaling, noise addition, etc., and then the gradient of the model on these transformed samples is calculated, and the perturbation is updated along the gradient direction. The updated perturbation is projected into a set of constraints to ensure that its size is within a reasonable range. Repeat the above process for multiple iterations until the stopping condition is met. The final generated perturbation is added to the clean sample to form an adversarial sample that can still deceive the model under multiple transformations.

[0063] Furthermore, BPDA_EOT is an adversarial attack method that combines BPDA and EOT. It is primarily used to attack models that defend against adversarial examples by obfuscating gradients. It is also suitable for testing the robustness of denoising-based methods. The algorithm works as follows: First, an adversarial perturbation is initialized, typically zero or a small random perturbation. Then, in each iteration, the perturbed sample is fed into the target model, and the forward propagation takes into account any random transformations that may be introduced by the model. For non-differentiable parts of the model, BPDA techniques are used to replace these non-differentiable layers with differentiable approximate functions to calculate gradients. EOT is also used to calculate the expected gradients of these random transformations. This involves calculating the gradients for multiple transformed samples and averaging them. Then, based on the calculated gradients, the perturbation is updated in a direction that causes the model output to be incorrect, and the updated perturbation is projected within the constraints. This process is repeated until the predetermined number of iterations is reached or the attack target is met, generating an adversarial example.

[0064] Random noise is a random perturbation introduced into adversarial examples to test or enhance model robustness. Random noise includes, but is not limited to, Gaussian noise, uniform noise, and salt-and-pepper noise. The sample to be denoised is the adversarial example to which random noise has been added.

[0065] S102. Based on the sample to be denoised, a noise prediction model is used to perform noise prediction and sample purification to obtain a purified clean sample; the noise prediction model adopts a high-level feature map reuse mechanism, and the high-level feature map reuse mechanism is used to reuse the high-level feature map sampled by the denoising step number t into the denoising step number tn, so as to splice it with the shallow feature map generated by the shallow network of the noise prediction model to obtain a spliced ​​feature map, n∈[1,2,…,N-1], where N is the preset cache interval.

[0066] The noise prediction model is a model that can predict or generate the noise components in the sample to be denoised. Noise prediction models include but are not limited to U-Net (U-shaped network) models and generative adversarial network models.

[0067] Sample purification is the process of cleaning and optimizing the sample to be denoised, so as to remove or reduce the noise, outliers and other interference factors in the sample to be denoised, and obtain a clean sample after purification.

[0068] The high-level feature map reuse mechanism is to directly reuse the high-level feature map sampled at the denoising step number t into the denoising step number tn during the multi-step denoising process, without calculating the high-level feature map at each denoising step, which can save denoising time overhead.

[0069] Before reusing the high-level feature map, the high-level feature map sampled at the denoising step number t is cached for reuse. The denoising step number t of the cached high-level feature map is the denoising step number of the complete forward propagation.

[0070] The preset cache interval is the interval between denoising steps for caching high-level feature maps. For example, if at denoising step x, the noise prediction model performs a full forward pass and caches the high-level feature map, then at denoising step xn-1, the denoised samples from step xn-1 are upsampled again to obtain the high-level feature map. The high-level feature map from step xn-1 is then used to update the cached high-level feature map, allowing it to be reused in the subsequent n-1 denoising steps.

[0071] In the above-mentioned sample purification method, by obtaining the adversarial sample in the edge device and adding random noise to the adversarial sample, the sample to be denoised is obtained. Based on the sample to be denoised, the noise prediction model is used to perform noise prediction and sample purification to obtain a purified clean sample; the noise prediction model adopts a high-level feature map reuse mechanism, and the high-level feature map reuse mechanism is used to reuse the high-level feature map sampled by the denoising step number t into the denoising step number tn, so as to splice it with the shallow feature map generated by the shallow network of the noise prediction model to obtain a spliced ​​feature map. In this way, redundant calculations can be reduced by reusing the high-level feature map, and the shallow feature map can be updated in real time to retain the generation details, thereby accelerating the reasoning without affecting the generation quality and quickly obtaining a clean sample.

[0072] In one embodiment, the noise prediction process includes:

[0073] If the noise prediction model performs forward propagation at the denoising step number t, the cache upsampling block upsamples the denoised samples at the denoising step number t to obtain the high-level feature map.

[0074] In the denoising step number tn, a shallow feature map is generated by the shallow network in the noise prediction model; the shallow feature map is generated based on the denoising samples to be denoised in the denoising step number tn.

[0075] The shallow feature map and the high-level feature map are spliced ​​together to obtain a spliced ​​feature map.

[0076] The concatenated feature map is upsampled using an upsampling block to obtain the predicted noise with the number of denoising steps tn.

[0077] The structure of the noise prediction model is as follows Figure 3 As shown. Among them, the noise prediction model includes a decoder and an encoder, and the decoder and the encoder are connected through a residual block and a spatial transformer. The decoder includes a convolutional layer, a downsampling block, and a residual block. The downsampling block includes two residual blocks, two spatial transformers, and a downsampling layer. The decoder includes a convolutional layer, a spatial transformer, a residual block, an upsampling block, an upsampling layer, and a residual block. The upsampling block includes three residual blocks, three spatial transformers, and an upsampling layer. There is a jump connection between the convolutional layer of the encoder and the convolutional layer of the decoder, a jump connection between the downsampling block of the encoder and the spatial transformer and residual block of the decoder, a jump connection between the downsampling block of the encoder and the upsampling block of the decoder, and a jump connection between the residual block of the encoder and the upsampling layer and residual block of the decoder. X2 means there are two, and X3 means there are 3. For example Figure 3 The X2 in the encoder part indicates that there are two residual blocks and two spatial transformers in the downsampling block. Figure 4 Schematic diagram of reusing high-level feature maps. Specifically, after the downsampling block 1 generates a shallow feature map, it is directly spliced ​​with the cached high-level feature map. The spliced ​​feature map is upsampled by the upsampling block 1 and the predicted noise is output.

[0078] When performing noise prediction, the current denoising step number is denoising step number t, and forward propagation is performed at denoising step number t. Then, the sample to be denoised of denoising step number t and the first noise level schedule of denoising step number t are input into the noise prediction model for noise prediction, or based on the second noise level schedule of denoising step number t, the proportion of remaining original information in the sample to be denoised of denoising step number t is calculated, and based on the second noise level schedule of denoising step number t-1, the proportion of remaining original information in the sample to be denoised of denoising step number t-1 is calculated, and the square root of the proportion and the sample to be denoised of denoising step number t are input into the noise prediction model for noise prediction.

[0079] The noise prediction model used to predict noise adopts a high-level feature map reuse mechanism. When using the high-level feature map reuse mechanism, each time the noise prediction model performs a forward propagation, the cached high-level feature map is directly reused in the subsequent N-1 step denoising process, and only the shallow feature map is updated. Specifically, the following steps are included:

[0080] Step 1: Perform the forward propagation in the reverse denoising step number t completely, and cache the high-level feature map obtained by sampling the denoising samples of the denoising step number t by the specified upsampling block;

[0081] Step 2: In the subsequent N-1-step denoising process, that is, in the denoising step tn, the high-level feature maps cached in the denoising step t are reused and the high-level calculations of the main branch of the noise prediction model are skipped. Only the shallow network of the noise prediction model is used to generate shallow feature maps.

[0082] Step 3: Splice the shallow feature map and the high-level feature map to obtain a spliced ​​feature map, and use the upsampling block of the noise prediction model to upsample the spliced ​​feature map to obtain the predicted noise of the denoising step number tn. The expression of the spliced ​​feature map is , Represents the shallow feature map sampled by the mth layer downsampling block when the denoising step number is t. It represents the high-level feature map collected by the upsampling block in the m+1th layer when the denoising step number t of forward propagation is performed.

[0083] In this embodiment, a shallow feature map is generated by a shallow network in the noise prediction model in the denoising step tn, the shallow feature map and the high-level feature map are spliced ​​to obtain a spliced ​​feature map, and the spliced ​​feature map is upsampled using an upsampling block to obtain the predicted noise of the denoising step tn. This can reduce redundant calculations by reusing high-level feature maps, and at the same time, update the shallow feature maps in real time to retain the generation details, thereby accelerating the reasoning without affecting the generation quality and quickly obtaining clean samples.

[0084] In one embodiment, Figure 5 As shown in the figure, based on the sample to be denoised, the noise prediction model is used to perform noise prediction and sample purification to obtain a clean sample after purification, including:

[0085] S201. When the current denoising step number is denoising step number t, use the formula Calculate the first noise level schedule for denoising step number t ;in, represents the curvature parameter, represents the maximum noise, represents the minimum noise, and T is the total number of denoising steps.

[0086] The first noise level scheduling is applied to adversarial examples obtained through adversarial attack on the CIFAR-10 dataset (Canadian Institute For Advanced Research - 10 Class Dataset). The CIFAR-10 dataset contains 60,000 32x32 color images in 10 categories, with 6,000 images in each category.

[0087] Sample cleanup is performed within the diffusion model. Specifically, when the adversarial example is a sample obtained from the CIFAR-10 dataset through an adversarial attack, it is fed into the EDM (Elucidated Diffusion Models) for sample cleanup, which includes adding random noise.

[0088] The first noise level scheduling is a nonlinear noise level scheduling that adapts to changes in step size. During the denoising process, a large step size is used in the early stages of denoising to quickly generate a rough overall result. Later, a smaller step size is used to refine image details, accelerating convergence without sacrificing sampling quality.

[0089] Curvature parameters It can be used to adjust the first noise level schedule to select the first noise level schedule corresponding to the optimal result. and minimum noise Can be used to control the range of the first noise level schedule.

[0090] S202: Scheduling the first noise level The sample to be denoised with the denoising step number t is input into the noise prediction model for noise prediction to obtain the first predicted noise with the denoising step number t.

[0091] Among them, when performing noise prediction, it will first determine whether the current denoising step number is still within the preset cache interval. If the cache step number has not been used up, the cached high-level feature map can be directly reused for noise prediction without obtaining the high-level feature map again.

[0092] S203: Determine the noise direction of the denoising step number t based on the first predicted noise.

[0093] When determining the noise direction of the denoising step number t, the first predicted noise of the denoising step number t is used.

[0094] S204: First noise level scheduling based on noise direction, denoising step number t, and denoising step number t-1 and the first noise level schedule of denoising step number t , determine the samples to be denoised with denoising step number t-1.

[0095] Among them, the first noise level scheduling of denoising step number t-1 The formula can be get, represents the curvature parameter, represents the maximum noise, represents the minimum noise, and T is the total number of denoising steps.

[0096] S205. Update the current denoising step number in step S201 to denoising step number t-1, update the denoising step number t-1 in step S204 to denoising step number t-2, and repeat steps S201, S202, S203 and S204 until the iterative step number meets the first preset condition and outputs a clean sample.

[0097] The first preset condition refers to the number of iteration steps reaching the first preset number of steps. If the number of iteration steps currently reached the first preset number of steps, a clean sample will be directly output without repeating steps S201, S202, S203, and S204. If the number of iteration steps currently did not reach the first preset number of steps, steps S201, S202, S203, and S204 will be repeated until the number of iteration steps currently reached the first preset number of steps, and a clean sample will be output.

[0098] When repeating steps S201, S202, S203, and S204, the current denoising step number will be updated, and the obtained first noise level schedule, first predicted noise, noise direction, and samples to be denoised will also be updated with the update of the current denoising step number. Specifically, when the current denoising step number is denoising step number t, if it is determined that the iteration step number does not meet the first preset condition, the current denoising step number is updated to t-1, and the formula is used Calculate the first noise level schedule for denoising step number t-1 ;in, represents the curvature parameter, represents the maximum noise, Represents the minimum noise, T is the total number of denoising steps; the first noise level is scheduled The samples to be denoised with the denoising step number t-1 are input into the noise prediction model for noise prediction to obtain the first predicted noise of the denoising step number t-1; based on the first predicted noise, the noise direction of the denoising step number t-1 is determined; based on the noise direction, the samples to be denoised with the denoising step number t-1, and the first noise level scheduling of the denoising step number t-2 And the first noise level schedule of denoising step number t-1 , determine the sample to be denoised with denoising step number t-2; judge whether the number of iteration steps meets the first preset condition when the current denoising step number is denoising step number t-1, and if so, output the sample to be denoised with denoising step number t-2 as a clean sample.

[0099] In this embodiment, sample purification is performed by using nonlinear first noise level scheduling, which can better balance the sampling efficiency under different noise levels, concentrate more sampling steps in the stage with lower noise level, and thus generate higher quality samples in fewer sampling steps. In addition, nonlinear noise scheduling can be better combined with the ordinary differential equation framework in the back diffusion process, thereby optimizing the denoising process to quickly obtain clean samples.

[0100] In one embodiment, the noise direction of the denoising step number t in step S203 is calculated as follows:

[0101] ,

[0102] Where d is the noise direction of the denoising step number t, is the sample to be denoised with denoising step number t, d t is the first predicted noise of denoising step number t, is the first noise level schedule for the denoising step number t.

[0103] In this embodiment, by calculating the noise direction, a filter can be better designed to remove or weaken the noise in a specific direction, thereby better reducing the noise of the sample to be denoised.

[0104] In one embodiment, the calculation formula for the samples to be denoised with denoising step number t-1 in step S204 is:

[0105] ,

[0106] in, is the sample to be denoised with denoising step number t-1, is the sample to be denoised with denoising step number t, is the first noise level schedule for denoising step number t-1, is the first noise level schedule for denoising step number t, and d is the noise direction.

[0107] In one embodiment, Figure 6 As shown in the figure, based on the sample to be denoised, the noise prediction model is used to perform noise prediction and sample purification to obtain a clean sample after purification, including:

[0108] S301. Obtaining a preset noise level scheduling sequence ,in The second noise level schedule representing the number of denoising steps n.

[0109] Among them, the second noise level scheduling is applicable to adversarial samples obtained after the ImageNet dataset is processed by adversarial attacks. The ImageNet dataset contains more than 14 million images, which are annotated into 1,000 categories.

[0110] Sample purification is performed within the diffusion model. Specifically, when the adversarial example belongs to the ImageNet dataset, it is fed into the Guided Diffusion Model for sample purification, which includes adding random noise.

[0111] Noise level scheduling sequence The second noise level schedule in the noise level schedule sequence is a linear noise level schedule.

[0112] S302. When the current denoising step number is denoising step number t, based on the second noise level scheduling of denoising step number t, calculate the proportion of remaining original information in the samples to be denoised with denoising step number t, and based on the second noise level scheduling of denoising step number t-1, calculate the proportion of remaining original information in the samples to be denoised with denoising step number t-1.

[0113] The proportion of the remaining original information in the sample to be denoised with denoising step number t refers to the proportion of the original un-noised information still retained in the current sample to be denoised at the t-th step in the reverse generation process.

[0114] As the denoising steps proceed, that is, the t value decreases, the proportion increases, which means that more original information is recovered and the residual noise in the sample to be denoised is reduced. Therefore, the proportion of the remaining original information in the sample to be denoised with denoising step number t-1 is greater than the proportion of the remaining original information in the sample to be denoised with denoising step number t.

[0115] S303 : Input the square root of the ratio and the sample to be denoised with the denoising step number t into a noise prediction model to perform noise prediction, and obtain a second predicted noise with the denoising step number t.

[0116] Among them, when performing noise prediction, it will first determine whether the current denoising step number is still within the preset cache interval. If the cache step number has not been used up, the cached high-level feature map can be directly reused for noise prediction without obtaining the high-level feature map again.

[0117] S304. Use formula , calculate the predicted clean samples predicted at the denoising step number t ;in, is the sample to be denoised with denoising step number t, is the ratio corresponding to the number of denoising steps t, is the second predicted noise after denoising step number t.

[0118] The predicted clean sample is the clean sample predicted at the current denoising step. However, the predicted clean sample obtained at this time is not the final clean sample, because the sample at this time is far away from the real clean sample.

[0119] Proportion It is the proportion of the remaining original information in the sample to be denoised with denoising step number t.

[0120] S305 : Calculate a first weight of the random noise and a second weight of the second predicted noise based on the proportion of the remaining original information in the samples to be denoised, respectively, at the denoising step number t and the denoising step number t-1.

[0121] S306, Use formula , determine the denoising samples to be denoised with denoising step number t-1 , where c1 represents the first weight, ε represents random noise, and c2 represents the second weight, represents the second predicted noise of denoising step number t, is the ratio corresponding to the denoising step number t-1.

[0122] S307. Update the current denoising step number in step S302 to denoising step number t-1, update the denoising step number t-1 in step S302 to denoising step number t-2, and repeat steps S302, S303, S304, S305 and S306 until the iterative step number meets the second preset condition and outputs a clean sample.

[0123] The second preset condition refers to the number of iteration steps reaching the second preset number of steps. The first preset number of steps and the second preset number of steps may be the same or different.

[0124] If the current number of iterated steps reaches the second preset number of steps, a clean sample will be directly output without repeating steps S302, S303, S304, S305 and S306; if the current number of iterated steps does not reach the second preset number of steps, steps S302, S303, S304, S305 and S306 will be repeated until the current number of iterated steps reaches the second preset number of steps and a clean sample is output.

[0125] When repeating step S302, step S303, step S304, step S305 and step S306, the current denoising step number in step S302 will be updated, and the obtained second noise level schedule, second predicted noise, and samples to be denoised will also be updated as the current denoising step number is updated. Specifically, when the current denoising step number is denoising step number t, and the iterative step number does not meet the second preset condition, the current denoising step number is updated to t-1. Based on the second noise level schedule of denoising step number t-1, the proportion of the remaining original information in the samples to be denoised of denoising step number t-1 is determined. Based on the second noise level schedule of denoising step number t-2, the proportion of the remaining original information in the samples to be denoised of denoising step number t-2 is calculated; the square root of the proportion and the samples to be denoised of denoising step number t-1 are input into the noise prediction model for noise prediction to obtain the second predicted noise of denoising step number t-1; the formula is used. , calculate the predicted clean sample predicted at denoising step t-1 ;in, is the sample to be denoised with denoising step number t-1, is the ratio corresponding to the denoising step number t-1, is the second predicted noise of denoising step number t-1; based on the proportion of the remaining original information in the samples to be denoised at denoising step number t-1 and denoising step number t-2, the first weight of the random noise and the second weight of the second predicted noise are calculated; using the formula , determine the denoising samples to be denoised with denoising step number t-2 , where c1 represents the first weight, ε represents random noise, and c2 represents the second weight, represents the second predicted noise of denoising step number t-1, is the ratio corresponding to the denoising step number t-2; when the current denoising step number is the denoising step number t-1, it is determined whether the iterative step number meets the second preset condition. If so, the denoised sample with the denoising step number t-2 is taken as a clean sample and the clean sample is output.

[0126] In this embodiment, clean samples are outputted by using a linear second noise level schedule, so that denoising can be performed at a constant rate throughout the purification process, so that the output clean samples are as close as possible to the original noise-free samples.

[0127] In one embodiment, step S302 includes:

[0128] Using the formula Calculate the proportion of the remaining original information in the denoising sample with denoising step number t .

[0129] Using the formula Calculate the proportion of the remaining original information in the denoising sample with denoising step number t-1 , is the intermediate parameter corresponding to the denoising step number t, is the intermediate parameter corresponding to the denoising step number t-1.

[0130] in, represents the second noise level schedule with denoising step number t, The second noise level schedule represents the denoising step number t-1.

[0131] In one embodiment, the calculation formula of the first weight c1 is:

[0132] ,

[0133] The calculation formula of the second weight c2 is:

[0134] ,

[0135] in, is the ratio corresponding to the number of denoising steps t, is the ratio corresponding to the denoising step number t-1.

[0136] In some embodiments, the sample purification method further includes: performing an adversarial attack on the clean sample to obtain an adversarial sample; inputting the purified clean sample into the classification model to evaluate the purification result of the clean sample based on the output classification result. Figure 7 shown.

[0137] In some embodiments, the method used in this application can also be compared with the prior art to obtain comparative results. Specifically, the purification performance is evaluated on the NVIDIA Jetson AGX Xavier edge device. Table 1 shows the purification performance and inference speed of ME-Net (Marix Estimation Net, matrix estimation network), LightPure (lightweight purification model), DiffPure (Diffusion-based Input Purification, diffusion purification model), GDMP (Guided Diffusion Model Purification, guided diffusion model purification method), ScoreOPT (Score Optimization, scoring optimization), MimicDiffusion (mimic diffusion model), and this application on the CIFAR-10 dataset. Table 2 shows the purification performance and inference speed of different purification methods on the ImageNet dataset. Tables 1 and 2 are as follows:

[0138]

[0139] As can be seen from Table 1, on the CIFAR-10 dataset, this application achieves the highest PGD_EOT accuracy compared to other methods, and also achieves a relatively high accuracy for the BPDA_EOT adaptive attack. Although the accuracy is slightly lower than other purification methods based on diffusion models, compared with these methods, the inference speed of this application reaches 45.6ms / img, which is more than 10 times that of other methods. At the same time, it also achieves an accuracy of 87% on clean data.

[0140] As shown in Table 2, this application also demonstrates good defense performance on the ImageNet dataset, with significantly faster inference speed than DiffPure. Its performance is significantly better than the NRP (Neural Representation Purifier) ​​method, achieving a balance between performance and inference latency.

[0141] Therefore, from the experimental results of the two data sets, it can be seen that the present application still maintains a high accuracy on clean data, and also shows good defense performance when facing different adversarial attacks, has good robustness, and the reasoning speed is significantly faster than the existing purification method based on the diffusion model. Based on the characteristics of adversarial robustness and fast reasoning, the present application can adapt well to the deployment requirements of edge devices and show excellent performance in edge computing scenarios.

[0142] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0143] Based on the same inventive concept, embodiments of the present application also provide a sample purification device for implementing the aforementioned sample purification method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more sample purification device embodiments provided below can be found in the above-described limitations of the sample purification method and will not be further elaborated here.

[0144] In one embodiment, Figure 8 As shown, a sample purification device is provided, comprising:

[0145] The sample acquisition module is used to obtain adversarial samples from edge devices and add random noise to the adversarial samples to obtain samples to be denoised;

[0146] The sample purification module is used to perform noise prediction and sample purification based on the sample to be denoised using the noise prediction model to obtain a clean sample after purification; the noise prediction model adopts a high-level feature map reuse mechanism, which is used to reuse the high-level feature map sampled by the denoising step number t into the denoising step number tn, and splice it with the shallow feature map generated by the shallow network of the noise prediction model to obtain a spliced ​​feature map, n∈[1,2,…,N-1], where N is the preset cache interval.

[0147] Each module in the sample purification device described above can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a computer device memory in software form, so that the processor can call and execute the corresponding operations of each module.

[0148] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 9 As shown. The computer device includes a processor, a memory and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store adversarial samples, random noise, samples to be denoised, clean samples, high-level feature maps, and preset cache intervals. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a sample purification method is implemented.

[0149] Those skilled in the art will understand that Figure 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0150] In one embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0151] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0152] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0153] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), data processing logic devices based on quantum computing, and the like.

[0154] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0155] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A sample purification method, characterized in that: The method comprises: S101: Obtain an adversarial sample from an edge device and add random noise to the adversarial sample to obtain a sample to be denoised; S102. Based on the sample to be denoised, use a noise prediction model to perform noise prediction and sample purification to obtain a purified clean sample; the noise prediction model adopts a high-level feature map reuse mechanism, and the high-level feature map reuse mechanism is used to reuse the high-level feature map sampled by the denoising step number t into the denoising step number tn, so as to splice it with the shallow feature map generated by the shallow network of the noise prediction model to obtain a spliced ​​feature map, n∈[1,2,…,N-1], where N is a preset cache interval; The noise prediction process includes: If the noise prediction model performs forward propagation at the denoising step number t, the cache upsampling block upsamples the denoised sample at the denoising step number t to obtain a high-level feature map; In the denoising step number tn, a shallow feature map is generated by a shallow network in the noise prediction model; the shallow feature map is generated based on the sample to be denoised in the denoising step number tn; Splicing the shallow feature map and the high-level feature map to obtain a spliced ​​feature map; Use an upsampling block to upsample the spliced ​​feature map to obtain the predicted noise of the denoising step number tn; The method of performing noise prediction and sample purification based on the sample to be denoised using a noise prediction model to obtain a purified clean sample includes: S201. When the current denoising step number is denoising step number t, use the formula Calculate the first noise level schedule of the denoising step number t ;in, represents the curvature parameter, represents the maximum noise, represents the minimum noise, T is the total number of denoising steps; S202: Schedule the first noise level The sample to be denoised with the denoising step number t is input into the noise prediction model to perform noise prediction, and a first predicted noise with the denoising step number t is obtained; S203, determining the noise direction of the denoising step number t based on the first predicted noise; S204: First noise level scheduling based on the noise direction, the samples to be denoised with the denoising step number t, and the denoising step number t-1. And the first noise level schedule of the denoising step number t , determine the samples to be denoised with the denoising step number t-1; S205. Update the current denoising step number in step S201 to the denoising step number t-1, update the denoising step number t-1 in step S204 to the denoising step number t-2, and repeat steps S201, S202, S203 and S204 until the iterative step number meets the first preset condition and outputs a clean sample.

2. The method according to claim 1, characterized in that The noise direction of the denoising step number t in step S203 is calculated as follows: ， Wherein, d is the noise direction of the denoising step number t, is the sample to be denoised with the denoising step number t, d t is the first predicted noise of the denoising step number t, is the first noise level schedule for the denoising step number t.

3. The method according to claim 1, characterized in that The calculation formula for the sample to be denoised with the denoising step number t-1 in step S204 is: ， in, is the sample to be denoised with the denoising step number t-1, is the sample to be denoised with the denoising step number t, is the first noise level schedule of the denoising step number t-1, is the first noise level schedule of the denoising step number t, and d is the noise direction.

4. The method according to claim 1, wherein The method of performing noise prediction and sample purification based on the sample to be denoised using a noise prediction model to obtain a purified clean sample includes: S301. Obtaining a preset noise level scheduling sequence ,in The second noise level schedule representing the number of denoising steps n; S302: When the current denoising step number is denoising step number t, based on the second noise level schedule for denoising step number t, calculate the proportion of remaining original information in the sample to be denoised for denoising step number t, and based on the second noise level schedule for denoising step number t-1, calculate the proportion of remaining original information in the sample to be denoised for denoising step number t-1; S303: Input the square root of the ratio and the sample to be denoised with the denoising step number t into a noise prediction model to perform noise prediction, thereby obtaining a second predicted noise with the denoising step number t; S304. Use formula , calculate the predicted clean samples predicted at the denoising step number t ;in, is the sample to be denoised with the denoising step number t, is the ratio corresponding to the denoising step number t, is the second predicted noise of the denoising step number t; S305: Calculate a first weight of the random noise and a second weight of the second predicted noise based on the proportion of remaining original information in the to-be-denoised samples of the denoising step number t and the denoising step number t-1, respectively; S306, Use formula , determine the denoising samples to be denoised with denoising step number t-1 , where c1 represents the first weight, ε represents the random noise, and c2 represents the second weight, represents the second predicted noise of the denoising step number t, is the ratio corresponding to the denoising step number t-1; S307. Update the current denoising step number in step S302 to denoising step number t-1, update the denoising step number t-1 in step S302 to denoising step number t-2, and repeat steps S302, S303, S304, S305 and S306 until the iterative step number meets the second preset condition and outputs a clean sample.

5. The method according to claim 4, characterized in that Step S302 includes: Using the formula Calculate the proportion of the remaining original information in the denoising sample with denoising step number t ; Using the formula Calculate the proportion of the remaining original information in the denoising sample with denoising step number t-1 ; in, represents the second noise level schedule of the denoising step number t, represents the second noise level schedule of the denoising step number t-1, is the intermediate parameter corresponding to the denoising step number t, is the intermediate parameter corresponding to the denoising step number t-1.

6. The method according to claim 4, characterized in that The calculation formula of the first weight c1 is: ， The calculation formula of the second weight c2 is: ， in, is the ratio corresponding to the denoising step number t, is the ratio corresponding to the denoising step number t-1.

7. A sample purification device, characterized in that: The device comprises: A sample acquisition module is used to obtain adversarial samples from edge devices and add random noise to the adversarial samples to obtain samples to be denoised; A sample purification module is configured to perform noise prediction and sample purification based on the sample to be denoised using a noise prediction model to obtain a purified clean sample; the noise prediction model adopts a high-level feature map reuse mechanism, which is configured to reuse the high-level feature map sampled by the denoising step number t into the denoising step number tn, so as to splice it with the shallow feature map generated by the shallow network of the noise prediction model to obtain a spliced ​​feature map, n∈[1,2,…,N-1], where N is a preset cache interval; The noise prediction process includes: If the noise prediction model performs forward propagation at the denoising step number t, the cache upsampling block upsamples the denoised sample at the denoising step number t to obtain a high-level feature map; In the denoising step number tn, a shallow feature map is generated by a shallow network in the noise prediction model; the shallow feature map is generated based on the sample to be denoised in the denoising step number tn; Splicing the shallow feature map and the high-level feature map to obtain a spliced ​​feature map; Use an upsampling block to upsample the spliced ​​feature map to obtain the predicted noise of the denoising step number tn; The method of performing noise prediction and sample purification based on the sample to be denoised using a noise prediction model to obtain a purified clean sample includes: S201. When the current denoising step number is denoising step number t, use the formula Calculate the first noise level schedule of the denoising step number t ;in, represents the curvature parameter, represents the maximum noise, represents the minimum noise, T is the total number of denoising steps; S202: Schedule the first noise level The sample to be denoised with the denoising step number t is input into the noise prediction model to perform noise prediction, and a first predicted noise with the denoising step number t is obtained; S203, determining the noise direction of the denoising step number t based on the first predicted noise; S204: First noise level scheduling based on the noise direction, the samples to be denoised with the denoising step number t, and the denoising step number t-1. And the first noise level schedule of the denoising step number t , determine the samples to be denoised with the denoising step number t-1; S205. Update the current denoising step number in step S201 to the denoising step number t-1, update the denoising step number t-1 in step S204 to the denoising step number t-2, and repeat steps S201, S202, S203 and S204 until the iterative step number meets the first preset condition and outputs a clean sample.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

Images