A PRGGNet-based adversarial attack sample generation method
By explicitly modeling the similarity and conflict relationships of gradients among multiple proxies using the PRGGNet framework and combining the output statistical features to generate adversarial examples, the problem of gradient conflict and insufficient adaptability in existing transfer attacks is solved, achieving more efficient attack effects and transferability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- GUILIN UNIV OF ELECTRONIC TECH
- Filing Date
- 2026-03-09
- Publication Date
- 2026-05-29
AI Technical Summary
Existing migration attack methods suffer from limitations in attack success rate and migration capability due to simple superposition of gradients from multiple proxies, insufficient utilization of conflict information, and weak dynamic adaptation.
We adopt the PRGGNet two-stage learnable perturbation fusion framework, explicitly model the similarity and conflict relationships of multiple agent gradients through ProposalNet, and combine the output statistical features to achieve conditional alignment and refined fusion of candidate perturbations.
It improves the accuracy and transferability of adversarial sample generation, enhances the stealth and engineering versatility of adversarial samples, and has the ability to dynamically adjust to different attack states.
Smart Images
Figure CN122116043A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of artificial intelligence security and robustness evaluation of deep learning models. Specifically, it relates to a method for generating transferable adversarial examples for classification models by using multi-agent gradient conflict relationship modeling and a two-stage learnable fusion network. Background Technology
[0002] With the widespread deployment of deep models such as convolutional neural networks and visual Transformers in scenarios such as image classification, object recognition, and autonomous driving, the sensitivity of these models to minute input perturbations has gradually become an important issue in the field of intelligent system security. Samples that can induce incorrect predictions by superimposing minute perturbations imperceptible to the human eye onto the original image are often referred to as adversarial examples.
[0003] In real-world black-box attack scenarios, attackers often find it difficult to directly obtain the parameters and gradients of the target model. Therefore, they typically employ a transfer attack strategy, which involves first constructing perturbations on one or more surrogate classification models, and then leveraging the transferability of these perturbations to the target model with an unknown structure to carry out the attack. Existing transfer attack methods often use simple averaging of gradients from a single model, multiple surrogate models, fixed weighting, or empirical rule-based fusion methods to obtain the perturbation direction.
[0004] However, different proxy classification models often produce gradients for the same input sample that differ significantly in direction, scale, and local regions of interest, sometimes even conflicting with each other. Directly averaging these gradients can easily cause effective gradients to cancel each other out, making it difficult to fully preserve discriminative information with cross-model transferability. Furthermore, using static weights fails to dynamically adjust the fusion strategy based on the current iteration state, prediction confidence, and changes in class boundaries, thus limiting attack success rate and transferability.
[0005] Meanwhile, traditional methods often treat gradients as single numerical tensors, lacking utilization of the statistical features of the surrogate model output. They fail to jointly consider changes in loss value, prediction confidence, class margin, and output distribution, making it difficult for the perturbation update process to adaptively respond to state differences between different surrogate classification models. Therefore, there is an urgent need to propose an adversarial example generation method that can simultaneously model the conflict relationships of surrogate gradients, statistical state information, and candidate perturbation sets. Summary of the Invention
[0006] The purpose of this invention is to overcome the problems of simple superposition of multiple agent gradients, insufficient utilization of conflict information, and weak dynamic adaptation ability in existing transfer attack methods, and to provide an adversarial attack sample generation method based on PRGGNet. This method constructs a two-stage learnable perturbation fusion framework using ProposalNet and RefineNet. During the iteration process, it explicitly models the similarity and conflict relationships of multiple agent gradients, and combines output statistical features to achieve conditional alignment and refined fusion of candidate perturbations.
[0007] A method for generating adversarial attack samples based on PRGGNet, characterized by the following steps:
[0008] 1) Obtain the original image to be attacked The original image is processed into a three-channel 224×224 resolution input image; real labels are configured according to the attack task. and optional target tags It also includes multiple pre-built proxy classification models, target classification models, and a PRGGNet network composed of ProposalNet and RefineNet;
[0009] 2) Using the original image Centered on the maximum disturbance radius The input image is randomly initialized to obtain the current iteration image. And crop the pixel values to the [0,1] range;
[0010] 3) In the t-th iteration, the current iteration image is... Input each proxy classification model separately, and calculate the gradient map of the loss function of each proxy classification model with respect to the input image for the current iteration image. And construct corresponding statistical feature vectors based on the logits output by each agent classification model. ;
[0011] 4) Convert each gradient plot With corresponding statistical eigenvectors Input ProposalNet, model the relationships between gradients from multiple agents, and generate multiple proposal gradient graphs. ;
[0012] 5) Transform the multiple proposal gradient maps With corresponding statistical eigenvectors Input RefineNet, output the fusion weights corresponding to the proposed gradient graph. The final fused gradient map is obtained by fusing the pixels in the pixel space. ;
[0013] 6) According to the attack mode, follow the final fusion gradient map The sign direction is updated in the current iteration image. And project the updated image onto the original image. Centered on, with radius of Within the perturbation constraint set, pixel value cropping is performed again to obtain the image for the next iteration. ;
[0014] 7) Repeat steps 3) to 6) until the preset number of iterations T is reached, and output the final iterative image. As an adversarial example.
[0015] 2. The method according to claim 1, characterized in that the random initialization in step 2) is performed through an interval Internal to original image After adding uniform random perturbation, pixel cropping and... Projection to enhance the diversity of starting points for attack iterations. And reduce the probability of getting trapped in local optima.
[0016] 3. The method according to claim 1, characterized in that the statistical feature vector in step 3) The loss value and the first type probability are given by the loss value and the first type probability. It consists of the margin, mean, and standard deviation (std), and is mapped to a fixed 16-dimensional statistical feature through zero-padding or truncation, as explained below:
[0017] A. Loss value: The value of the loss function of the proxy attack model under the alternative attack target;
[0018] B. Type I probability The probability value corresponding to the predicted category output by the proxy attack model;
[0019] C. Margin: The difference between the logit of the current attack tag and the logit of the optimal non-current attack tag;
[0020] D. Mean: The mean of all elements in the corresponding logit.
[0021] E. Standard deviation (std): The standard deviation is calculated by taking the absolute values of all elements of the corresponding logit.
[0022] 4. The method according to claim 1, characterized in that, in step 3), each gradient map Each sample underwent L2 normalization independently. Specifically, the 3×224×224 gradient plot was first flattened according to channel and spatial dimensions, and then... The lower bound is used to truncate the corresponding L2 norm, and the truncated L2 norm is used to normalize the gradient graph to eliminate the difference in gradient scale between different surrogate classification models.
[0023] 5. The method according to claim 1, characterized in that, in step 4), the ProposalNet includes a gradient fusion encoder, an information fusion encoder, a relation modeling module, and a conditional alignment module; the gradient fusion encoder includes at least two 3×3 convolutions with a stride of 2 and one 3×3 convolution with a stride of 1, and each convolution is followed by 8 sets of normalization and summation. Activation function.
[0024] 6. The method according to claim 4, characterized in that, in step 4), the relationship modeling module constructs an attention matrix A and a conflict gating matrix gate based on the pairwise cosine similarity between each proxy token, wherein the attention matrix A is configured according to... Calculate the conflict gating matrix (gate) according to... The mixture matrix M is calculated, and the diagonal elements of the two matrices are set to zero. The mixture is then multiplied element by element to obtain the mixed matrix M, which is then normalized by row and used to aggregate the conflict context vector.
[0025] 7. The method according to claim 4, characterized in that the condition alignment module in step 4) adopts... Modulation method: Generate gamma and beta parameters consistent with the number of encoded feature channels based on the conflict context vector, and then... The encoded features corresponding to each agent classification model are modulated, and then refined by 3×3 convolution to output an aligned feature map.
[0026] 8. The method according to claim 1, wherein the RefineNet in step 5) includes a gradient fusion encoder, an information fusion encoder, a Transformer encoder, and a weight prediction network; the Transformer encoder adopts a batch-first approach to perform set modeling on multiple proposal tokens, and includes two encoding layers, four multi-head attention heads, and a feedforward network with a dimension of 512.
[0027] 9. The method according to claim 7, wherein the weight prediction network in step 5) is a two-layer fully connected network, the first layer having an output dimension of 128 and employing... The activation function has a second-layer output dimension of 1; multiple proposed tokens are processed by the weight prediction network to obtain multiple scalar logits, which are then normalized to fusion weights using softmax. .
[0028] 10. The method according to claim 1, characterized in that, in step 5), before performing a weighted summation on the multiple proposal gradient maps, the gradient maps of each proposal are first... Perform L2 normalization or sign mapping to make the final fused gradient map The scale is independent of the absolute magnitude of a single proposed gradient map, thus stabilizing the subsequent iterative update process.
[0029] 11. The method according to claim 1, wherein the image update in step 6) adopts a non-targeted attack mode. In target attack mode, ;in Indicates that simultaneously satisfy Projection operation with perturbation constraints and pixel value range [0,1] constraints.
[0030] Compared with existing technologies, the present invention has at least the following advantages: 1) Jointly encoding the statistical features and gradient space features output by the proxy model can more accurately represent the state of each proxy classification model in the current iteration round; 2) By using cosine similarity-driven relationship modeling and conflict gating mechanism, the utilization of conflict gradient information is explicitly enhanced, avoiding the cancellation of effective directions caused by simple averaging; 3) By performing channel-wise modulation on the encoded features through the FiLM conditional alignment module, the consistency between the proposed gradient map and the current attack state is improved; 4) By performing global correlation modeling on the candidate proposal set through Transformer, the fusion weight is adaptively predicted, improving the stability, mobility, and concealment of the final perturbation direction; 5) The same attack framework can be compatible with both targeted and non-targeted attacks, and has strong engineering versatility. Attached Figure Description
[0031] Figure 1 This is a schematic diagram of the PRGGNet two-stage network framework.
[0032] Figure 2 This is a schematic diagram of the overall process of generating adversarial samples.
[0033] Figure 3 This is a schematic diagram of the gradient fusion encoder structure.
[0034] Figure 4 This is a schematic diagram of the information fusion encoder structure.
[0035] Figure 5 A schematic diagram of the relation modeling module structure.
[0036] Figure 6 This is a schematic diagram of the conditional alignment module structure.
[0037] Figure 7This is a schematic diagram of the feature decoder structure.
[0038] Figure 8 This is a schematic diagram of the weight prediction module structure. Detailed Implementation
[0039] To more clearly illustrate the purpose, technical solution, and advantages of this invention, the following detailed description will be provided in conjunction with specific examples.
[0040] This embodiment uses an input size of 3×224×224, the number of surrogate classification models K=4, the statistical feature dimension stat_dim=16, the maximum perturbation radius eps=8 / 255, the single step size alpha=2 / 255, and the number of iterations T=10 as an example for illustration, but the present invention is not limited to this set of parameter configurations.
[0041] A method for generating adversarial attack samples based on PRGGNet, such as Figure 1 and Figure 2 As shown, the PRGGNet adversarial attack sample generation method of this invention consists of four main stages: multi-proxy gradient extraction, ProposalNet proposal generation, RefineNet refinement and fusion, and constrained iterative update. ProposalNet is responsible for mining conflict and complementary information from multiple proxy gradients, while RefineNet performs global fusion at the candidate perturbation set level, ultimately outputting a perturbation direction that simultaneously considers attack strength and transferability.
[0042] Step 1: Input preparation and disturbance initialization.
[0043] Obtain the original image to be attacked The pixel values are normalized to the [0,1] range and then adjusted to a resolution of 224×224. For non-targeted attacks, the true class label of the original image is recorded. For targeted attacks, an additional target label is given in addition to the true class label y. Furthermore, K proxy classification models, one classification model as the attack target, and pre-trained ProposalNet and RefineNet parameters are pre-configured.
[0044] Before the formal iteration, to improve the diversity of attack starting points, the original image was processed within the interval [-eps,eps]. Add uniform random noise and perform pixel cropping and Projection yields the initial iterative image. The correspondence is shown in formula (1):
[0045] (1)
[0046] in, This represents a uniform random perturbation tensor with the same shape as the input image. This means limiting the pixels to the range [0,1]. This indicates projecting the current image onto the original image. Centered on a radius of eps Constrained within the sphere.
[0047] Step 2: Proxy gradient and statistical feature extraction.
[0048] In the t-th iteration, the current image Input K proxy classification models respectively The logits output of each proxy classification model is obtained. For non-targeted attacks, the cross-entropy loss is constructed using the real label y; for targeted attacks, the cross-entropy loss is constructed using the target label. The two loss calculation forms are shown in formulas (2) and (3), respectively:
[0049] (2)
[0050] (3)
[0051] After determining the loss function, the gradient of the loss function with respect to the input image is calculated to obtain the input gradient map corresponding to each proxy classification model. As shown in formula (4):
[0052] (4)
[0053] In addition to gradient graphs, this invention also extracts statistical features from the logits output by the surrogate classification model to help determine the prediction status of different surrogate classification models on the current sample. Specifically, the logits are processed... Obtain the category probability distribution, and take the maximum probability. The margin is obtained by taking the difference between the logit corresponding to the current attack label and the logit of the best non-current attack label; then the mean and standard deviation of the logits are calculated, and the loss value is combined to form the basic statistical vector.
[0054] The basic statistical vectors are expanded into fixed 16-dimensional statistical features after being zero-padded or truncated. Its expression is shown in formula (5):
[0055] (5)
[0056] in, This means zero-padding is performed on basic statistical vectors with dimensions lower than 16, and truncation is performed on statistical vectors with dimensions higher than 16, to ensure that different agent classification models have a uniform statistical feature dimension in subsequent networks. Table 1 gives the definition of the basic statistics.
[0057] Step 3: ProposalNet proposal phase.
[0058] ProposalNet is used to generate multiple higher-quality proposal gradient maps from multiple proxy gradients. Its structure is as follows: Figure 1 As shown.
[0059] First, for each gradient map Sample-independent L2 normalization is performed to mitigate the impact of differences in gradient scales among different surrogate classification models on subsequent relationship modeling. The normalization form is shown in formula (6):
[0060] (6)
[0061] Subsequently, the normalized gradient maps are fed into the gradient fusion encoder. The gradient fusion encoder employs a three-layer convolutional cascade structure: the first and second layers are 3×3 convolutions with a stride of 2, used to progressively downsample the input resolution from 224×224 to 56×56; the third layer is a 3×3 convolution with a stride of 1, used to map the channel dimension to 128. Each convolutional layer is followed by a group of normalization and SiLU activation functions to improve stability and nonlinear expression capabilities in small-batch scenarios.
[0062] Mean pooling is performed on the encoded feature maps in the spatial dimension to obtain 128-dimensional gradient feature vectors corresponding to each proxy classification model; simultaneously, 16-dimensional statistical features are... Inputting a two-layer fully connected information fusion encoder yields a 128-dimensional statistical embedding vector. The two vectors are then added together to form the proxy token, as shown in formula (7):
[0063] (7)
[0064] After obtaining all proxy tokens, calculate the cosine similarity between any two tokens to obtain a K×K similarity matrix, as shown in formula (8):
[0065] (8)
[0066] Based on the cosine similarity matrix, an attention matrix A and a conflict gating matrix gate are further constructed. The attention matrix emphasizes information propagation between similar agents, while the conflict gating matrix emphasizes the utilization of conflict information between agents with significant differences. Their calculation forms are shown in formulas (9) and (10), respectively.
[0067] (9)
[0068] (10)
[0069] In this embodiment, alpha is 8.0, beta is 12.0, and tau is 0.0. When the cosine similarity is less than 0, the conflict gating matrix increases the weight of the corresponding proxy pair, thereby preferentially retaining the conflict directions with complementarity. Further, the diagonal elements of the attention matrix A and the conflict gating matrix gate are set to zero, and the elements are multiplied one by one to obtain the mixture matrix M. After row normalization, it is used to aggregate the conflict context vector. The calculation form is shown in formula (11) and formula (12):
[0070] (11)
[0071] (12)
[0072] The resulting conflict context vector The encoded features are mapped to conditional vectors through a two-layer fully connected conditional generation network, and then channel-by-channel modulation is performed on the encoded features by the FiLM conditional alignment module. Specifically, the conditional vectors generate a sequence of channels with the same number of channels. and And calibrate the encoded features according to formula (13):
[0073] (13)
[0074] Finally, the aligned feature map is input into the feature decoder. The feature decoder first bilinearly upsamples the 56×56 feature map to 224×224, then performs two 3×3 convolutions and one 1×1 convolution to restore it to the three-channel gradient map size, and outputs the proposed gradient map. As shown in formula (14):
[0075] (14)
[0076] Table 2 shows the main structural configuration of ProposalNet. Through the above processing, ProposalNet no longer directly uses the original proxy gradient as the final update direction, but first generates a set of proposal gradient maps after conflict resolution and condition alignment, providing a more discriminative set of candidate perturbations for subsequent refinement and fusion.
[0077] Step 4: RefineNet Refinement Stage.
[0078] RefineNet is used to further perform global correlation modeling at the level of the proposed gradient graph set. Internally, it includes a gradient fusion encoder and a weight prediction module, with the corresponding local structures as follows: Figure 3 and Figure 8 As shown.
[0079] First, the gradient maps of each proposal output by ProposalNet are... Input a gradient fusion encoder with the same structure as in step 3 to obtain a proposal feature map; then perform spatial mean pooling on the proposal feature map to obtain a 128-dimensional proposal feature vector, and add it to the 128-dimensional statistical embedding generated by the information fusion encoder to form a proposal token.
[0080] Multiple proposal tokens are grouped into a sequence of length K and input into a Transformer encoder to model the global dependencies between candidate proposals. The Transformer encoder contains two encoding layers, each configured with four multi-head attention heads, and the feedforward network has a dimension of 512, implemented using batch_first and norm_first methods. The Transformer fusion process can be expressed as formula (15):
[0081] (15)
[0082] Each fusion token output by the Transformer is input into a two-layer fully connected weight prediction network to obtain a scalar logit, which is then normalized to fusion weights using softmax. The calculation form is shown in formula (16).
[0083] (16)
[0084] After obtaining the fusion weights, a psi mapping is performed on each proposal gradient map. This embodiment uses L2 normalization as the psi mapping; that is, each proposal gradient map is first normalized to L2 for each sample, and then a weighted sum is performed according to the fusion weights to obtain the final fused gradient map. As shown in formula (17):
[0085] (17)
[0086] In addition, RefineNet can optionally output a confidence scalar, conf, to characterize the reliability of the fusion result of the current proposal set. In implementations requiring adaptive step size control, the confidence scalar can be used to adjust the perturbation update magnitude, but in this embodiment, the fixed step size alpha is retained. Table 3 shows the main structural configurations of RefineNet.
[0087] Step 5: Constrained iterative update and adversarial example output.
[0088] The final fusion gradient map is obtained in each iteration. Then, the current image is updated along the gradient sign direction according to the attack mode. For non-target attacks, the update is performed in the positive direction to increase the true class loss; for target attacks, the update is performed in the negative direction to decrease the target class loss. The update relationship is shown in formula (18):
[0089] (18)
[0090] Non-targeted attacks are represented by a positive sign, and targeted attacks by a negative sign. After completing the current update step, the image is reprojected onto a plane centered at the original image x0 with a radius of eps. Within the perturbation constraint sphere, pixel interval cropping is performed again to obtain the next iteration image that meets the constraints. This process is shown in formula (19):
[0091] (19)
[0092] After T iterations, the output is... As the final adversarial example, since the proxy gradient and statistical features are recalculated in each iteration and the two-stage fusion is performed again through ProposalNet and RefineNet, the perturbation direction generated by this invention can be dynamically adjusted according to the attack state, rather than being updated using a one-time static gradient. Therefore, it is more suitable for robustness testing of complex target models in transfer attack scenarios.
[0093] In one specific embodiment, a proxy model set can be constructed using four proxy classification models with different structures. These proxy models can be selected from image classification networks such as ResNet, DenseNet, MobileNet, or Vision Transformer. The target classification model does not participate in gradient backpropagation and is only used to evaluate the success rate of transfer attacks that generate adversarial examples. This invention does not limit the specific network structures of the proxy and target classification models; any model capable of outputting classification logits is applicable.
[0094] Those skilled in the art should understand that, without departing from the concept of this invention, the number of proxy models K, the number of feature channels, the statistical feature dimension, the number of Transformer layers, the step size alpha, the maximum perturbation radius eps, and the psi mapping method can all be equivalently adjusted according to the specific task. Based on the core principles of this invention, any other implementation methods derived from the concept of this invention should be considered to fall within the protection scope of this invention.
[0095] Table 1: Definitions of Basic Statistics
[0096] Statistic Name Definition method effect loss The cross-entropy loss value of the current proxy classification model Characterizing the optimization pressure under the current attack target top1_prob The maximum value in the softmax probability Characterizing the current prediction confidence of the surrogate classification model margin The difference between the current attack label logit and the optimal non-current attack label logit Reflects the relative distance of the sample to the current attack classification boundary mean logits is the mean of all elements. Describe the overall bias of the output distribution. std The standard deviation of all elements of logits Describe the degree of dispersion of the output distribution
[0097] Table 2: ProposalNet Structure
[0098] Module Name Detailed structural information Gradient fusion encoder 1) Input: 3×224×224 gradient map; 2) Convolutional layer 1: 3→64, kernel=3×3, stride=2, padding=1 + GroupNorm(8) + SiLU; 3) Convolutional layer 2: 64→64, kernel=3×3, stride=2, padding=1 + GroupNorm(8) + SiLU; 4) Convolutional layer 3: 64→128, kernel=3×3, stride=1, padding=1 + GroupNorm(8) + SiLU; 5) Output: 128×56×56 feature map. Information Fusion Encoder 1) Input: 16-dimensional statistical features; 2) Fully connected layer: 16→256 + SiLU; 3) Fully connected layer: 256→128; 4) Output: 128-dimensional statistical embedding vector. Conflict Modeling Module 1) Perform global average pooling on the encoded features to obtain a 128-dimensional gradient token; 2) Add the gradient token to the statistical embedding element by element; 3) Calculate the K×K cosine similarity matrix; 4) Generate the attention matrix A by softmax(alpha×cos); 5) Generate the conflict gating matrix gate by sigmoid(beta×(tau-cos)); 6) Set the diagonal to zero and normalize by row to obtain the mixture matrix M. Weight Calculation Module 1) Fully connected layer: 128 → 128 + SiLU; 2) Fully connected layer: 128 → 128; 3) Output: 128-dimensional condition vector for each agent. Conditional alignment module 1) Gamma generation layer: Linear 128→128; 2) Beta generation layer: Linear 128→128; 3) Perform channel-wise modulation; 4) Thinned by 3×3 convolution + GroupNorm(8) + SiLU; 5) Output: 128×56×56 aligned feature map. Feature Decoder 1) Bilinear interpolation upsampling: 56×56→224×224; 2) Convolutional layer 1: 128→128, kernel=3×3, padding=1 + GroupNorm(8) + SiLU; 3) Convolutional layer 2: 128→64, kernel=3×3, padding=1 + GroupNorm(8) + SiLU; 4) Convolutional layer 3: 64→3, kernel=1×1; 5) Output: 3×224×224 proposal gradient map.
[0099] Table 3: RefineNet Structure
[0100] Module Name Detailed structural information Gradient fusion encoder 1) Input: 3×224×224 proposed gradient map; 2) Convolutional layer 1: 3→64, kernel=3×3, stride=2, padding=1 + GroupNorm(8) + SiLU; 3) Convolutional layer 2: 64→64, kernel=3×3, stride=2, padding=1 + GroupNorm(8) + SiLU; 4) Convolutional layer 3: 64→128, kernel=3×3, stride=1, padding=1 + GroupNorm(8) + SiLU; 5) Output: 128×56×56 feature map. Information Fusion Encoder 1) Input: 16-dimensional statistical features; 2) Fully connected layer: 16→256 + SiLU; 3) Fully connected layer: 256→128; 4) Output: 128-dimensional statistical embedding vector. Token Fusion 1) Add the candidate gradient direction map encoding result to the statistical embedding element by element; 2) Form a 128-dimensional token sequence of length K; 3) Use it as the input feature of the weight prediction module. Weight prediction module 1) TransformerEncoderLayer × 2; 2) d_model = 128, nhead = 4; 3) dim_feedforward = 512; 4) batch_first = True, norm_first = True; 5) Output scalar logit through Linear 128 → 128 + SiLU + Linear 128 → 1; 6) Perform softmax normalization on K logit to obtain the fusion weight w. Weighted fusion module <![CDATA[1) Perform psi mapping on each proposed gradient map; 2) Preferably use L2 norm normalization as psi mapping; 3) Perform weighted summation according to weight W i ; 4) Output: a 3×224×224 final fused gradient map.]]>
Claims
1. A method for generating adversarial attack samples based on PRGGNet, characterized in that, Includes the following steps: 1) Obtain the original image to be attacked The original image is processed into a three-channel 224×224 resolution input image; real labels are configured according to the attack task. and optional target tags It also includes multiple pre-built proxy classification models, target classification models, and a PRGGNet network composed of ProposalNet and RefineNet; 2) Using the original image Centered on the maximum disturbance radius The input image is randomly initialized to obtain the current iteration image. And crop the pixel values to the [0,1] range; 3) In the t-th iteration, the current iteration image is... Input each proxy classification model separately, and calculate the gradient map of the loss function of each proxy classification model with respect to the input image for the current iteration image. And construct corresponding statistical feature vectors based on the logits output by each proxy classification model. ; 4) Convert each gradient plot With corresponding statistical eigenvectors Input ProposalNet, model the relationships between gradients from multiple agents, and generate multiple proposal gradient graphs. ; 5) Transform the multiple proposal gradient maps With corresponding statistical eigenvectors Input RefineNet, output the fusion weights corresponding to the proposed gradient graph. The final fused gradient map is obtained by fusing the pixels in the pixel space. ; 6) According to the attack mode, follow the final fusion gradient map The sign direction is updated in the current iteration image. And project the updated image onto the original image. Centered on, with radius of Within the perturbation constraint set, pixel value cropping is performed again to obtain the image for the next iteration. ; 7) Repeat steps 3) to 6) until the preset number of iterations T is reached, and output the final iterative image. As an adversarial example.
2. The method according to claim 1, characterized in that, The random initialization in step 2) is achieved through the interval Internal to original image After adding uniform random perturbation, pixel cropping and... Projection is used to enhance the diversity of attack iteration starting points and reduce the probability of getting trapped in local optima.
3. The method according to claim 1, characterized in that, The statistical feature vector mentioned in step 3) The loss value and the first type probability are given by the loss value and the first type probability. It consists of the margin, mean, and standard deviation (std), and is mapped to a fixed 16-dimensional statistical feature through zero-padding or truncation, as explained below: A. Loss value: The value of the loss function of the proxy attack model under the alternative attack target; B. Type I probability The probability value corresponding to the predicted category output by the proxy attack model; C. Margin: The difference between the logit of the current attack tag and the logit of the optimal non-current attack tag; D. Mean: The mean of all elements in the corresponding logit. E. Standard deviation (std): The standard deviation is calculated by taking the absolute values of all elements of the corresponding logit.
4. The method according to claim 1, characterized in that, Gradient plots in step 3) Each sample underwent L2 normalization independently. Specifically, the 3×224×224 gradient plot was first flattened according to channel and spatial dimensions, and then... The lower bound is used to truncate the corresponding L2 norm, and the truncated L2 norm is used to normalize the gradient graph to eliminate the difference in gradient scale between different surrogate classification models.
5. The method according to claim 1, characterized in that, The ProposalNet mentioned in step 4) includes a gradient fusion encoder, an information fusion encoder, a relation modeling module, and a conditional alignment module; the gradient fusion encoder includes at least two 3×3 convolutions with a stride of 2 and one 3×3 convolution with a stride of 1, and each convolution is followed by 8 sets of normalization and summation. Activation function.
6. The method according to claim 4, characterized in that, In step 4), the relationship modeling module constructs an attention matrix A and a conflict gating matrix gate based on the pairwise cosine similarity between each proxy token. The attention matrix A is then configured according to... Calculate the conflict gating matrix (gate) according to... The mixture matrix M is calculated, and the diagonal elements of the two matrices are set to zero. The mixture is then multiplied element by element to obtain the mixed matrix M, which is then normalized by row and used to aggregate the conflict context vector.
7. The method according to claim 4, characterized in that, The conditional alignment module in step 4) uses Modulation method: Generate gamma and beta parameters consistent with the number of encoded feature channels based on the conflict context vector, and then... The encoded features corresponding to each agent classification model are modulated, and then refined by 3×3 convolution to output an aligned feature map.
8. The method according to claim 1, characterized in that, Step 5) RefineNet includes a gradient fusion encoder, an information fusion encoder, and a weight prediction module. The weight prediction module uses a batch-first approach to model multiple proposed tokens as a set, and includes two encoding layers, four multi-head attention heads, and a feedforward network with a dimension of 512.
9. The method according to claim 7, characterized in that, The weight prediction network in step 5) is a two-layer fully connected network. The first layer has an output dimension of 128 and uses... The activation function has a second-layer output dimension of 1; multiple proposed tokens are processed by the weight prediction network to obtain multiple scalar logits, which are then normalized to fusion weights using softmax. .
10. The method according to claim 1, characterized in that, In step 5), before performing a weighted summation of multiple proposal gradient graphs, the gradient graphs of each proposal are first processed. Perform L2 normalization or sign mapping to make the final fused gradient map The scale is independent of the absolute magnitude of a single proposed gradient map, thus stabilizing the subsequent iterative update process.
11. The method according to claim 1, characterized in that, The image update in step 6) is performed in non-targeted attack mode. In target attack mode, ;in It means that both conditions are met. Projection operation with perturbation constraints and pixel value range [0,1] constraints.