Adversarial sample transferability enhancement method based on neighborhood variance adaptive weight

Abstract

The application discloses a method for enhancing transferability of adversarial samples based on neighborhood variance adaptive weight, relates to the technical field of adversarial attack, and comprises the following steps: in step S01, in a black-box attack scene, a substitute model which is consistent with a target model task and whose parameters are known is introduced; in step S02, original samples with real class labels are obtained, and adversarial samples are initialized; in step S03, neighborhood color perturbation samples are constructed on the basis of the current adversarial samples, and semantic correction is combined to inhibit non-semantic drift; in step S04, the adversarial samples are iteratively updated, the final adversarial samples are input into a target model which does not participate in generation for reasoning, the proportion of predicted labels deviating from real labels on the target model is counted, and the average value of the proportion on each target model is recorded as a transfer success rate. The application improves the robustness of a deep neural network in the face of diversified perturbation and complex defense mechanism and the transferability of attack samples.

Description

Technical Field

[0001] This invention relates to the field of adversarial attack technology, and in particular to an adversarial sample mobility enhancement method based on neighborhood variance adaptive weights. Background Technology

[0002] With the rapid development of deep learning technology, deep neural networks (DNNs) have made groundbreaking progress in fields such as computer vision and natural language processing. However, deep neural networks are highly vulnerable to adversarial examples. Adding subtle perturbations to the original input, which are imperceptible to humans, can significantly reduce the model's predictive performance. This characteristic not only exposes the model's security vulnerabilities but also provides an important direction for studying model robustness and interpretability. Research on adversarial attack techniques has gradually become a hot topic in the field of artificial intelligence security, with applications covering adversarial training, model robustness evaluation, and privacy protection.

[0003] To address these vulnerabilities, researchers have proposed various adversarial attack strategies to analyze and test the robustness of models. Currently, the mainstream adversarial attack methods are mainly based on gradient optimization strategies, iteratively updating the adversarial perturbation along the gradient direction of the loss function. However, while current mainstream adversarial attack methods can effectively break through the decision boundaries of deep neural networks in white-box scenarios through gradient optimization, achieving a near 100% success rate, their transferability remains severely limited in black-box attack scenarios. Perturbations formed by methods such as input transformation, multi-model ensemble, and optimization of perturbation direction currently exhibit strong local concentration and structural repetition, making them easily learned and "overfitted" by specific defense mechanisms. This leads to a significant decrease in the success rate of cross-model attacks, especially in cross-architecture scenarios where the attack effect is more unstable and the generalization ability is extremely weak.

[0004] To improve attack transferability, some studies have attempted to introduce diverse strategies such as input transformation and multi-scale perturbation to enrich the perturbation distribution. However, most of these strategies adopt static or pre-defined transformation methods, lacking dynamic perception of the target model's response behavior and unable to adjust the perturbation direction and intensity according to predicted fluctuations. Due to the lack of adaptive mechanisms, these methods often suffer from reduced attack efficiency and decreased stability while improving transferability, making it difficult to balance accuracy and generalization ability. Summary of the Invention

[0005] In view of the above-mentioned shortcomings and deficiencies of the existing technology, the present invention provides an adversarial sample transferability enhancement method based on neighborhood variance adaptive weights. By integrating neighborhood consistency regularization, semantic perturbation guidance and adaptive gradient weighting mechanism, the method improves the robustness of deep neural networks in the face of diverse perturbations and complex defense mechanisms and the transferability of attack samples. It is particularly suitable for black-box attacks and cross-model adversarial scenarios.

[0006] To achieve the above objectives, the main technical solutions adopted by the present invention include: The adversarial sample transferability enhancement method based on neighborhood variance adaptive weights includes the following steps: Step S01: In the black-box attack scenario, introduce an alternative model that is consistent with the target model task and whose parameters are known, as a proxy carrier for attack and robustness evaluation. Step S02: Obtain the original samples with real class labels, initialize the adversarial samples, and determine the default hyperparameters for optimization; Step S03: Construct neighborhood color perturbation samples based on the current adversarial samples, and combine semantic correction to suppress non-semantic drift; guide the perturbation to converge in a transferable and stable direction by maximizing the weighted combination of the neighborhood consistency regularization loss and the main sample loss. Step S04: Iteratively update the adversarial examples. Input the final adversarial examples into the target models that were not generated for inference. Calculate the proportion of predicted labels deviating from the true labels on the target models. Record the average of this proportion on each target model as the transfer success rate, which is used to quantify the transferability and generalization ability of the method across models.

[0007] Further, step S01 includes setting the black-box target model as f, selecting an accessible model g consistent with the task, using the decision boundary of f to replace model g for attack optimization adversarial perturbation, and transferring the obtained adversarial sample to f.

[0008] Further, step S02 includes scaling each sample, normalizing the pixel values ​​to [0,1], and then initializing the adversarial samples.

[0009] Furthermore, in step S02, the default hyperparameters for optimization are determined, including setting the number of iterations T=10 and the step size. , This is the upper limit of the perturbation radius, which is the maximum magnitude of change allowed for each pixel.

[0010] Furthermore, in step S03, constructing neighborhood color perturbation samples based on the current adversarial samples includes the following steps: Step S311: Apply uniform, random, small-amplitude pixel perturbations to the input adversarial examples: ; in, This represents the i-th neighborhood perturbation sample generated during the t-th iteration; This represents the adversarial example at the t-th iteration; This represents the neighborhood perturbation range of the i-th sample at the t-th iteration; This indicates the upper limit of the perturbation radius, which is the maximum magnitude of change allowed for each pixel; This is a factor for adjusting the intensity of the neighborhood disturbance range; This indicates that uniformly distributed neighborhood perturbation samples are independently generated from the interval. The value is taken from n; n is the number of neighborhood samples; Step S312: For each neighborhood perturbation sample Perform color gamut perturbation transformation: ; in, The color perturbation sample generated in the t-th iteration is represented by β; Colortrans() represents the perturbation transformation function; b β c β s These are the adjustment ranges for controlling brightness, contrast, and saturation, collectively referred to as hue perturbation factors.

[0011] Furthermore, in step S03, combining semantic correction to suppress non-semantic drift includes the following steps: Step S321, Adversarial Examples With real category labels Cross-entropy loss Calculate the gradient value: ; Step S322: Obtain the result through channel-space mean absolute value normalization. To obtain the normalized direction ; Step S323: Calculate the cosine similarity between the current adversarial sample and the color perturbation sample: ; in, Let represent the cosine similarity between the i-th color perturbation sample and the current adversarial sample in the pixel vector space at the t-th iteration; Represents vectorization operators; Step S324: The calculated similarity values ​​between the color perturbation samples and the adversarial samples are transformed to a minimum interval through a linear mapping. The similarity after the linear transformation is: ,but: ; in, This represents a linear similarity mapping function, yielding the final color perturbation samples. This yields a set of color perturbation samples for the "neighborhood term". .

[0012] Furthermore, the minimum interval is [0, 0.01].

[0013] Furthermore, in step S03, the perturbation is guided to converge in a transferable and stable direction by maximizing the weighted combination of the neighborhood consistency regularization loss and the main sample loss, including: Step S331: Combine adversarial samples with neighborhood color perturbation samples. Regularize the neighborhood color perturbation samples to penalize the current adversarial sample. The target loss function is: ; in, To counter the target function, the attack should maximize it. Let be the cross-entropy loss function of the current adversarial example; Indicates the true label of the original sample; Parameters for the alternative model; For control coefficients, This represents the mean loss of n neighborhood color perturbation samples; This represents the neighborhood color perturbation vector relative to the current adversarial example; Step S332: Introduce dynamic equilibrium coefficient Alternative The model adaptively adjusts the ratio of the gradient of the current adversarial sample to that of the neighboring color perturbation sample based on the output fluctuation, and calculates the model output for each neighboring color perturbation sample. The variance of the output within its neighborhood is taken as the "local instability index". After normalization, the instability coefficient is obtained. : .

[0014] Further, step S04 includes the following steps: Step S401: Calculate the gradient of the objective function, varying it along the direction of increasing loss. The resulting gradient expression is: ; With step size Combine gradient-based adversarial examples: ; Where L is the adversarial objective function. This represents the adversarial example at the (t+1)th iteration; Represents a symbolic function; Indicates the iteration at time t. The gradient of the joint objective with respect to the input is calculated at the given location; Step S402, to Crop to the displayable area: ; in, This represents the intermediate image obtained in the (t+1)th iteration, a candidate adversarial sample that has not undergone cropping and projection operations; For trimming operators; The cropped result is projected onto a surface centered at the original clean sample x, with a radius of... of On the sphere, we obtain the adversarial examples for this iteration: ; in, This represents a point centered at x with a radius of... of ball; Represents a set The projection operator; when When the final adversarial sample is obtained T is the number of iterations; Step S403: Obtain the final adversarial sample Input a set of target black-box models that did not participate in the generation process Perform forward reasoning one by one; M1, ..., Mk represent different target attack models, and k is the number of models to be attacked; Step S404: For each model Statistical indicator attack success rate (ASR): ; Where N is the number of evaluation samples, For indicator functions; This is the final adversarial sample corresponding to the j-th sample. ; The true label of the j-th original clean sample; Step S405: Incorporate the results into uncertainty measurement and security constraints to form a systematic and robust evaluation result of the target model in an unlabeled black-box scenario.

[0015] The beneficial effects of this invention are: This invention proposes an adversarial example generation method with neighborhood perturbation consistency and adaptive adjustment mechanism, aiming to improve the migration ability and attack stability of adversarial examples between different models. This invention first samples multiple perturbation images in the neighborhood of the adversarial example and applies joint transformations such as brightness, contrast, and saturation to these perturbation samples to enhance the input diversity of the perturbation. Based on this, a perturbation direction adjustment mechanism based on cosine similarity is introduced to further optimize the exploration capability of the perturbation direction, guiding the perturbation into a flatter and more robust region. Then, the output variance of the prediction results of these perturbation samples in the target model is calculated to measure the stability among the perturbation samples, and this stability is mapped to a balance factor in the loss function to dynamically adjust the relationship between attack strength and stability. Finally, an adaptive loss function that integrates the mean of the perturbation loss and the variance of the predicted output is constructed to effectively suppress the oscillation and uncertainty of the perturbation direction, improving the overall transfer performance of the adversarial example. Through a gradient-based projection update strategy, an efficient and stable attack process is achieved while ensuring controllable perturbation amplitude. This method not only performs well in white-box attacks but also achieves a high success rate of transfer attacks when facing black-box models with unknown structures or defense strategies, demonstrating good universality and scalability, further enhancing its practical value. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the adversarial sample mobility enhancement method based on neighborhood variance adaptive weights of the present invention; Figure 2 A visual comparison of the adversarial perturbations of the original sample, the adversarial sample generated by the PGN method, and the adversarial sample generated by the method of this invention. Detailed Implementation

[0017] To better explain and facilitate understanding of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0018] This invention provides a method for enhancing the transferability of adversarial examples based on neighborhood variance adaptive weights. Specifically, it is a method that integrates neighborhood consistency regularization, semantic perturbation guidance, and adaptive gradient weighting mechanism. It aims to improve the robustness of deep neural networks in the face of diverse perturbations and complex defense mechanisms, as well as the transferability of attack examples. It is particularly suitable for black-box attacks and cross-model adversarial scenarios.

[0019] The present invention provides a method for enhancing the mobility of adversarial samples based on adaptive weights of neighborhood variance, comprising the following steps: Step S01: In the black-box attack scenario, in order to approximate its decision boundary, an alternative model with parameters that are consistent with the target model task is introduced as a proxy carrier for attack and robustness evaluation.

[0020] Step S02: Obtain the original samples with real class labels and initialize the adversarial samples accordingly to determine the number of iteration rounds for subsequent optimization of default hyperparameters. Step length .

[0021] Step S03: Using an alternative model as a proxy, a neighborhood consistency-enhanced adversarial perturbation generation strategy is proposed: diverse neighborhood instances and perceptual transformations of brightness, contrast, and saturation are constructed on the local input of the current adversarial example, and semantic correction is combined to suppress non-semantic drift. By maximizing the weighted combination of neighborhood consistency regularization loss and main sample loss, the perturbation is guided to converge in a transferable and stable direction.

[0022] Step S04: Iteratively update the adversarial examples, input the final adversarial examples into the target model that did not participate in the generation for inference, and count the proportion of predicted labels deviating from the true labels on the target model. The average of this proportion on each target model is recorded as the transfer success rate, which is used to quantify and evaluate the transferability and generalization ability of the method across models.

[0023] The following is a detailed analysis of the features of this method and a flowchart of the adversarial example generation process. Figure 1 As shown: Let the black-box target model be Select an accessible model that aligns with the task. to approximate The decision boundary. Subsequent attacks in the alternative model. Optimize the adversarial perturbation and transfer the resulting adversarial examples to To test its effectiveness and robustness, original image samples with true class labels are obtained from the evaluation set. Each sample undergoes a scaling transformation, and the pixel values ​​are normalized to [value missing]. The system is then initialized with adversarial examples and default hyperparameters are set for subsequent optimizations. To balance efficiency and effectiveness, the number of iteration rounds is set. Step length .

[0024] Around the current adversarial examples The specific process of developing a neighborhood consistency-enhanced adversarial perturbation generation strategy is as follows: First, construct neighborhood color perturbation samples based on the current adversarial sample. Then, transform the input adversarial sample through the following steps, starting with uniform and random small-amplitude pixel perturbations: ; in Indicates the first The generation during the nth iteration One neighborhood disturbance sample; Indicates the first Adversarial examples in the next iteration; Indicates the first During the nth iteration The neighborhood perturbation range of each sample; This indicates the upper limit of the perturbation radius, which is the maximum magnitude of change allowed for each pixel; This is a factor for adjusting the intensity of the neighborhood disturbance range; This indicates that uniformly distributed neighborhood perturbation samples are independently generated from the interval. Take the value from; This represents the number of neighborhood samples. Because in reality, there are situations where local inputs are uncertain, neighborhood sampling is used to increase the diversity of input scenarios.

[0025] Then, for each neighborhood perturbation sample Perform color gamut perturbation transformation: ; in Indicates the first The generation during the nth iteration One color perturbation sample; Represents the perturbation transformation function; The adjustment ranges for brightness, contrast, and saturation, which are controlled separately, are collectively referred to as hue perturbation factors. The color gamut transformation process described above is used to simulate the changes in image brightness, contrast, and saturation during real-world shooting.

[0026] To avoid introducing non-semantic information shifts into the image through color perturbation, the current color perturbation sample is semantically fine-tuned and corrected using the gradient information of the original input adversarial sample. With real category labels Cross-entropy loss The gradient values ​​are calculated as follows: ; Then, the "channel-space mean absolute value normalization" process is performed to obtain... Finally, the normalized direction is obtained. ,in The calculation formula is: Calculate the cosine similarity between the current adversarial sample and the color-perturbed sample: ; in Indicates the first During the nth iteration The cosine similarity between a color perturbation sample and the current adversarial sample in the pixel vector space; Cosine similarity calculation formula, range of values , The vectorization operator flattens a tensor into a one-dimensional vector in a fixed order.

[0027] The similarity values ​​between the color perturbation samples and adversarial samples calculated above are transformed into a minimum interval through a linear mapping. The similarity after linear transformation is denoted as , Let the similarity linear mapping function be represented, then Represented as: ; The final color perturbation sample is obtained after semantic-guided fine-tuning and correction using cosine similarity. This yields a set of color perturbation samples for the "neighborhood term". .

[0028] The core idea of ​​the neighborhood variance-based adaptive weight consistency adversarial example transferability enhancement strategy is not only to improve the model's performance on original input adversarial examples, but also to enhance its transferability across different adversarial examples. To prevent misclassification, this method aims to ensure that all samples in its neighborhood also misclassify in the same direction. Therefore, it employs a combined approach of adversarial examples and neighborhood color perturbation samples. Regularization of the neighborhood color perturbation samples penalizes the current adversarial example, forcing it and its surrounding samples to robustly change in the same direction. This is reflected in the target loss function as follows: ; in The adversarial objective function designed for this invention maximizes this value during an attack; Let be the cross-entropy loss function of the current adversarial example; Indicates the true label of the original sample; The parameters of the alternative model are kept constant during the attack; This is a control coefficient used to control the strength of the "difference penalty" between the average loss of the current adversarial sample and the neighboring color perturbation samples; express The mean loss of each neighborhood color perturbation sample; This represents the neighborhood color perturbation vector relative to the current adversarial example.

[0029] The first term in the objective loss function of this method makes the current adversarial sample difficult to classify, and the second term makes the "average loss of neighboring color perturbation samples" approach the "loss of the current sample", making the samples in the neighborhood of the current adversarial sample also difficult to classify, thereby improving transferability.

[0030] Due to direct fixation The possibility of overlooking the situation where "some neighborhood samples have stable gradients while others have steep gradients" is overlooked, so a dynamic balance coefficient is introduced. To replace static The model adaptively adjusts the ratio of the gradient of the current adversarial sample to that of the neighboring color perturbation sample based on the output fluctuation, and calculates the model output for each neighboring color perturbation sample. The variance of these outputs within the neighborhood is taken as the "local instability index". After normalization, the instability coefficient is obtained. : ; The greater the fluctuation, the more unstable the neighboring samples. The smaller the value, the more emphasis is placed on the consistency of neighboring color perturbation samples; conversely, the larger the value, the more emphasis is placed on the current adversarial sample.

[0031] This method updates the current adversarial sample by optimizing the target loss function, and then calculates the gradient of the target function, changing it in the direction that increases the loss of the target function.

[0032] The gradient expression after differentiation is: .

[0033] With step size Combining the gradient update adversarial examples above: ; in Indicates the first Adversarial examples in the next iteration; Represents a symbolic function; Indicates the first During iteration The gradient of the joint objective with respect to the input is calculated at the given location.

[0034] During the sample update process described above, the samples are modified in a direction that is more easily misclassified by the model. However, the upper limit of the perturbation radius and the pixel boundary should be considered in this process. Therefore, a cropping operation is needed to legalize the pixel domain, that is... First, crop to the displayable range to avoid exceeding the value range: ; in Indicates the first Intermediate images obtained from the next iteration, candidate adversarial samples without cropping and projection operations; The cropping operator restricts the value of each pixel in the image to the display domain, and then projects the cropped result onto the original clean sample. Centered on, with radius as of On the sphere, we obtain the adversarial examples for this iteration: ; in Indicates Centered on, with radius as of ball; Represents a set The projection operator; when When the final adversarial sample is obtained The final adversarial sample is obtained after iterative updates. Then, the sample is input into the target black-box model set that did not participate in the generation process. Forward reasoning is performed one by one, relying only on the output decision results and not accessing the parameters and gradients.

[0035] For each model Statistical indicator attack success rate (ASR): ; in To measure the sample size, This is an indicator function. : No. The final adversarial sample corresponding to each sample , : No. The true label of the original clean sample.

[0036] By incorporating the above results into uncertainty metrics and security constraints, a systematic and robust evaluation of the target model in an unlabeled black-box scenario is formed.

[0037] Example: This invention conducts experiments on an ImageNet-compatible dataset, which has been widely used in previous research. This dataset contains 1000 images of size 299×299×3, covering 1000 categories. Each image is labeled with both the ground truth label and the target attack label. The experiments cover seven types of deep learning models, including four popular pre-trained models: Inception-v3 (Inc-v3), Inception-v4 (Inc-v4), InceptionResnetv2 (IncRes-v2), and ResNet-101 (Res-101). Adversarial training enhancement models, including Inc-v3-ens3, Inc-v4-ens4, and IncRes-v2-ens, are also considered.

[0038] Six popular gradient-based iterative adversarial attack methods were selected as baselines, including MI-FGSM (Momentum Iterative Fast Gradient Sign Method), NI-FGSM (Nesterov Iterative Fast Gradient Sign Method), VMI-FGSM (Variance-tuning Momentum Iterative FGSM), EMI-FGSM (Enhanced Momentum Iterative FGSM), RAP (Reverse Adversarial Perturbation), and PGN (Penalizing Gradient Norm). The hyperparameter settings are as follows, and all methods follow... Norm constraints set the maximum perturbation of the parameters to... Number of iterations Step length .

[0039] For MI-FGSM and NI-FGSM, the attenuation factor will be... Set to 1.0; for NI-FGSM, acceleration coefficient For VMI-FGSM, the number of samples N is set to 20, and the upper limit of the neighborhood size is set to... upper limit of disturbance Variance weights For EMI-FGSM, set the number of samples N to 11, and the sampling interval limit... Set to 7, and use linear sampling, linear scaling factor .

[0040] For the attack method RAP, the step size Set to 2.0 / 255, iteration count K to 400, internal iteration count T to 10, delayed start KL to 100, and neighborhood size. Set to 16.0 / 255; for the attack method PGN, set the number of samples N to 20, and the balance coefficient... Set to 0.5. The upper limit is set to The sampling number N is set to 35 in the method of this invention. The upper limit is set to The core evaluation metric for the experiment is the attack success rate (ARS). The mean and standard deviation of 10 independent experiments are calculated to ensure the reliability of the results.

[0041] 1. Overall performance comparison: Table 1 shows the attack success rates (% ± standard deviation) of different gradient-based attack methods on multiple models under a single-model setting. It can be observed that in most black-box transfer attack scenarios, i.e., attacking different architectures or different defense models, the method of this invention achieves significant performance improvements. Whether for ordinary models or defense models, it outperforms mainstream methods such as MI-FGSM (MI), NI-FGSM (NI), VMI-FGSM (VIM), EMI-FGSM (EMI), and RAP. For example, when attacking the defense model Inc-v3ens3 using Inc-v3 as the source model, the attack success rate of the method of this invention reaches 65%, while PGN achieves 64.6%, significantly outperforming traditional methods such as MI, NI, and VMI. When attacking IncRes-v2ens using IncRes-v2 as the source model, the success rate reaches 73.8%, which is 7.18% higher than PGN's 66.62%, demonstrating highly competitive transfer performance.

[0042] The method exhibits stable performance in cross-architecture attack tasks. When attacking the Inc series models from Res-101, the method of this invention still maintains a high attack success rate, such as 88.4% for Inc-v3 and 85.6% for Inc-v4, demonstrating good cross-architecture attack transferability. Compared with traditional methods, which generally show a significant decline in cross-architecture attacks, the method of this invention has better transfer robustness.

[0043] Table 1: Transfer success rate (ASR) of various gradient attacks on multiple target models under different alternative models: .

[0044] Table 1 shows the success rate (% ± standard deviation, results of 10 random runs) of various gradient-based attacks without targeting under a single model setting. The adversarial samples were generated on Inc-v3, Inc-v4, IncRes-v2, and Res-101 models using MI-FGSM (MI), NI-FGSM (NI), VMI-FGSM (VMI), EMI-FGSM (EMI), RAP, PGN, and the attack method of this invention. The first four columns represent attack model data, and the last three columns represent defense model data. In the table, * indicates a white-box model, and no * indicates a black-box model.

[0045] 2. Advantages of the defense model: The method exhibits strong transferability across defense enhancement models, particularly strong defense models Inc-v3ens3, Inc-v3ens4, and IncRes-v2ens. The method demonstrates a significant advantage in transfer attack success rate, exceeding that of VMI, EMI, and RAP methods. For instance, when attacking Inc-v3ens3 using IncRes-v3 as the source model, the method achieves a success rate of 65%, a significant improvement over PGN (64.6%) and RAP (56.9%). This demonstrates that, with the support of natural visual change simulation and semantic direction-guided perturbation mechanisms, the method effectively enhances the adaptability of perturbation samples to complex defense mechanisms, thereby strengthening the stability and attack intensity of cross-model transfer attacks.

[0046] 3. White-box attack performance analysis: In white-box attack scenarios, the method of this invention is on par with mainstream methods, achieving an attack success rate of nearly 100%. Although in a very few cases there is a very small difference (less than 0.1%) compared with methods such as MI-FGSM, this difference is negligible and has a limited impact on the overall robustness assessment.

[0047] The comprehensive experimental results show that the method of the present invention has achieved empirical improvements in the above-mentioned experimental scenarios: it achieves higher ASR on most target models; the standard deviation in some experimental scenarios is lower than the mainstream baseline, and the results are more stable. In summary, the method of the present invention significantly improves the transfer attack capability of adversarial examples across models and defense mechanisms through ensemble neighborhood consistency regularization, tone perturbation enhancement and semantic direction guidance mechanism, while taking into account both attack success rate and stability, verifying the effectiveness and practical value of the method in the field of adversarial transferability improvement.

[0048] 4. Visual presentation: This invention selects 1000 clean images and their ground truth labels from the ImageNet validation set, uses Inc-v3 as the source model for attack, and sets the constraint norm as follows: Disruption budget The input is preprocessed and normalized according to the model's standard procedures before entering the attack process. For each clean image, two adversarial methods are visualized, and five images are extracted and presented here, such as... Figure 2 As shown, Figure 2 The leftmost column shows the original clean samples, the middle column shows the adversarial sample images generated using the PGN method, and the last column shows the visualization of the adversarial samples generated by the method of this invention.

[0049] like Figure 2As shown, a comparison of the visualization effects of a clean image, PGN, and the method of this invention is presented: Under the same perturbation budget, the noise generated by PGN appears as a uniform, fine-grained "gritty" texture, with obvious undulations in the texture of the sky and leaf areas from light to dark, and local contrast is weakened; while the perturbation of this invention is more structural and directional, mainly adhering to the semantic edges of the object and large-scale textures such as the outline of a bridge, the highlights and edges of the insect's back, achieving a stronger adversarial effect while maintaining the overall color and details. Intuitively, Ours produces fewer meaningless random noises than PGN, and the sharpness at the edges and scene layers are better preserved, demonstrating the "stable and transferable" perturbation characteristics.

[0050] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Any modifications, alterations, substitutions, and variations made by those skilled in the art to the above embodiments are within the scope of the present invention.

Claims

1. A method for enhancing the mobility of adversarial examples based on adaptive weights of neighborhood variance, characterized in that, Includes the following steps: Step S01: In the black-box attack scenario, introduce an alternative model that is consistent with the target model task and whose parameters are known, as a proxy carrier for attack and robustness evaluation. Step S02: Obtain the original samples with real class labels, initialize the adversarial samples, and determine the default hyperparameters for optimization; Step S03: Construct neighborhood color perturbation samples based on the current adversarial samples, and combine semantic correction to suppress non-semantic drift; guide the perturbation to converge in a transferable and stable direction by maximizing the weighted combination of the neighborhood consistency regularization loss and the main sample loss. In step S03, combining semantic correction to suppress non-semantic drift includes the following steps: Step S321, Adversarial Examples With real category labels Cross-entropy loss Calculate the gradient value: ； Step S322: Obtain the result through channel-space mean absolute value normalization. To obtain the normalized direction ; Step S323: Calculate the cosine similarity between the current adversarial sample and the color perturbation sample: ； in, Indicates the first t During the nth iteration i The cosine similarity between a color perturbation sample and the current adversarial sample in the pixel vector space; Represents vectorization operators; Indicates the first t The generation during the nth iteration i One color perturbation sample; Step S324: The calculated similarity values ​​between the color perturbation samples and the adversarial samples are transformed to a minimum interval through a linear mapping. The similarity after the linear transformation is: ,but: ； in, This represents a linear similarity mapping function, yielding the final color perturbation samples. This yields a set of color perturbation samples for the "neighborhood term". ; Step S04: Iteratively update the adversarial examples. Input the final adversarial examples into the target models that were not generated for inference. Calculate the proportion of predicted labels deviating from the true labels on the target models. Record the average of this proportion on each target model as the transfer success rate, which is used to quantify the transferability and generalization ability of the method across models.

2. The adversarial sample mobility enhancement method based on neighborhood variance adaptive weights according to claim 1, characterized in that: Step S01 includes setting the black-box target model as... f Select an accessible model that aligns with the task. g to approximate f Decision boundary, alternative model g Used to attack optimized adversarial perturbations, and the resulting adversarial samples are transferred to... f .

3. The adversarial sample mobility enhancement method based on neighborhood variance adaptive weights according to claim 1, characterized in that: Step S02 includes scaling each sample, normalizing the pixel values ​​to [0,1], and then initializing the adversarial samples.

4. The adversarial sample mobility enhancement method based on neighborhood variance adaptive weights according to claim 1, characterized in that: In step S02, the default hyperparameters for optimization are determined, including setting the number of iteration rounds. T =10, step size , This is the upper limit of the perturbation radius, which is the maximum magnitude of change allowed for each pixel.

5. The adversarial sample migration enhancement method based on neighborhood variance adaptive weights according to claim 1, characterized in that, In step S03, constructing neighborhood color perturbation samples based on the current adversarial samples includes the following steps: Step S311: Apply uniform, random, small-amplitude pixel perturbations to the input adversarial examples: ； in, Indicates the first t The generation during the nth iteration i One neighborhood disturbance sample; Indicates the first t Adversarial examples in the next iteration; Indicates the first t During the nth iteration i The neighborhood perturbation range of each sample; This indicates the upper limit of the perturbation radius, which is the maximum magnitude of change allowed for each pixel; This is a factor for adjusting the intensity of the neighborhood disturbance range; This indicates that uniformly distributed neighborhood perturbation samples are independently generated from the interval. Take the value from; n The number of samples in the neighborhood; Step S312: For each neighborhood perturbation sample Perform color gamut perturbation transformation: ； in, Indicates the first t The generation during the nth iteration i One color perturbation sample; Colortrans () denotes the perturbation transformation function; β b , β c , β s These are the adjustment ranges for controlling brightness, contrast, and saturation, collectively referred to as hue perturbation factors.

6. The adversarial sample mobility enhancement method based on neighborhood variance adaptive weights according to claim 1, characterized in that: The minimum interval is [0, 0.01].

7. The adversarial sample mobility enhancement method based on neighborhood variance adaptive weights according to claim 1, characterized in that, In step S03, the perturbation is guided to converge in a transferable and stable direction by maximizing the weighted combination of the neighborhood consistency regularization loss and the main sample loss, including: Step S331: Combine adversarial samples with neighborhood color perturbation samples. Regularize the neighborhood color perturbation samples to penalize the current adversarial sample. The target loss function is: ； in, To counter the target function, the attack should maximize it. Let be the cross-entropy loss function of the current adversarial example; Indicates the true label of the original sample; Parameters for the alternative model; For control coefficients, express n The mean loss of each neighborhood color perturbation sample; This represents the neighborhood color perturbation vector relative to the current adversarial example; Step S332: Introduce dynamic equilibrium coefficient Alternative The model adaptively adjusts the ratio of the gradient of the current adversarial sample to that of the neighboring color perturbation sample based on the output fluctuation, and calculates the model output for each neighboring color perturbation sample. The variance of the output within its neighborhood is taken as the "local instability index". After normalization, the instability coefficient is obtained. : 。 8. The adversarial sample mobility enhancement method based on neighborhood variance adaptive weights according to claim 7, characterized in that: Step S04 includes the following steps: Step S401: Calculate the gradient of the objective function, varying it along the direction of increasing loss. The resulting gradient expression is: ； With step size Combine gradient-based adversarial examples: ； Where L is the adversarial objective function. Indicates the first t Adversarial examples at +1 iteration; Represents a symbolic function; Indicates the first t During iteration The gradient of the joint objective with respect to the input is calculated at the given location; Step S402, to Crop to the displayable area: ； in, Indicates the first t Intermediate images obtained after +1 iterations, candidate adversarial examples without cropping and projection operations; For trimming operators; Project the cropped result onto the original clean sample. x Centered on, with radius as of On the sphere, we obtain the adversarial examples for this iteration: ； in, Indicated by x Centered on, with radius as of ball; Represents a set The projection operator; when When the final adversarial sample is obtained T is the number of iterations; Step S403: Obtain the final adversarial sample Input a set of target black-box models that did not participate in the generation process Perform forward reasoning one by one; M1, ..., Mk represent different target attack models, and k is the number of models to be attacked; Step S404: For each model Statistical indicator attack success rate (ASR): ； in, N To measure the sample size, For indicator functions; For the first j The final adversarial sample corresponding to each sample ; For the first j The true label of the original clean sample; Step S405: Incorporate the results into uncertainty measurement and security constraints to form a systematic and robust evaluation result of the target model in an unlabeled black-box scenario.

Images