Low-perceptibility confrontation sample generation method based on human visual characteristic optimization

Abstract

The invention relates to a low perceptible adversarial sample generation method based on human vision optimization, and belongs to the field of artificial intelligence safety. In order to solve the problem that existing adversarial sample disturbance is easily perceived by naked eyes, after an image, a target model and disturbance parameters are initialized, a comprehensive target function fusing classification loss and perception constraint is constructed for iterative optimization. The method specifically comprises the steps that disturbance is controlled to be within a human eye sensitive threshold value through visual brightness perception and color channel differentiation distribution constraint; structural constraints such as gradient regularization, norm limitation and space sparsity are adopted to optimize disturbance distribution; constraint weight parameters are dynamically adjusted to accommodate different image features. And finally, fusing the optimized disturbance with the original image to generate an adversarial sample. According to the method, the visibility of disturbance in brightness, color and texture structures is remarkably reduced while the attack efficiency is guaranteed, the visual difference between the generated adversarial sample and the original image is small, and higher concealment is achieved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence security, and more specifically, to adversarial attack and defense techniques for deep learning models, particularly a method for generating adversarial samples that are highly covert to human vision. Background Technology

[0002] Deep neural networks have made revolutionary progress in many fields such as image recognition and natural language processing. However, research shows that these high-performance models are highly vulnerable to carefully designed, minor modifications to the input data, leading to incorrect judgments—a phenomenon known as adversarial attacks. These attacks pose a serious threat to deep learning systems deployed in security-critical environments.

[0003] Early adversarial example generation methods primarily focused on the effectiveness of the attack, often employing techniques such as Fast Signed Gradient Scaling (FGSM) and Projected Gradient Descent (PGD). These methods typically search for perturbations within the Lp norm that maximize the model loss. While the Lp norm somewhat limits the "size" of the perturbation, it doesn't perfectly align with human visual perception. Even small norm perturbations can introduce easily perceptible noise patterns into smooth regions of an image, or alter the overall tone and brightness distribution, thus reducing the attack's stealth.

[0004] To enhance the visual concealment of adversarial examples, subsequent research has begun to incorporate characteristics of the human visual spectroscopy (HVS). Some works have utilized the differences in HVS sensitivity to different spatial frequencies, concentrating perturbations in high-frequency regions. Others have considered color perception, such as applying perturbations in a color space more consistent with human vision, or leveraging the human eye's insensitivity to specific color changes. Further research has focused on perceptual metrics such as brightness, contrast, and structural similarity. However, existing methods often emphasize one or a few aspects of the HVS, lacking a unified framework that can systematically and dynamically integrate multiple key HVS features and adaptively optimize them. Therefore, there is still room for improvement in balancing visual quality and attack performance in generated adversarial examples, especially when facing complex scenes and diverse models. Generating adversarial examples that are both robust and difficult for human observers to discern remains a challenging problem. Summary of the Invention

[0005] To address the shortcomings of existing adversarial example generation methods in improving attack concealment, particularly their failure to fully and systematically optimize for the complex perceptual characteristics of human vision regarding brightness, color, and spatial structure, resulting in generated adversarial examples still containing traces easily detectable by the human eye, this invention provides a low-imperceptibility adversarial example generation method optimized based on human visual characteristics, comprising the following steps: Step S1: Receive the original digital image and the target deep learning classifier model, and initialize the adversarial perturbation to be applied to the original digital image. Step S2: Generate the adversarial perturbation through an iterative optimization process; Step S3: The adversarial perturbation finally generated by the iterative optimization process is superimposed and fused with the original digital image to obtain the low-perceptibility adversarial sample.

[0006] Optionally, step S2 includes the following steps: Step S21: Based on the target deep learning classifier model and the adversarial perturbation of the current iteration cycle, estimate the gradient of an overall objective function containing classification loss and perceptual constraints relative to the adversarial perturbation using a numerical approximation method or by utilizing the gradient information inside the model. Step S22: Apply one or more constraints based on visual brightness perception to the adversarial perturbation. The constraints utilize the differences in human eye response to different spectral components to adjust the allocation strategy of the perturbation in different color channels and introduce a dynamic brightness adjustment mechanism to suppress significant and easily noticeable abrupt changes in the overall and local brightness of the adversarial sample image. Step S23: Apply one or more structural constraints based on energy functional theory to the adversarial perturbation. The structural constraints comprehensively consider the gradient regularization, spatial sparsity and norm restriction of the perturbation, aiming to improve the smoothness and local concentration of the perturbation distribution and control its overall energy. Step S24: Implement a dynamic parameter adjustment strategy. During the iteration process, based on the evaluation feedback of the attack effect and perception quality, adaptively adjust the intensity of the adversarial perturbation and the weight coefficients of various constraints to maximize visual concealment while ensuring the effectiveness of the attack. Step S25: Update the adversarial perturbation based on the parameters adjusted in step S24 and the gradient estimated in step S21.

[0007] Compared to existing technologies, the method proposed in this invention systematically integrates and dynamically optimizes brightness perception, color channel characteristics, and spatial structure constraints based on human vision, enabling the generation of adversarial examples that are visually highly similar to the original image and difficult for humans to detect, while maintaining strong attack capabilities. Its dynamic parameter adjustment strategy enhances the method's adaptability to different image content and target models, while energy functional constraints make the perturbation distribution more natural, avoiding abrupt noise patterns that may arise in traditional methods, thus achieving a better balance between attack effectiveness and visual concealment. Attached Figure Description

[0008] Figure 1This is a flowchart of a method for generating low-perceptibility adversarial examples based on human visual characteristics, according to an embodiment of the present invention. Detailed Implementation

[0009] To make the objectives, technical solutions, and advantages of the present invention clearer, a specific embodiment of the present invention will be described in detail below with reference to the accompanying drawings.

[0010] This embodiment provides an advanced adversarial example generation method with the core objective of enhancing visual concealment. For example... Figure 1 As shown, this method details how to meticulously construct adversarial perturbations that are difficult for human vision to detect through an iterative optimization process: Step S1: Receive the original digital image I and the target deep learning classifier model M(c|I) img ), where I img 'c' represents the input image, and 'c' represents the model output type. Then, the adversarial perturbation P to be optimized is initialized, set to a zero matrix. Simultaneously, a series of preset hyperparameters are defined. Step S2: Iteratively optimize and generate adversarial perturbations. The adversarial perturbation P is generated through iterative loops. This iterative process continues until the preset maximum number of iterations N is reached. iter Alternatively, the generated adversarial example I+P may successfully cause the target model M to misclassify. This iterative optimization process mainly includes the following sub-steps: Step S21, gradient estimation: In each iteration, first estimate the comprehensive objective function J. total The gradient g relative to the current adversarial perturbation P P The comprehensive objective function J total To balance the effectiveness of attacks with the imperceptibility of perturbations, its expression is: J total (I+P)=J classifier (M(c|I+P))+φ g E g (P)+φ n E n (P)+φ s E s (P)+φ l E l (P)+φ x E x (P) Among them, J classifier E represents the classifier loss term, using cross-entropy loss; g (P),E n (P),E s (P),E l (P),E x(P) represent perception-related constraints such as gradient regularization constraint, norm constraint, spatial sparsity constraint, brightness perception constraint, and color perception constraint; φ g ,φ n ,φ s ,φ l ,φ x These are the weighting coefficients corresponding to each constraint; For black-box attack scenarios, gradient g P The estimation is performed using the finite difference method. The estimation expression is: Where ζ is a tiny perturbation step size, and u is a random unit vector with the same dimension as the perturbation P; Step S22: Apply constraints based on visual brightness perception and color channel allocation, adjusting the perturbation according to the characteristics of human visual perception of brightness and color. Image brightness calculation and dynamic influence factor: Calculate the brightness value L of each pixel in the original image I. v The calculation method is based on the ITU-R BT.709 standard for high-definition television (HDTV) color space and image parameters, established by the International Telecommunication Union Radiocommunication Sector (ITU-R). The pre-processed red, green, and blue color channel intensity values ​​C... r' C g' C b' Through the standard conversion factor k r ,k g ,k b Convert to brightness. The conversion relationship is L. v =k r ·C r′ +k g ·C g′ +k b ·C b′ Where the standard conversion factor k r ,k g ,k b The values ​​are 0.2126, 0.7152, and 0.0722; To simulate the difference in sensitivity of the human eye to perturbations in different brightness areas, a value based on the original pixel brightness L is defined. v The dynamic influence factor function F(L) v This function uses an exponential decay form. Where ω and κ are preset parameters. This function allows the perturbation to have different weights in different brightness regions; Brightness variation control aims to limit the changes in image brightness caused by disturbances P, making them less noticeable to the human eye. This is reflected in the brightness perception constraint term E. l In (P), the implementation method is to directly constrain the brightness component L of the adversarial sample.v,adv Compared with the original image brightness L v The difference between them is less than the preset threshold. L That is, it requires |L v,adv (I+P)-L v (I)|≤ò L E l (P) is reflected in the overall objective function as a penalty for exceeding the tolerance for brightness differences; Color channel perturbation allocation takes into account the varying sensitivities of the human eye to different color perturbations, with a higher tolerance for the blue channel. The perturbation P has components ΔX in the red, green, and blue color channels. r ,ΔX g ,ΔX b The generation is adjusted based on the following expression: ΔX r =w r ·p r ΔX g =w g ·p g ΔX b =(w b +ρ b )·p b , Where p r ,p g ,p b w represents the underlying disturbance potential of each channel r ,w g ,w b It is the preset basic allocation ratio, ρ b It is an additional perturbation adjustment factor for a specific color channel that can be dynamically adjusted; These channel perturbations also need to satisfy the overall color perturbation budget constraint, which is reflected in the color perception constraint term E. x (P) In this constraint, the norm of the perturbation for all color channels must not exceed a preset upper limit Θ, either pixel-by-pixel or globally. x E x (P) Color distortion is calculated based on the human eye color difference model CIEDE2000; Step S23 applies structural constraints based on energy functional theory, designed to make the spatial distribution of the perturbation P more natural and smoother. Gradient regularization penalizes high-frequency variations within the perturbation P to encourage spatial smoothness. g (P) is defined as the square of the L2 norm of the spatial gradient of the perturbation P. The spatial gradient is approximated by calculating the partial derivative in the horizontal direction through the difference between adjacent pixels. partial derivatives in the vertical direction E g The expression for (P) is: The summation iterates through all pixel positions (i,j); The L2 norm constraint directly controls the overall energy of the perturbation P to prevent the perturbation from becoming too significant. n (P) is represented in the objective function as the L2 norm ||P||2 of the perturbation P. The definition of the L2 norm is: Where Δp i,j,c Let represent the specific value of the perturbation in channel c of pixel (i,j). This L2 norm must satisfy ||P||² ≤ ∞. N , among which N The upper limit of the preset L2 norm; Spatial sparsity constraint encourages the energy of perturbation P to be concentrated in a few key regions. s (P) is achieved in the objective function by penalizing the L0 norm ‖P‖0 of the perturbation P, i.e., the number of non-zero elements. The L0 norm must satisfy ‖P‖0 ≤ K. s K s The sparsity is achieved by iterative hard thresholding, which is set to the maximum number of non-zero perturbation elements. Step S24, Dynamic parameter adjustment, constraint weight parameters φ i and additional perturbation adjustment factor ρ for specific color channels b Key parameters, such as attack effectiveness and perceived quality, are adaptively adjusted during the iteration process based on feedback. The expression for adjusting the weights is: Where η d This represents adjusting the step size. This represents the current value of a metric related to a certain constraint. This represents the target value for the indicator. Similarly, ρ b Update and average brightness change and its target value Ξ L Related, update rule is Step S25, update the adversarial perturbation based on the gradient g estimated in step S21. P The current adversarial perturbation P is updated using the stochastic gradient descent (SGD) optimization algorithm with momentum. The update rule is P←P-λ. opt ·g P , where λ opt The learning rate; After the update, the amplitude of the disturbance P is clipped to ensure that its element values ​​are within the preset range [-P]. max ,P maxThe clipping operation is represented as P←Clip(P, [-P]). max ,P max ]); Then, the updated and cropped perturbation P is superimposed on the original image I to generate candidate adversarial examples I. adv ←I+P; Finally, for candidate adversarial sample I adv Perform pixel value cropping to ensure that the pixel values ​​are within the effective image display area [V] min V max [Inside]. The clipping operation is represented as I. adv ←Clip(I adv ,[V min V max ]); After generating candidate adversarial examples, check whether they successfully deceive the target model and determine whether the iteration termination condition is met. Step S3: Output adversarial sample. When the iteration terminates, whether the attack is successful or the maximum number of iterations is reached, output the best adversarial sample found during the iteration process or the last generated adversarial sample I. adv The best adversarial sample is defined as the sample that achieves a successful attack while minimizing visual cost.

[0011] This invention, through the aforementioned steps, particularly by combining brightness perception constraints, color channel optimization, energy functional (gradient regularization), sparsity, L2 norm, and dynamic parameter adjustment, can generate adversarial examples that are difficult for humans to perceive but can effectively deceive deep learning models. Experiments show that this method achieves an attack success rate of over 91.26% on datasets such as CIFAR-10 and ILSVRC2012 against mainstream networks such as ResNet, VGG, and MobileNet, in both white-box and black-box scenarios. Furthermore, it demonstrates excellent performance in perceptual metrics such as structural similarity index (SSIM), perturbation rate, and CIELCh color difference.

[0012] The above embodiments are merely one specific implementation of the present invention and are not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that, without departing from the core ideas and spirit of the present invention, various equivalent substitutions, modifications, or improvements can be made to the specific formulas, parameter settings, projection methods, optimizer selection, etc., and all such changes should be included within the scope of protection of the present invention. For example, the gradient estimation method can be flexibly selected according to white-box or black-box scenarios; the components of the energy functional can be added, removed, or have their weights adjusted according to requirements; and there can be multiple implementation methods for the specific strategies of dynamic parameter adjustment.

Claims

1. A method for generating low-perceptibility adversarial examples based on human visual characteristics, characterized in that, Includes the following steps: Step S1: Receive the original digital image and the target deep learning classifier model, and initialize the adversarial perturbation to be applied to the original digital image. Step S2: Generate the adversarial perturbation through an iterative optimization process; Step S3: The adversarial perturbation finally generated by the iterative optimization process is superimposed and fused with the original digital image to obtain the low-perceptibility adversarial sample.

2. The method according to claim 1, characterized in that, Step S2 includes the following steps: Step S21: Based on the target deep learning classifier model and the adversarial perturbation of the current iteration cycle, estimate the gradient of an overall objective function containing classification loss and perceptual constraints relative to the adversarial perturbation using a numerical approximation method or by utilizing the gradient information inside the model. Step S22: Apply one or more constraints based on visual brightness perception to the adversarial perturbation. The constraints utilize the differences in human eye response to different spectral components to adjust the allocation strategy of the perturbation in different color channels and introduce a dynamic brightness adjustment mechanism to suppress significant and easily noticeable abrupt changes in the overall and local brightness of the adversarial sample image. Step S23: Apply one or more structural constraints based on energy functional theory to the adversarial perturbation. The structural constraints comprehensively consider the gradient regularization, spatial sparsity and norm restriction of the perturbation, aiming to improve the smoothness and local concentration of the perturbation distribution and control its overall energy. Step S24: Implement a dynamic parameter adjustment strategy. During the iteration process, based on the evaluation feedback of the attack effect and perception quality, adaptively adjust the intensity of the adversarial perturbation and the weight coefficients of various constraints to maximize visual concealment while ensuring the effectiveness of the attack. Step S25: Update the adversarial perturbation based on the parameters adjusted in step S24 and the gradient estimated in step S21.

3. The method according to claim 2, characterized in that, In step S21, when the gradient is estimated using a numerical approximation method, the gradient is determined based on the difference in function values ​​of the objective function at the current perturbation point and at the perturbation point after a small offset along a predetermined direction, combined with the magnitude of the small offset.

4. The method according to claim 2, characterized in that, The dynamic influence factor used in the dynamic brightness adjustment mechanism in step S22 is an exponential decay function based on the original brightness value of the pixel; and when the brightness component of the adversarial sample is updated, its adjustment amount is jointly affected by the dynamic influence factor and the brightness-related gradient component.

5. The method according to claim 2 or 4, characterized in that, In step S22, the generation of perturbation components for the red, green, and blue primary color channels of the image is achieved by multiplying their respective basic perturbation components by their respective basic perturbation allocation ratios. Furthermore, for the basic perturbation component of one of the preset color channels, an additional dynamically adjusted perturbation adjustment factor is added before multiplying it by its basic perturbation allocation ratio. The sum of the absolute values ​​of the perturbation components of all color channels is constrained by a preset overall color perturbation budget.

6. The method according to claim 2, characterized in that, The gradient regularization term in step S23 is defined by summing the squares of the partial derivatives of the perturbation in each direction of the image space.

7. The method according to claim 2, characterized in that, The spatial sparsity constraint in step S23 requires that the number of non-zero elements in the perturbation does not exceed a preset maximum number.

8. The method according to claim 2, characterized in that, The norm constraint in step S23 is to require that the L2 norm of the perturbation does not exceed a preset threshold.

9. The method according to claim 2, characterized in that, The dynamic parameter adjustment strategy in step S24 updates the weight coefficients of the constraints in a manner proportional to the change in the metric value related to the constraint, and updates the additional perturbation adjustment factor for a specific color channel in a manner proportional to the difference between the average brightness change caused by the current iteration and the preset ideal brightness change range.

10. A low-perceptibility adversarial example generation system optimized based on human visual characteristics, characterized in that, The system is configured to perform the method as described in any one of claims 1 to 9; wherein, when performing the method of claim 2, the processing module for applying structural constraints based on energy functional theory is configured to improve the smoothness and local concentration of the perturbation distribution and control its overall energy by combining a gradient regularization term, defined as the sum of the squares of the partial derivatives of the perturbation in each direction of the image space, a spatial sparsity constraint requiring that the number of non-zero elements in the perturbation does not exceed a preset maximum number, and an L2 norm limit requiring that the L2 norm of the perturbation does not exceed a preset threshold.

Images