Disturbance image generation method and device, equipment and storage medium

Abstract

The invention discloses a disturbance image generation method and device, equipment and a storage medium, and the method comprises the steps: generating disturbance sub-images for different disturbance characteristics through employing a multi-dimensional disturbance sub-network; calculating the importance weight of each disturbance sub-image in the spatial dimension, and performing weighted fusion on each disturbance sub-image according to the importance weight to generate an initial fusion image; receiving environment data fed back by the black box, and adjusting the fusion weight of each disturbance sub-image according to the environment data; and performing feature re-calibration on the initial fusion image by using the fusion weight of the disturbance sub-image to generate a disturbance fusion image adapted to the environmental characteristics. According to the method and the device, the generated multi-dimensional disturbance sub-images are subjected to a strategy of combining static fusion and dynamic fusion to obtain the disturbance fusion image adaptive to the environment, so that the technical problems of poor generalization and weak adaptability of the existing attack resisting technology are solved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence security technology, and in particular to a method, apparatus, device, and storage medium for generating a disturbance image. Background Art

[0002] With the widespread application of artificial intelligence (AI) image recognition algorithms in fields such as security surveillance and autonomous driving, potential security risks are becoming increasingly prominent. Adversarial perturbation patterns, as a defensive or jamming method, aim to disrupt the recognition capabilities of malicious algorithms through covert pattern perturbations. This is crucial for protecting privacy-sensitive information and defending against automated attacks.

[0003] However, existing adversarial attack technologies face significant limitations in practical applications. First, they rely on white-box information about the target algorithm, resulting in insufficient generalization of the generated perturbation patterns, making them difficult to migrate to unknown algorithms or updated models. Second, existing technologies mostly use static perturbation patterns that cannot dynamically adapt to algorithm iterative optimization or environmental changes, and the attack effect can easily decay rapidly due to upgrades to the target system. In addition, a single perturbation dimension makes it difficult to cover the feature extraction preferences of different architectural models. These limitations make it difficult for existing adversarial attack technologies to meet actual needs when faced with unknown algorithms, dynamic defense mechanisms, and complex scenarios. Summary of the Invention

[0004] The main purpose of this application is to provide a method, device, equipment and storage medium for generating a disturbed image, aiming to solve the technical problems of poor generalization and weak adaptability of existing anti-attack technologies.

[0005] To achieve the above-mentioned object, the present application provides a method for generating a disturbed image, which comprises the following steps:

[0006] Using a multi-dimensional perturbation sub-network, perturbation sub-images targeting different perturbation features are generated;

[0007] Calculating the importance weight of each perturbed sub-image in the spatial dimension, and performing weighted fusion on each perturbed sub-image according to the importance weight to generate an initial fused image;

[0008] Receive environmental data fed back by the black box, and adjust the fusion weights of each perturbed sub-image according to the environmental data;

[0009] The initial fused image is feature recalibrated using the fusion weights of the perturbed sub-images to generate a perturbed fused image adapted to environmental characteristics.

[0010] In one embodiment, after the step of recalibrating the features of the initial fused image using the fusion weights of the perturbed sub-images to generate a perturbed fused image adapted to environmental characteristics, the following steps are included:

[0011] Using a pre-built proxy model library, attack simulation is performed on the perturbed fusion image to generate an evaluation matrix containing the attack success rate of each model;

[0012] When the attack success rate is lower than a preset threshold, the adversarial gradient of the proxy model is extracted through an online adaptation mechanism, and the parameter weights of the multi-dimensional perturbation sub-network are reversely optimized;

[0013] Using the evaluation matrix to construct a feature vector representing the state of the black box environment, parsing the feature vector through a proximal strategy optimization algorithm to generate a multi-disturbance combination optimization strategy;

[0014] Based on the multi-perturbation combination optimization strategy and the optimized parameter weights, the fusion ratio of each perturbation sub-image is adjusted to generate an adaptive perturbation fusion image.

[0015] In one embodiment, the step of generating perturbed sub-images for different perturbation features using a multi-dimensional perturbation sub-network includes:

[0016] Using a texture perturbation subnetwork, a texture perturbation image is generated by adding adversarial high-frequency noise to a local area of ​​the image;

[0017] Using a geometric perturbation subnetwork, slightly deforming key areas of the image to generate a geometrically perturbed image;

[0018] The frequency domain perturbation subnetwork is used to generate a frequency domain perturbation image by injecting an interference signal of a specific frequency band into the frequency domain of the image.

[0019] In one embodiment, the step of generating perturbed sub-images for different perturbation features using a multi-dimensional perturbation sub-network further includes:

[0020] A multi-dimensional perturbation generator is constructed based on a lightweight network to generate perturbed sub-images with different perturbation features in parallel.

[0021] In one embodiment, the step of calculating the importance weights of the perturbed sub-images in the spatial dimension and performing weighted fusion on the perturbed sub-images according to the importance weights to generate the initial fused image includes:

[0022] Based on the spatial attention mechanism, the spatial features of each perturbed sub-image within the preset grid unit are weighted and the attention weight matrix of each perturbed sub-image is generated;

[0023] The attention weight matrix of each perturbed sub-image is weightedly summed with the corresponding spatial features to generate the initial fused image.

[0024] In one embodiment, the step of receiving environmental data fed back by the black box system and adjusting the fusion weights of the perturbed sub-images according to the environmental data includes:

[0025] The heterogeneous environment data fed back by the black box is received, the heterogeneous environment data is input into the fine-tuning network, the environment-sensitive features are extracted through three fully connected layers, and the fusion weight corresponding to each perturbed sub-image is output.

[0026] In one embodiment, the step of receiving environmental data fed back by the black box system and adjusting the fusion weights of the perturbed sub-images according to the environmental data further includes:

[0027] The algorithm response characteristics and target device parameters fed back by the black box are received, and an update rate of the fusion weight is adjusted according to the algorithm response characteristics and target device parameters.

[0028] In addition, to achieve the above-mentioned purpose, the present application further provides a device for generating a disturbed image, the device for generating a disturbed image comprising:

[0029] Multi-perturbation generation module, used to generate perturbation sub-images targeting different perturbation features using a multi-dimensional perturbation sub-network;

[0030] a first fusion module, configured to calculate the importance weight of each perturbed sub-image in a spatial dimension, and perform weighted fusion on each perturbed sub-image according to the importance weight to generate an initial fused image;

[0031] An adjustment module, configured to receive environmental data fed back by the black box and adjust the fusion weights of each perturbed sub-image according to the environmental data;

[0032] The second fusion module is used to recalibrate the features of the initial fused image using the fusion weights of the perturbed sub-images to generate a perturbed fused image adapted to the environmental characteristics.

[0033] In addition, to achieve the above-mentioned purpose, the present application also provides a terminal device, which includes a memory, a processor, and a disturbed image generation program stored in the memory and runnable on the processor, wherein the disturbed image generation program, when executed by the processor, implements the steps of the disturbed image generation method described above.

[0034] In addition, to achieve the above-mentioned purpose, the present application also provides a computer-readable storage medium, on which a disturbed image generation program is stored. When the disturbed image generation program is executed by a processor, the steps of the disturbed image generation method described above are implemented.

[0035] One or more technical solutions proposed in this application have at least the following technical effects:

[0036] First, the present application generates perturbation sub-images covering a variety of attack features through a multi-dimensional perturbation sub-network. The generated perturbations are compatible with different model architectures and data distributions, thereby enhancing cross-model generalization capabilities. Secondly, the importance weights of each perturbation sub-image in the spatial dimension are calculated, and each perturbation sub-image is weightedly fused according to the importance weights to generate an initial fused image. By quantitatively evaluating the contribution of each sub-image in the spatial dimension, the perturbation intensity of the key area is enhanced, while suppressing the interference noise in the non-key area, so that the fused initial image has a stronger attack directionality. Furthermore, the environmental data fed back by the black box is received, and the fusion weights of each perturbation sub-image are adjusted according to the environmental data, so that the generated perturbation fusion image can adapt to the spatial feature distribution and defense mechanism of the target environment, thereby improving the robustness of the adversarial sample in unknown defense scenarios. Finally, the initial fusion image is feature recalibrated to generate a perturbation fusion image adapted to the environmental characteristics, solving the technical problems of poor generalization and weak adaptability of existing adversarial attack technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is a flowchart of a first exemplary embodiment of a method for generating a disturbed image according to the present application;

[0038] Figure 2 This is a flowchart of a second exemplary embodiment of a method for generating a disturbed image of the present application;

[0039] Figure 3 This is a flowchart of a third exemplary embodiment of a method for generating a disturbed image of the present application;

[0040] Figure 4 This is a schematic diagram of the module structure of the device for generating a disturbed image according to an embodiment of the present application;

[0041] Figure 5 Schematic diagram of the device structure of the hardware operating environment involved in the method for generating a disturbed image in an embodiment of the present application.

[0042] The realization of the objectives, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0043] It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0044] The main technical solution of this application is: using a multi-dimensional perturbation sub-network to generate perturbation sub-images for different perturbation features; calculating the importance weight of each perturbation sub-image in the spatial dimension, and performing weighted fusion of each perturbation sub-image according to the importance weight to generate an initial fused image; receiving environmental data fed back by a black box, and adjusting the fusion weight of each perturbation sub-image according to the environmental data; using the fusion weight of the perturbation sub-image to recalibrate the features of the initial fused image to generate a perturbation fused image adapted to the environmental characteristics.

[0045] This application addresses two major challenges facing the current field of AI security. On the one hand, with the rapid iteration of algorithms like ResNet, YOLO, and Vision Transformer, traditional adversarial sample generation techniques frequently fail due to their over-reliance on specific model gradient information. On the other hand, scenarios like security monitoring and autonomous driving place higher demands on real-time black-box attacks, while existing technologies are limited by static perturbation patterns and a single attack dimension, making it difficult to penetrate the defense mechanisms of unknown algorithms. For example, adversarial samples generated by methods like FGSM and PGD generally have low attack success rates in black-box scenarios, and physical attacks based on adversarial patches are easily filtered by image preprocessing modules.

[0046] Based on this, the present application proposes a method, apparatus, device and storage medium for generating a disturbance image.

[0047] Specifically, the following are the detailed steps of the first exemplary embodiment of the method for generating a disturbed image of the present application:

[0048] Reference Figure 1 , Figure 1 This is a flowchart of a first exemplary embodiment of a method for generating a disturbed image of the present application. In this embodiment, the method for generating a disturbed image includes steps S10 to S40:

[0049] Step S10, using a multi-dimensional perturbation sub-network to generate perturbation sub-images for different perturbation features;

[0050] It should be noted that the multidimensional perturbation subnetwork is a deep learning-based neural network architecture specifically designed to apply multi-dimensional perturbations to the input image to generate sub-images with different perturbation characteristics. The multidimensional perturbation subnetwork contains multiple branches, each responsible for introducing a specific type of perturbation. In this way, the multidimensional perturbation subnetwork can generate a series of sub-images reflecting different perturbation characteristics.

[0051] Specifically, the image is input into a multidimensional perturbation subnetwork. This subnetwork contains multiple independent perturbation modules, each of which applies perturbations tailored to specific image features. For example, one module might increase the image's brightness, while another might adjust its contrast. These perturbation modules control the degree of perturbation using preset parameters. After perturbation processing by the different perturbation modules, the multidimensional perturbation subnetwork outputs multidimensional perturbed subimages, each of which reflects the changes in the input image under specific perturbation features.

[0052] Step S20, calculating the importance weight of each perturbed sub-image in the spatial dimension, and performing weighted fusion on each perturbed sub-image according to the importance weight to generate an initial fused image;

[0053] It's important to note that spatial importance weights refer to the contribution of pixels at different locations in the two-dimensional image space to the final image features. By calculating the importance weight of each pixel in each perturbed sub-image, we can determine which areas are more critical in the image fusion process. Weighted fusion is an image processing technique that assigns a weight to each perturbed sub-image and then performs a weighted sum of these images to generate a new fused image. This method preserves important features in each perturbed sub-image while suppressing irrelevant information.

[0054] Each perturbed sub-image is subjected to a spatial dimension analysis. Specifically, a convolutional neural network or other image analysis algorithm can be used to assess the importance of each pixel in the image. For example, an edge detection algorithm can be used to identify important edge regions in the image and assign higher weights to pixels in these regions, while pixels in the background region are assigned lower weights. After calculating the importance weight of each pixel in each perturbed sub-image, the perturbed sub-images are weightedly fused based on the importance weight of each pixel in each perturbed sub-image.

[0055] Specifically, the pixel value of each perturbed sub-image is multiplied by its corresponding importance weight, and then the weighted pixel values ​​of all perturbed sub-images are summed to generate the initial fused image.

[0056] Step S30, receiving environmental data fed back by the black box, and adjusting the fusion weights of each disturbed sub-image according to the environmental data;

[0057] It should be noted that the environmental data fed back by the black box refers to information about the current environmental state obtained from the external system. This data can include light intensity, weather conditions, scene type, etc. The fusion weight refers to the proportion of each perturbed sub-image in the image fusion process.

[0058] Specifically, the black box receives external environmental data feedback, which can be transmitted in real time through the sensor network. Based on this environmental data, the fusion weight of each perturbed sub-image is dynamically adjusted. For example, if the camera fill light intensity of the target device is high, the weight of the color cast perturbation may need to be increased; if the target algorithm is sensitive to texture perturbation, the weight of the texture perturbation sub-image needs to be increased. The adjustment process can be implemented through a fine-tuning network that automatically adjusts the weight matrix according to the characteristics of the environmental data. The adjusted weights will be used in the subsequent feature recalibration step to generate a perturbation fusion image that adapts to the environmental characteristics.

[0059] Step S40 : recalibrating the features of the initial fused image using the fusion weights of the perturbed sub-images to generate a perturbed fused image adapted to the environmental characteristics.

[0060] It should be noted that the feature recalibration of the initial fused image is performed using the fusion weights of the adjusted perturbed sub-images. The purpose is to generate a perturbed fused image that is adapted to the characteristics of the target environment, thereby improving the success rate of the adversarial attack.

[0061] Specifically, fusion weights are first applied to each pixel in the initial fused image. This step redistributes the different perturbation features in the initial fused image in a weighted manner, enhancing those perturbations that are more effective in the current environment while suppressing those that may negatively impact the attack's effectiveness. For example, if environmental data indicates that the target system is more sensitive to texture perturbations, the weight of texture perturbations will be appropriately increased at this stage, thereby highlighting the perturbation's features in the fused image.

[0062] It is feasible to normalize the weighted image to ensure that the pixel values ​​of the generated perturbed fusion image remain within the legal range while avoiding feature imbalance caused by weight adjustment. Normalization not only helps maintain the visual consistency of the image, but also ensures that the generated image can be correctly processed by the target system, thus avoiding the failure of the attack due to image format or pixel value range issues.

[0063] Through this feature recalibration method, the generated perturbation fusion image can adapt to the characteristics of the target environment more accurately.

[0064] Further, refer to Figure 2 , Figure 2 This is a flow chart of a second exemplary embodiment of the method for generating a disturbed image of the present application. In this embodiment, step S40 is followed by steps S50 to S80:

[0065] Step S50, using a pre-built proxy model library to perform attack simulation on the perturbed fusion image, and generate an evaluation matrix including the attack success rate of each model;

[0066] It should be noted that we used a pre-built proxy model library to simulate attacks on the perturbed fusion image and generate an evaluation matrix containing the attack success rate of each model. The purpose of this simulated attack is to evaluate the effectiveness of the perturbed fusion image on different models, thereby providing data support for subsequent optimization.

[0067] Specifically, the proxy model library can include a variety of mainstream AI models, encompassing diverse architectures and algorithm types, such as CNNs, Transformers, and hybrid architectures. By inputting perturbed fusion images into these proxy models, various scenarios likely encountered in a real-world black-box environment can be simulated. Each proxy model then identifies or classifies the input perturbed fusion image and outputs its image processing results, which may include classification confidence, detection box location, and other information. Based on these outputs, the attack success rate for each proxy model under the current perturbed fusion image is calculated—that is, the probability that the model can be successfully deceived. The attack success rates are recorded to form an evaluation matrix, with each row representing a proxy model and each column representing statistical information about the attack success rate, such as the average success rate and standard deviation of the success rate.

[0068] Step S60: when the attack success rate is lower than a preset threshold, extracting the adversarial gradient of the proxy model through an online adaptation mechanism, and reversely optimizing the parameter weights of the multi-dimensional perturbation sub-network;

[0069] Specifically, the online adaptation mechanism monitors changes in the attack success rate in real time during attack simulations, triggering the optimization process when the success rate falls below a preset threshold. When the attack success rate falls below the preset threshold, adversarial gradient information for the corresponding model is extracted from the proxy model library. This gradient information reflects the features of the current perturbed image that are less effective against the model. This gradient information is then backpropagated to the multidimensional perturbation subnetwork, guiding it to adjust its parameter weights.

[0070] For example, if a proxy model is more resistant to texture perturbations, the weight of the texture perturbation sub-network will be specifically enhanced based on the adversarial gradient information, while the weights of the geometric and frequency domain perturbation sub-networks will be adjusted to balance the overall adversarial effect.

[0071] Step S70, constructing a feature vector representing the state of the black box environment using the evaluation matrix, parsing the feature vector using a proximal strategy optimization algorithm, and generating a multi-disturbance combination optimization strategy;

[0072] First, feature extraction is performed on the evaluation matrix. This can be done by statistically analyzing the attack success rates of each model in the matrix and calculating statistics such as the mean, variance, maximum, and minimum. These statistics reflect the overall performance and fluctuations of the perturbed fusion image on different models. These statistics are then combined into a feature vector, which characterizes the state of the black box environment under the current perturbed image. For example, one element in the feature vector might represent the mean of the attack success rate, another the variance, and so on.

[0073] The purpose of constructing a feature vector is to provide a concise and informative input. By compressing the high-dimensional evaluation matrix into a low-dimensional feature vector, we can reduce data complexity and improve computational efficiency while preserving key information. Each element in the feature vector carries important information about the state of the black-box environment, which is crucial for generating effective optimization strategies.

[0074] Next, the proximal policy optimization (PPO) algorithm is used to analyze the feature vector and generate a multi-perturbation combination optimization strategy. The PPO algorithm is a policy optimization algorithm used in reinforcement learning. It can effectively improve the performance of the strategy while maintaining the stability of the policy update. In this process, the PPO algorithm learns how to adjust the perturbation combination to optimize the attack effect based on the information in the feature vector and a pre-set reward function (such as the improvement of the attack success rate). Specifically, if the feature vector shows a large variance in the attack success rate, it means that the perturbation image is effective on some models but not on others. In this case, the PPO algorithm is used to generate a more balanced perturbation combination strategy, making the perturbation image effectively offensive on more models.

[0075] Step S80 : Based on the multi-perturbation combination optimization strategy and the optimized parameter weights, the fusion ratio of each perturbation sub-image is adjusted to generate an adaptive perturbation fusion image.

[0076] Specifically, the fusion strategy for each perturbed sub-image during the fusion process is first determined based on the guidance provided by the multi-perturbation combination optimization strategy. This fusion strategy reflects the contribution of different perturbation types to the attack success rate within the current black-box environment. For example, if the optimization strategy indicates that frequency-domain perturbations are more critical to improving the attack success rate, the weight of the frequency-domain perturbation sub-image will be increased accordingly, while the weights of the texture and geometry perturbation sub-images will be adjusted to maintain an overall balance.

[0077] Furthermore, the fusion ratio of each perturbed sub-image is adjusted based on the parameter weights obtained through reverse optimization. This adjustment reflects the internal optimization state of the multi-dimensional perturbation sub-network when generating the perturbed image, and can further guide the adjustment of the fusion ratio. This ensures that the generated perturbed fusion image achieves the optimal balance between attack success rate and concealment.

[0078] For example, in some scenarios, stronger texture perturbations are needed to disrupt CNN models, while in other scenarios, more emphasis is placed on frequency domain perturbations to counter the Transformer model. These changes can be flexibly addressed by dynamically adjusting the fusion ratio.

[0079] In a feasible implementation, step S10 may include steps A1 to A3:

[0080] Step A1, using a texture perturbation subnetwork, generating a texture perturbation image by adding adversarial high-frequency noise to a local area of ​​the image;

[0081] It is feasible to use the texture perturbation subnetwork to generate high-frequency noise in local areas such as edges and corners of the image through a generative adversarial network, thereby interfering with the edge detection and feature matching capabilities of the target algorithm.

[0082] Specifically, the texture perturbation subnetwork uses a generative adversarial network. The generator receives an input image and, through a series of convolutional layers and activation functions, generates a texture-perturbed image with the same size as the input image but containing high-frequency noise. The power spectral density of the high-frequency noise is constrained to within the range of 0.5-2.5 MHz to ensure visual concealment while interfering with the target algorithm.

[0083] It should be noted that the generator of the generative adversarial network adds high-frequency noise to the edge area of ​​the image because the edge area of ​​the image is usually the key part for the target algorithm to extract features, thereby interfering with the edge detection layer of algorithms such as CNN, thereby reducing its ability to accurately recognize the image content.

[0084] Step A2: using a geometric perturbation subnetwork to slightly deform key areas of the image to generate a geometrically perturbed image;

[0085] It is feasible to utilize the geometric perturbation subnetwork to generate a non-rigid deformation field based on the parameterized thin plate spline transformation to slightly deform the key areas of the image, thereby destroying the geometric feature extraction ability of the target algorithm.

[0086] Specifically, after receiving the input image through the geometric perturbation subnetwork, it first identifies key areas in the image, such as facial features and vehicle outlines. These areas are important bases for the target algorithm to perform feature matching and object detection. Then, a non-rigid deformation field is generated using the TPS transformation, which defines the displacement vector for each pixel in the image. By controlling the deformation amplitude within a preset pixel range, key areas can be slightly deformed while maintaining the subject's recognizability. Although this micro-deformation is visually imperceptible, it is sufficient to disrupt the key point matching process of the target detection algorithm.

[0087] In step A3, a frequency domain perturbation subnetwork is used to inject an interference signal of a specific frequency band into the frequency domain of the image to generate a frequency domain perturbation image.

[0088] It is feasible to utilize the frequency domain perturbation subnetwork to inject interference signals of a specific frequency band into the frequency domain of the image through Fourier transform to generate a frequency domain perturbation image, aiming to interfere with the frequency domain feature reconstruction process of the target algorithm.

[0089] Specifically, after receiving the input image through the frequency domain perturbation subnetwork, the image is converted from the spatial domain to the frequency domain. In the frequency domain, a dual-channel Fourier perturbator is used to perturb the mid- and high-frequency bands of the image. Specifically, the dual-channel Fourier perturbator implements phase perturbation and amplitude modulation in the mid- and high-frequency bands of the image. For example, setting the phase offset to π / 6 to π / 3 can effectively interfere with algorithms based on the global attention mechanism, such as the Transformer, because these algorithms are more sensitive to global features in the frequency domain.

[0090] It should be noted that by injecting interference signals into the frequency domain, the frequency domain perturbation subnetwork can disrupt the target algorithm's reconstruction of the image's frequency domain features, thereby reducing its ability to accurately identify image content. The generated frequency domain perturbation image will be converted back to the spatial domain and used as one of the inputs of the subsequent fusion module. It will be combined with other perturbation images to form the final perturbation fusion image.

[0091] In another feasible implementation, step S10 may further include step B1:

[0092] In step B1, a multi-dimensional perturbation generator is constructed based on a lightweight network to generate perturbed sub-images with different perturbation features in parallel.

[0093] Specifically, a multidimensional perturbation generator is first constructed based on a lightweight network architecture. Lightweight networks are characterized by low computational complexity and a small number of parameters, enabling rapid generation of perturbed images with limited computing resources. The multidimensional perturbation generator can contain multiple parallel subnetworks, each responsible for generating a specific type of perturbation feature, such as texture perturbation, geometric perturbation, and frequency domain perturbation. These subnetworks share the input image but operate independently, generating corresponding perturbation subimages in parallel.

[0094] By generating different types of perturbed sub-images in parallel, the multidimensional perturbation generator is able to provide rich perturbation features for subsequent fusion steps in a short period of time. This parallel processing approach not only improves generation efficiency but also ensures the independence of different perturbation features, avoiding the limitations that may arise from a single perturbation pattern. Ultimately, these parallel perturbation sub-images are passed to the subsequent fusion module to generate a fused image that integrates multiple perturbation features, thereby improving the success rate and adaptability of adversarial attacks.

[0095] In one possible embodiment, referring to Figure 3 , Figure 3 This is a flowchart of a third exemplary embodiment of a method for generating a disturbed image of the present application. In this embodiment, step S20 may include steps S21 to S22:

[0096] Step S21: Based on the spatial attention mechanism, weights are assigned to the spatial features of each perturbed sub-image within a preset grid unit to generate an attention weight matrix for each perturbed sub-image;

[0097] First, each perturbed sub-image of the input is divided into grid cells of preset specifications. Assume that the space is divided according to the preset 128×128 grid cells, and each grid cell corresponds to a local area in the image. For each perturbed sub-image, its spatial features within each grid cell are extracted through the spatial attention mechanism. The calculation of spatial features is implemented using a convolutional neural network. Specifically, the pixels within each grid cell are feature encoded and local features such as edge response strength, texture complexity, and deformation gradient are extracted. Subsequently, for the feature vector of each grid cell, the corresponding attention weight value is calculated through the fully connected layer.

[0098] To ensure the rationality of weight distribution, the Sigmoid function can be used to normalize the original scores so that their value range is limited to between 0 and 1. Finally, the weight values ​​of each perturbed sub-image on all 128×128 grid cells constitute its corresponding attention weight matrix.

[0099] This technical solution weights the spatial features of each perturbed sub-image within a preset grid cell, enhancing the perturbation effect in key areas of the fused image while suppressing unimportant areas, thereby improving the overall adversarial performance of the fused image. Through a spatial attention mechanism, it adaptively identifies and strengthens areas that have a stronger perturbation effect on the target algorithm, resulting in a more successful attack on the resulting fused image.

[0100] For example, in facial image recognition, suppose the geometric perturbation sub-image generates a stronger deformation gradient in the eye region, while the frequency-domain perturbation sub-image generates weaker frequency-domain energy. In this case, the spatial attention mechanism assigns a higher weight to the geometric perturbation in this region, while the frequency-domain perturbation is given a lower weight. This dynamic allocation method effectively highlights the perturbation type that has the most significant impact on the target algorithm.

[0101] Step S22: performing weighted summation on the attention weight matrix of each perturbed sub-image and the corresponding spatial features to generate the initial fused image.

[0102] Specifically, the pixel data of each perturbed sub-image is element-wise multiplied by its corresponding attention weight matrix to achieve pixel-level weighting. The weighted perturbed sub-image not only retains the original perturbation pattern but also strengthens the adversarial strength of key areas through the weight matrix.

[0103] For example, regions with larger deformations in the geometrically perturbed image are assigned higher weights, while regions with significant phase shifts in the frequency-domain perturbed image are also weighted accordingly. Subsequently, the weighted perturbed sub-images are pixel-wise superimposed to generate the initial fused image. This superposition process uses a linear weighted summation method to ensure balanced strength across the perturbation components.

[0104] To further optimize the fusion effect, the superimposed result can be subjected to feature calibration through a lightweight convolutional layer to eliminate mutual interference between different perturbations. For example, the convolutional layer can suppress the conflict between high-frequency noise and frequency-domain perturbations in overlapping frequency bands, or enhance the spatial complementarity between geometric deformation and texture perturbations. The resulting initial fused image not only contains the comprehensive adversarial information of the multi-dimensional perturbations, but also achieves precise focus on the perturbation effects through a spatial attention mechanism, providing high-quality basic input for subsequent dynamic weight adjustments.

[0105] In a feasible implementation, step S30 may include step C1:

[0106] Step C1: Receive heterogeneous environment data fed back by the black box, input the heterogeneous environment data into the fine-tuning network, extract environment-sensitive features through three fully connected layers, and output the fusion weight corresponding to each perturbed sub-image.

[0107] It should be noted that heterogeneous environment data can include the hardware parameters of the target device and the response characteristics of the algorithm, such as the camera's focal length, fill light intensity, sensor type, detection frame jitter frequency, changes in classification confidence, etc. Heterogeneous environment data reflects the diversity and complexity of the target environment and is an important basis for dynamically adjusting the fusion weight. For example, if the target device's camera fill light intensity is high, it may have stronger resistance to certain types of disturbances; and if the detection frame jitter frequency is high, it may indicate that the target algorithm is more sensitive to geometric deformation.

[0108] Specifically, the heterogeneous environment data fed back by the black box is received and input into the fine-tuning network. The fine-tuning network is a lightweight neural network whose main task is to extract sensitive features related to the environment from complex heterogeneous data. The fine-tuning network consists of three fully connected layers, each of which enhances the expressiveness of features through nonlinear activation functions. The first fully connected layer performs preliminary processing on the input heterogeneous data and extracts low-level features related to the environment; the second fully connected layer further integrates these low-level features to generate higher-level environment-sensitive features; the third fully connected layer maps these high-level features to specific fusion weights. For example, if the input data indicates that the camera focal length of the target device is short and the fill light intensity is high, the fine-tuning network may output a weight vector indicating the weight of enhancing the frequency domain perturbation while appropriately reducing the weight of the geometric perturbation.

[0109] Through layer-by-layer processing across three fully connected layers, the fine-tuned network effectively extracts the most valuable information from heterogeneous environment data for adjusting fusion weights. Ultimately, the fine-tuned network outputs the fusion weights corresponding to each perturbed sub-image. These fusion weights reflect the potential contribution of different perturbed sub-images to the attack success rate in the current environment.

[0110] For example, if the target algorithm is more sensitive to texture features, it may output a higher texture perturbation weight; if the target algorithm focuses more on frequency domain features, it may output a higher frequency domain perturbation weight. In this way, the fusion weights can be dynamically adjusted according to the characteristics of the target environment, thereby generating more targeted perturbation fusion images, improving its attack success rate and adaptability in practical applications.

[0111] In a feasible implementation, step S30 may include step D1:

[0112] Step D1: receiving the algorithm response characteristics and target device parameters fed back by the black box, and adjusting the update rate of the fusion weight according to the algorithm response characteristics and target device parameters.

[0113] Specifically, the algorithm response features collected from black box feedback can include the jitter frequency of the detection box, changes in classification confidence, misclassification rate, etc. This reflects the operating status of the target algorithm under the current perturbed image and is an important basis for evaluating the effectiveness of the attack.

[0114] At the same time, parameter information of the target device is collected, which can include hardware characteristics such as the camera's focal length, fill light intensity, and sensor type. This reflects the physical environment and hardware configuration of the target system and has a direct impact on the generation and effect of the perturbed image.

[0115] Based on the collected algorithm response characteristics and target device parameters, the update rate of the fusion weight is adjusted. The core is to flexibly control the update speed of the fusion weight according to the dynamic changes of feedback information.

[0116] For example, if the algorithm response characteristics show that the current perturbation image has a good attack effect on the target algorithm, the update rate of the fusion weights can be appropriately slowed down to maintain the current effective perturbation characteristics. Conversely, if the attack effect is poor, the update rate of the fusion weights needs to be accelerated to quickly adjust the characteristics of the perturbation image and find a more effective attack strategy. In addition, if the target device has a high fill light intensity, the update rate of the color deviation perturbation weights can be accelerated to better adapt to the attack requirements under high light conditions; and if the camera focal length is short, the update rate of the geometric perturbation weights can be appropriately slowed down to ensure the effectiveness of the geometric perturbation in close-up shooting scenarios.

[0117] In this way, the update rate of the fusion weights can be dynamically adjusted based on real-time feedback from the algorithm's response characteristics and target device parameters. This dynamic adjustment mechanism not only improves the adaptability of the perturbed fusion image to changes in the target environment, but also ensures that the attack strategy remains efficient and flexible under different hardware and algorithmic conditions. Ultimately, by dynamically adjusting the update rate of the fusion weights, a more targeted and adaptable perturbed fusion image is generated, enabling more effective countermeasures against attacks in complex black-box environments.

[0118] In addition, this application also proposes a device for generating a disturbed image, such as Figure 4 As shown, the device for generating the disturbed image includes:

[0119] A multi-perturbation generation module 10 is used to generate perturbation sub-images for different perturbation features using a multi-dimensional perturbation sub-network;

[0120] A first fusion module 20 is configured to calculate the importance weights of the perturbed sub-images in the spatial dimension, and perform weighted fusion on the perturbed sub-images according to the importance weights to generate an initial fused image;

[0121] An adjustment module 30 is configured to receive environmental data fed back by the black box and adjust the fusion weights of the perturbed sub-images according to the environmental data;

[0122] The second fusion module 40 is configured to perform feature recalibration on the initial fused image using the fusion weights of the perturbed sub-images to generate a perturbed fused image adapted to environmental characteristics.

[0123] The perturbed image generation device provided in this application utilizes the perturbed image generation method described in the aforementioned embodiments, aiming to address the technical issues of poor generalization and adaptability of existing countermeasure attack technologies. Compared to the prior art, the perturbed image generation device provided in this application achieves the same beneficial effects as the perturbed image generation method described in the aforementioned embodiments. Other technical features of the perturbed image generation device are the same as those disclosed in the aforementioned embodiments and are not further elaborated here.

[0124] The present application provides a device for generating a disturbed image, the device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method for generating a disturbed image in the first embodiment described above.

[0125] The device for generating the disturbance image in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (e.g., vehicle-mounted navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The device for generating a disturbance image shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0126] like Figure 5As shown, the device for generating a disturbed image may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes based on a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the device for generating a disturbed image. The processing device 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: an input device 1007 including, for example, a touch screen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the disturbed image generation device to communicate wirelessly or wired with other devices to exchange data. Although the figure shows a disturbed image generation device with various systems, it should be understood that it is not required to implement or have all of the systems shown. More or fewer systems may be implemented or have alternatively.

[0127] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0128] The perturbed image generation device provided in this application utilizes the perturbed image generation method described in the aforementioned embodiment, aiming to address the technical issues of poor generalization and adaptability of existing countermeasure attack technologies. Compared to the prior art, the perturbed image generation device provided in this application achieves the same beneficial effects as the perturbed image generation method described in the aforementioned embodiment. Other technical features of this perturbed image generation device are the same as those disclosed in the aforementioned embodiment and are not further elaborated here.

[0129] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0130] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0131] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, a computer program) stored thereon, wherein the computer-readable program instructions are used to execute the method for generating a disturbed image in the above-mentioned embodiment.

[0132] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0133] The computer-readable storage medium may be included in the device for generating the disturbed image, or may exist independently without being incorporated into the device for generating the disturbed image.

[0134] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0135] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0136] The modules described in the embodiments of the present application may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0137] The computer-readable storage medium provided in this application stores computer-readable program instructions (i.e., a computer program) for executing the aforementioned method for generating a perturbed image. This medium aims to address the technical issues of existing countermeasures against attack techniques, which suffer from poor generalization and adaptability. Compared to the prior art, the beneficial effects of the computer-readable storage medium provided in this application are similar to those of the perturbed image generation method provided in the aforementioned embodiments, and are not further elaborated here.

[0138] The present application also provides a computer program product, comprising a computer program, which implements the steps of the above-mentioned method for generating a disturbed image when executed by a processor.

[0139] The computer program product provided in this application aims to address the technical issues of poor generalization and adaptability of existing anti-attack technologies. Compared to existing technologies, the beneficial effects of the computer program product provided in this application are the same as those of the perturbed image generation method provided in the above-mentioned embodiments, and are not further elaborated here.

[0140] Compared with the prior art, the disturbance image generation method, device, equipment, medium and computer product proposed in the embodiments of the present application extract the business feature information of the target business, perform data standardization processing on the business feature information to obtain standard feature data, perform hash processing on the standard feature data to obtain unique feature data, perform numerical processing and splicing on the unique feature data to obtain a first business feature value, accumulate the first business feature value of the target business to obtain a target business feature value, and finally compare the target business feature value with the feature value set to obtain the generation result of the disturbance image. Compared with the traditional method of identifying duplicate businesses by generating a unique key value or a continuous serial number for each business, it is more efficient, flexible and reliable. Based on the solution of the present application, by subjecting the business of complex scenarios to a series of simple transformations and finally converting it into a comparison of two numbers, the comparison process is very intuitive and efficient. The system only needs to simply compare whether the two values ​​are equal to quickly determine whether the two businesses are exactly the same.

[0141] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0142] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0143] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as mentioned above, and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, controlled terminal, or network device, etc.) to execute the method of each embodiment of the present application.

[0144] The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for generating a disturbed image, characterized in that: The method for generating the disturbed image includes: Using a multi-dimensional perturbation sub-network, perturbation sub-images targeting different perturbation features are generated; Calculating the importance weight of each perturbed sub-image in the spatial dimension, and performing weighted fusion on each perturbed sub-image according to the importance weight to generate an initial fused image; Receive environmental data fed back by the black box, and adjust the fusion weights of each perturbed sub-image according to the environmental data; The initial fused image is feature recalibrated using the fusion weights of the perturbed sub-images to generate a perturbed fused image adapted to environmental characteristics.

2. The method for generating a disturbed image according to claim 1, wherein: After the step of recalibrating the features of the initial fused image using the fusion weights of the perturbed sub-images to generate a perturbed fused image adapted to the environmental characteristics, the following steps are included: Using a pre-built proxy model library, attack simulation is performed on the perturbed fusion image to generate an evaluation matrix containing the attack success rate of each model; When the attack success rate is lower than a preset threshold, the adversarial gradient of the proxy model is extracted through an online adaptation mechanism, and the parameter weights of the multi-dimensional perturbation sub-network are reversely optimized; Using the evaluation matrix to construct a feature vector representing the state of the black box environment, parsing the feature vector through a proximal strategy optimization algorithm to generate a multi-disturbance combination optimization strategy; Based on the multi-perturbation combination optimization strategy and the optimized parameter weights, the fusion ratio of each perturbation sub-image is adjusted to generate an adaptive perturbation fusion image.

3. The method for generating a disturbed image according to claim 1, wherein: The step of generating perturbation sub-images for different perturbation features by using a multi-dimensional perturbation sub-network includes: Using a texture perturbation subnetwork, a texture perturbation image is generated by adding adversarial high-frequency noise to a local area of ​​the image; Using a geometric perturbation subnetwork, slightly deforming key areas of the image to generate a geometrically perturbed image; The frequency domain perturbation subnetwork is used to generate a frequency domain perturbation image by injecting an interference signal of a specific frequency band into the frequency domain of the image.

4. The method for generating a disturbed image according to claim 1, wherein: The step of generating perturbation sub-images for different perturbation features by using a multi-dimensional perturbation sub-network includes: A multidimensional perturbation generator is constructed based on a lightweight network, and perturbed sub-images with different perturbation features are generated in parallel by the multidimensional perturbation generator.

5. The method for generating a disturbed image according to claim 1, wherein: The step of calculating the importance weight of each perturbed sub-image in the spatial dimension and performing weighted fusion on each perturbed sub-image according to the importance weight to generate an initial fused image comprises: Based on the spatial attention mechanism, the spatial features of each perturbed sub-image within the preset grid unit are weighted and the attention weight matrix of each perturbed sub-image is generated; The attention weight matrix of each perturbed sub-image is weightedly summed with the corresponding spatial features to generate the initial fused image.

6. The method for generating a disturbed image according to claim 1, wherein: The step of receiving environmental data fed back by the black box system and adjusting the fusion weights of the perturbed sub-images according to the environmental data includes: The heterogeneous environment data fed back by the black box is received, the heterogeneous environment data is input into the fine-tuning network, the environment-sensitive features are extracted through three fully connected layers, and the fusion weight corresponding to each perturbed sub-image is output.

7. The method for generating a disturbed image according to claim 1, wherein: The step of receiving environmental data fed back by the black box system and adjusting the fusion weights of the perturbed sub-images according to the environmental data further includes: The algorithm response characteristics and target device parameters fed back by the black box are received, and an update rate of the fusion weight is adjusted according to the algorithm response characteristics and target device parameters.

8. A device for generating a disturbed image, characterized in that: The device comprises: Multi-perturbation generation module, used to generate perturbation sub-images targeting different perturbation features using a multi-dimensional perturbation sub-network; a first fusion module, configured to calculate the importance weight of each perturbed sub-image in a spatial dimension, and perform weighted fusion on each perturbed sub-image according to the importance weight to generate an initial fused image; An adjustment module, configured to receive environmental data fed back by the black box and adjust the fusion weights of each perturbed sub-image according to the environmental data; The second fusion module is used to recalibrate the features of the initial fused image using the fusion weights of the perturbed sub-images to generate a perturbed fused image adapted to the environmental characteristics.

9. A device for generating a disturbed image, characterized in that: The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the method for generating a disturbance image according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the method for generating a disturbed image according to any one of claims 1 to 7 are implemented.

Images