Target detection model-oriented anti-attack method

Abstract

The invention discloses a target detection model-oriented anti-attack method, which comprises the following steps of: extracting bounding box information of a target in an image, and dividing a data set into a far view angle and a near view angle according to the proportion of a bounding box in the image; aiming at the characteristic difference caused by the change of the far and near visual angles, designing a corresponding self-adaptive mask strategy to carry out disturbance area control; a condition generator is introduced to generate adversarial disturbance, an NRSU module is constructed in a lower sampling layer, label information is aligned to a feature space through the NRSU module, and label features are guided to be fused into image features through an adaptive mask; and constructing a dynamic weight balance mechanism based on a visual angle, and carrying out multi-visual-angle adaptive optimization on the generated disturbance to obtain an adversarial sample. The method shows stable attack performance under various view angle changes, and has strong view angle adaptability and robustness.

Description

Technical Field

[0001] The present invention belongs to the technical field of anti-attack methods and relates to an anti-attack method for a target detection model. Background Art

[0002] In recent years, deep learning has made significant progress in the field of computer vision. It has been widely applied to a variety of tasks, including image classification, object detection, semantic segmentation, and natural language processing, demonstrating outstanding performance. Object detection, as one of the core tasks in computer vision, aims to accurately locate and identify the categories of all objects of interest in an image, and has consistently garnered significant attention from both academia and industry. However, research has shown that object detection models have potential security vulnerabilities and are susceptible to adversarial examples. Adversarial examples are carefully crafted images that, by adding subtle and imperceptible perturbations to the original image, cause the object detection model to produce erroneous outputs, seriously threatening its reliability in real-world scenarios. For example, in autonomous driving systems, adding subtle perturbations to pedestrians or traffic signs can cause the model to misidentify or even completely fail to detect the object, leading to serious safety issues. Researchers are conducting adversarial attacks on object detection models to uncover potential vulnerabilities and further improve their robustness and security. Therefore, studying adversarial attack methods for object detection models has important theoretical and practical value.

[0003] In the study of adversarial attack methods, they are divided into two types: white-box attacks and black-box attacks, based on the attacker's accessibility to deep learning models. White-box attacks rely on a comprehensive understanding of the model's structure and weight parameters to generate adversarial samples through precise calculations; black-box attacks refer to attacks in which the attacker cannot directly access the internal structure or parameters of the target model and can only generate adversarial samples through limited input and output. In general, white-box attack methods are based on a complete understanding of the model's internal information and have a strong attack effect on white-box models. However, some methods are overly dependent on the model's internal information, resulting in weak transfer attack capabilities. Black-box attack methods use ensemble training or self-ensemble algorithms to optimize the training process and have better black-box attack capabilities. However, some methods do not consider the impact of the target's perspective on the attack results, resulting in poor perspective robustness of the attack method. Summary of the Invention

[0004] The purpose of this invention is to provide a counterattack method for target detection models, which exhibits stable attack performance under various viewing angle changes and has strong viewing angle adaptability and robustness.

[0005] The technical solution adopted by the present invention is a method for countering attacks on target detection models, which is specifically implemented according to the following steps: Step 1: Extract the bounding box information of the target in the image and divide the dataset into two categories: far and near perspectives according to the proportion of the bounding box in the image; Step 2: Based on the feature differences caused by the changes in far and near perspectives, a corresponding adaptive masking strategy is designed to control the disturbance area; Step 3: Introduce a conditional generator to generate adversarial perturbations and construct an NRSU module in its downsampling layer. The NRSU module aligns the label information to the feature space and guides the label features to be integrated into the image features through an adaptive mask. Step 4: Construct a perspective-based dynamic weight balancing mechanism to perform multi-perspective adaptive optimization on the perturbation generated in step 3 to obtain adversarial samples.

[0006] The beneficial effects of the present invention are: (1) The method of the present invention divides the image into far and near perspectives based on the proportion of the target bounding box in the image, and adopts an adaptive mask strategy to control the perturbation area based on the feature differences of targets at different perspectives: a background mask is generated at the far perspective to retain background features, and an edge mask is used at the near perspective to retain target edge information; the target category label is mapped to the feature space using the designed NRSU module, and the multi-level feature fusion is guided by the adaptive mask (far perspective background mask / near perspective edge mask) to generate perspective-sensitive adversarial perturbations; a dynamic weight balancing mechanism for multi-perspective loss is further constructed to adaptively adjust the training weights according to the loss differences between far and near perspectives, thereby enhancing the attack robustness of adversarial samples under perspective changes. The method of the present invention exhibits stable attack performance under various perspective changes, and has strong perspective adaptability and robustness, solving the problem that the existing black-box adversarial attack methods for target detection models are insufficiently adaptable under perspective changes, resulting in poor robustness of the attack methods; (2) The performance of different adversarial attack methods was compared under multi-view conditions. The experimental results show that on the COCO-Person dataset, the attack success rate of the proposed method under white-box and black-box conditions is 1.5% and 2.8% higher than that of the mainstream method, respectively, and the average precision of the target detection model is reduced by 1.2% and 3.5%, respectively. The proposed method effectively improves the attack success rate under different viewpoints and demonstrates better viewpoint robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 It is the overall framework diagram of the method of the present invention; Figure 2 It is a schematic diagram of the structure of the NRSU module in the method of the present invention; Figure 3 It is a schematic diagram of the structure of the second and third layers of the NRSU module in the method of the present invention; Figure 4is a close-up image of the COCO-Person dataset in an embodiment of the present invention; Figure 5 is a far-angle image of the COCO-Person dataset in an embodiment of the present invention; Figure 6 This is the attack visualization effect at a long viewing angle on the YOLOv5 target detection model in an embodiment of the present invention; Figure 7 This is the attack visualization effect at close viewing angle on the YOLOv5 target detection model in an embodiment of the present invention; Figure 8 2 is a comparison chart of the attack success rates of various methods at long viewing angles in the embodiments of the present invention; Figure 9 3 is a comparison chart of the attack success rates of various methods in the embodiments of the present invention at close viewing angles. DETAILED DESCRIPTION

[0008] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0009] The present invention is a method for countering attacks on target detection models. The overall framework is as follows: Figure 1 As shown in the figure, YOLO is used to extract the bounding box information of the target in the image. The image is divided into far and near view channels based on the proportion of the bounding box in the image, comprehensively considering the differences in target features under multiple viewpoints. To address the feature differences caused by viewpoint changes, a corresponding adaptive masking strategy is designed to control the perturbation area: a background mask is generated in the far view channel to preserve the feature information of the background area; an edge mask is constructed in the near view channel to emphasize the edge features of the target. Then, a conditional generator is introduced to generate adversarial samples, and a Nonlinear Residual Scaling Unit (NRSU) module is constructed in its downsampling layer. Through adaptive mask guidance, the category label is aligned with the input image and its feature map in the spatial dimension. Finally, a view-based dynamic weight balancing mechanism is constructed. Based on the loss value calculated by the object detection model under different view channels, the weight distribution of each channel is dynamically adjusted. This adaptively optimizes the attack preference for different viewpoints during training and enhances the adaptability of generated adversarial samples to viewpoint changes.

[0010] The present invention is directed to the anti-attack method of the target detection model, which is specifically implemented according to the following steps: In step 1, the target detection model is used to extract the bounding box information of the target in the image, and the far and near viewing channels are divided according to the proportion of the bounding box in the image to comprehensively consider the differences in target features under multiple perspectives.

[0011] The perspective division process in the method of the present invention is based on the proportion of the target bounding box in the image. This division method is derived from the relationship between the target size and the distance of the perspective. In computer vision, targets at far and near perspectives are generally represented differently in images. Targets at far perspectives usually occupy a smaller space in the image and have a relatively smaller bounding box area. Conversely, targets at near perspectives occupy a larger space in the image and have a larger bounding box area. Based on this characteristic, dividing the perspectives by the proportion of the bounding box can effectively distinguish the relative position and size of the target in the image, thereby processing the target features at different perspectives in a targeted manner.

[0012] The perspective division process of the method of the present invention is as follows: first, the input image is detected by the target detection model (YOLOv5 is used in this invention). x Perform target detection to identify human targets in the image y , and generate the bounding box of each target. For each target O i (No. i targets), the bounding box output by the model is represented by the coordinates of the upper left and lower right corners: ,in is the coordinate of the upper left corner of the bounding box, Is the coordinate of the lower right corner of the bounding box. The width of the target bounding box can be calculated based on the bounding box coordinates w i and height h i : (1) in, 、 Represent the maximum and minimum values ​​of the horizontal axis respectively; 、 Represent the maximum and minimum values ​​of the vertical axis respectively.

[0013] The area of ​​the target bounding box is for: (2) This area represents the area actually occupied by the target bounding box in the image. For the entire image, its total area can be expressed as: (3) in, is the area of ​​the entire image, and is the width and height of the image. In order to quantify the proportion of the target bounding box in the entire image, the ratio of the bounding box area to the total image area is calculated. R i : (4) Based on the proportion R i , set the threshold = 0.3. To distinguish between far and near perspectives, the division rules are as follows: (1) If R i < , indicating that it accounts for a small proportion in the image and is located far away, then the target O i It is divided into distant perspectives; (2) If R i ≥ , indicating that it accounts for a large proportion in the image and is located nearby, then the target O i Classified as near perspective.

[0014] Based on the classification of the targets, the dataset is also divided into two categories: far-view and near-view. It should be noted that in the images divided by the method of the present invention, each image only contains targets of far-view or near-view, and the coexistence of far and near targets is not considered. The final perspective division of the input image can be expressed as: (5) in, represents the perspective division process, x represents the input image, O far and O near are the collections of far-view and near-view targets respectively.

[0015] Step 2: Based on the feature differences caused by the change in viewing angle, a corresponding adaptive mask strategy is designed to control the disturbance area: a background mask is generated in the far viewing angle channel to retain the feature information of the background area; an edge mask is constructed in the near viewing angle channel to emphasize the target edge features.

[0016] For the divided image, according to the target bounding box B i Crop out sub-image For different targets at far and near angles, a differentiation strategy is adopted to fully preserve the target features.

[0017] For distant viewing targets, the method of the present invention uses a background mask for processing and uses rembg to segment the foreground and background of the image. Rembg is implemented based on the U-Net network structure, and a symmetrical contraction path and expansion path are constructed through the encoding-decoding structure. The encoder performs multi-layer downsampling through continuous maximum pooling layers, and each layer uses convolution and ReLU activation functions to extract multi-level features; the decoder implements progressive upsampling through transposed convolution, and finally outputs a pixel-level segmentation mask with the same resolution as the input. After rembg segmentation, a binary mask can be obtained. M i , used to distinguish the foreground and background of the target.

[0018] (6) in, represents the downsampling operation, represents the upsampling operation, represents a 3×3 convolution operation, represents a 1×1 convolution operation, σ is the Sigmoid activation function. But M i What is segmented is the foreground information in the image, and reverse processing is required to extract the background information.

[0019] (7) in, Indicates that it is aimed at a long-angle target O far The core of the adaptive optimization processing function is to extract the background information of the distant target through the background mask.

[0020] By comparing pixel by pixel with background mask, the background information of distant viewing targets can be extracted. , providing a basis for subsequent guidance of disturbance positions.

[0021] For near-angle targets, the method of the present invention focuses on the edge information of the target in the image, performs edge detection on the sub-image, extracts the significant edges of the area, and then uses this edge information to guide the perturbation. Specifically, in order to reduce the interference of noise on the edge detection results, Gaussian blur is first applied to the image sub-region for preprocessing; then the direction and intensity of the edge are obtained by calculating the gradient of the smoothed image, and non-maximum suppression (NMS) is used to retain the position of the local maximum value and suppress other non-edge points; finally, the suppressed image is binarized using the double threshold method to obtain the edge image. This process can be expressed as: (8) in, The standard deviation is αGaussian filter (determines the smoothness of the filter); Indicates that the input image x Convolve with a Gaussian filter to smooth the image; Here, we represent the composite function related to gradient, including gradient calculation, magnitude and direction extraction; Indicates applying non-maximum suppression to the gradient magnitude image, retaining local maxima and excluding irrelevant edge points; Indicates that the double threshold method is applied to the image after NMS processing to separate strong edges, weak edges and non-edges. T high The pixels with a strength less than the low threshold are "strong edges". These pixels are definitely the edges of the target. T low Pixels with an intensity between the high and low thresholds are considered "non-edge" and are completely suppressed. If the pixel intensity is between the high and low thresholds, they are marked as "weak edges". Such pixels are not immediately judged as edges, but if they are connected to strong edges, they will be retained, otherwise they will be suppressed as "non-edges". It is the final edge image, which contains the edge information of the image.

[0022] The entire process of adaptive optimization can finally be expressed as: (9) in, It is an adaptive optimization function, based on the result of perspective division S ( x ), select the corresponding processing method, x mask is the adaptive mask generated after adaptive optimization, Indicates that the target is close to the horizon O near Adaptive optimization processing function.

[0023] In step 3, a conditional generator is introduced to generate adversarial samples, and an NRSU module is constructed in its downsampling layer. The NRSU module aligns the label information to the feature space, and the adaptive mask obtained in step 2 is used to guide the label features to be integrated into the image features to generate adversarial samples from different perspectives.

[0024] After the image is divided into near and far scales and adaptively optimized, the conditional generator is used to generate adversarial perturbations. The input of the conditional generator includes not only the original image and the target category, but also the mask obtained through adaptive optimization, which is used to guide the injection of label information. The target category is first mapped to the feature space through the category mapping module and aligned with the image features through the designed NRSU module. At the same time, the adaptive mask is also input into the generator. Since it only contains one dimension, it needs to be aligned with the image feature dimension. Through the three NRSU modules of the encoding layer, the target information is embedded in the image features at multiple levels. Finally, after subsequent convolution, residual connection and decoding processes, the adversarial perturbation is finally generated. The NRSU module ensures that the target information and image features are deeply integrated in the multi-dimensional space, thereby achieving more accurate and effective adversarial sample generation.

[0025] (1) Category mapping module The target category is encoded as a feature vector in this module and used as conditional information to input into the NRSU module. Since category labels are usually expressed in scalar form, if used directly, the model may not be able to fully capture the semantic features contained therein. Therefore, this module embeds the category label into a high-dimensional space and enhances the expression of category information by constructing a semantic feature vector. For any specified target category, c , and its mapped eigenvector can be expressed as: (10) in, e c is the target category c The one-hot encoding representation of is equal to the total number of all categories. Taking the COCO dataset as an example, the dataset contains 80 categories, so each category can be represented as an 80-dimensional one-hot vector, where only the index position of the target category is 1 and the rest are 0. Subsequently, by parameter Z The defined fully connected layer completes the linear transformation and maps it to the feature dimension required by the generator. Through the above process, the category mapping module completes the effective mapping of the target category from discrete labels to continuous feature space.

[0026] (2) NRSU module The NRSU (Nonlinear Residual Scaling Unit) module designed by the present invention is as follows Figure 2 As shown in Figure 1, the target category is processed by the category mapping module and mapped to a category feature map of the same size as the input image or upper layer feature. Due to its low dimensionality (single-channel image), the adaptive mask needs to be aligned to the dimensional space of the category feature map to guide the category feature map to be added to the input image. This process is expressed as: (11) in, Represents the process of dimension alignment, and the resulting high-dimensional mask is . The adaptive perturbation generated by fusing the high-dimensional mask with the category feature map guides the label information contained in the category feature map to be added to the input image, and further uses the Conv convolution layer in combination with the downsampling + convolution steps to further compress the features and extract information, thereby extracting richer spatial features. The Transposed Conv layer is used to perform upsampling operations to restore the spatial resolution of the image, and to perform residual connections with the features of the upper layer to enhance the target information embedded therein, ensuring that effective target class information can still be extracted even when the number of subsequent convolutional layers is large. Finally, after processing with the ReLU activation function, the nonlinear representation capability of the output feature map is enhanced. The first layer NRSU processing is shown in Equation (12): (12) Due to the downsampling process, the image size and dimension are deepened. Therefore, in order to correspond to the features in the deeper image, the adaptive perturbation will also be input into the NRSU module. The second and third layers of the NRSU module are simplified as follows Figure 3 As shown, it can be expressed as: (13) Finally, the entire NRSU module can be expressed as: (14) in, represents the perspective division process, is the one-hot encoding representation of the true category y.

[0027] (3) Perturbation generation After the NRSU module completes feature extraction and category information injection, it is further processed by the subsequent residual block and decoder in the generator to finally generate the perturbation. The conditional generator can be expressed as: (15) in, Represents a condition generator, y is the true category of human targets, c is the target category to be attacked, Decoder( ) represents the decoding process, which includes upsampling processing, used to map the generated perturbation to the same size as the input image; ResBlock( ) is the process of extracting features, including convolution and residual processing. The perturbation generated by the generator δ Will be added to the original image x On the other hand, we get the adversarial sample: (16) The perturbation size is limited to In (the method of the present invention sets =16 / 255), ensuring that the perturbation is not too obvious visually and avoids excessively destroying the structure of the original image while still being able to play an offensive role.

[0028] Step 4: Construct a dynamic weight balancing mechanism based on perspective. According to the loss value calculated by the target detection model under different perspective channels, dynamically adjust the weight distribution of each channel, thereby adaptively optimizing the attack preference for different perspectives during training and enhancing the adaptability of generated adversarial samples to perspective changes.

[0029] While integrating near and far perspectives, the proposed method introduces a dynamic weight balancing mechanism to achieve dynamic adjustment of perspective weights during training. By optimizing the perspective channel weights during training, the model can more effectively capture target information from different perspectives, thereby improving the perspective robustness of generated adversarial samples. The loss function for a single perspective channel is defined as shown in Equation (17): (17) in, V i Indicates the i The loss value from each perspective; represents the cross entropy loss, which aims to minimize the distance to the target category; represents the target detection model, c represents the label of the target category, represents an adversarial example with added perturbations; Indicates the target category of the target detection model c The predicted probability of Represents the model's response to the original category y The predicted probability of By maximizing the prediction probability of the target detection model for the target category, adversarial samples are more likely to be misclassified. c Class; at the same time, It is used to minimize the model's predicted probability of the original category, reduce its confidence in the correct category, thereby increasing the probability of misclassification and weakening the overall prediction accuracy of the model.

[0030] After calculating the loss value of each view channel, the corresponding weight is adjusted according to the size of the loss of different view channels. If the loss of a certain view channel is large, it means that the attack success rate under this view channel is low. Therefore, it is necessary to optimize the attack effect of this view channel and normalize the loss values ​​of different view channels to calculate the weight of each view channel. The formula for calculating the weight after normalization of the loss values ​​of each view channel is shown in Equation (18): (18) in, represents the weights at different viewpoints during training, and C represents the total number of viewpoints.

[0031] The loss function of the entire training process is defined as shown in Equation (19).

[0032] (19) By dynamically adjusting the weights of each view loss, the weight balancing strategy effectively combines information from different viewpoints, ensuring that adversarial attacks are not limited to a specific viewpoint. During training, the weights are adaptively adjusted based on the performance of the view loss, ensuring that generated adversarial examples maintain good attack effectiveness across different viewpoints, thereby improving the method's robustness across different viewpoints.

[0033] Example 1: (1) Experimental setup In order to verify the effectiveness of the adversarial sample generation method of the present invention, it is compared with three papers, namely, paper 1 (LIANG S, WU B, FAN Y, et al. Parallel rectangle flip attack: a query-basedblack-box attack against object detection [C] / / Proceedings of IEEE / CVFInternational Conference on ComputerVision. 2022: 7677–7687.), paper 2 (YANG Z,LIU S, HU H, et al. RepPoints: point set representation for object detection[C] / / Proceedings of IEEE / CVF International Conference on Computer Vision.2020: 9656–9665.) and paper 3 (NGUYEN DA, XUAN LN, TRANG VHM, et al. A Query-based Black-box Attack to Evaluate the Robustness of One-Stage ObjectDetectors [C] / / Proceedings of the 15th International Conference on Knowledgeand Systems Engineering. 2023: 1–6.), and are compared with three methods, which are named TOG, OSFD, and TDAA respectively.

[0034] The experimental hardware configuration for this example is: an Intel Core i7-13700K 3.4GHz processor, an NVIDIA GeForce RTX 4090 graphics card with 24GB of video memory. The software environment uses Ubuntu 20.04 as the operating system, Python 3.8 as the programming language, and PyTorch 1.12.1 as the deep learning framework to ensure efficient execution of the experiment.

[0035] This embodiment uses the COCO-Person human dataset, a subset of the MS-COCO target detection dataset, the INRIA-Person dataset, and the VOC-Person human dataset, a subset of the VOC dataset. COCO-Person focuses on the detection of human targets, covering human images in various scenarios, including approximately 64,115 images and 262,465 human instances. The INRIA-Person dataset focuses on pedestrian detection, including 1,832 training images and 1,288 test images. Its positive samples are mainly full-body images of upright pedestrians, with diverse background samples covering complex natural scenes. These two datasets can provide rich and diverse training samples, which will help the model learn more comprehensive feature representations, thereby improving its adaptability in different fields and scenarios. The dataset images after dividing the perspective are as follows: Figure 4 and Figure 5 As shown in Figure 2, they are the near-view dataset and the far-view dataset, respectively.

[0036] In order to better evaluate the effectiveness of the attack, this embodiment uses the targeted attack success rate and the average accuracy of the model as the main evaluation criteria. The targeted attack success rate is defined as p adv = N adv_target / N ,in, p adv is the attack success rate, N is the total number of samples in the test set, N adv_target The average precision represents the area under the precision-recall curve, comprehensively evaluating the accuracy at different recall rates without any preference between precision and recall. It indicates the detection ability of the detection model for the category. A larger value indicates a better detection effect of the detection model for the category. The definition of average precision is: Average precision is the core evaluation indicator of target detection, and quantitative evaluation is achieved by calculating the integrated area under the precision-recall curve: (20) in, P ( R ) indicates the recall rate is R This indicator eliminates the subjective influence of artificially set confidence thresholds and can objectively reflect the comprehensive performance of the model at all possible operating points. AP The closer the value is to 1, the stronger the model's ability to identify the category is.

[0037] Example 2: Based on Example 1, the attack effect of the method of the present invention is as follows Figure 6 and Figure 7 As shown, Figure 6 Shows the attack effect from a long perspective. Figure 7 Demonstrates the attack effect from a close perspective. Figure 6 and Figure 7 The upper middle part is the original image, and the pedestrian targets in the image are all accurately identified; however, in the lower half, the pedestrian targets in the adversarial sample are mistakenly identified as frisbees. At the same time, due to the disturbance, the positions of the prediction boxes in some images are also shifted, indicating that the adversarial attack method of the present invention can effectively interfere with the target detection model.

[0038] Example 3: Based on Example 1, an ablation experiment was performed: Tables 1 and 2 show comparative ablation experiments on different modules in the proposed method. Using the classic two-stage Faster R-CNN model and the single-stage YOLOv5 model as source models, the effectiveness of these modules was tested on different black-box object detection models. Different schemes were employed to analyze the impact of different modules and their combinations on the attack effectiveness. Scheme 1: Multi-view scaling, Scheme 2: NRSU, Scheme 3: Multi-view scaling + NRSU, and Scheme 4: Multi-view scaling + NRSU + dynamic weight balancing.

[0039] Table 1 shows the ablation comparison results of the proposed method in terms of average precision. It can be seen that by introducing multi-view scaling and the NRSU module, the proposed method can significantly reduce the average precision on the white-box model, while also performing better on the black-box model. Taking the YOLOv5 model as an example, when the multi-view scaling module is used alone, the average precision of the model is reduced by 45.7%. When the multi-view scaling and NRSU modules are combined, the overall average precision of the black-box model is reduced by 47.2%. After further introducing dynamic weight balancing, the overall attack effect is further improved, and the overall average precision of the black-box model is reduced by 48.8%. The combination of multi-view scaling and dynamic weight balancing strategies can produce stronger attack effects on different types of target detection models.

[0040] Table 1 Average precision ablation comparison

[0041] The ablation comparison results of the attack success rate of the method of the present invention are shown in Table 2. With the gradual introduction of each module, the overall attack success rate shows an upward trend. When the multi-view scale module is used alone, the average attack success rate of the adversarial samples generated with YOLOv5 as the source model reaches 59.4%, while the attack success rate of Faster R-CNN is 56.1%. After the introduction of the NRSU module, the attack success rate increased slightly to 60.6% and 57.0%, indicating that the NRSU module has improved the aggressiveness of the perturbation to a certain extent. After combining the multi-view scale and NRSU modules, the attack success rates increased to 65.1% and 57.2%, respectively. Finally, after adding the dynamic weight balancing mechanism, the attack effect is optimal, and the attack success rate is further increased to 68.7% and 60.9%, and it shows good results on multiple target models.

[0042] Table 2 Comparison of attack success rate ablation

[0043] Comparing the two metrics shows that the multi-view scale training method can effectively improve the attack capability of adversarial examples. By optimizing perturbations at the level of multiple viewpoint feature representations, the perturbations further influence multiple decision channels of the model, thereby enhancing the attack effectiveness of adversarial examples. This effect is further enhanced by the introduction of dynamic weight balancing. The dynamic weight balancing strategy adaptively adjusts the weight distribution of each viewpoint during training, strengthening the learning of features from different viewpoints. The generated perturbations not only have attack capabilities within a single viewpoint, but also introduce attention to multiple viewpoints within the multi-layer structure of the model, further enhancing the comprehensiveness and robustness of the attack.

[0044] In addition, the NRSU module also plays an important role in improving the effectiveness of attacks. By introducing residual and skip connections in the convolutional layer, the NRSU module optimizes the concatenation of the latent vector generated by the label and the image features. This design helps preserve the target information contained in the label and improves its propagation ability in the target model, allowing the perturbation to interfere at different levels and feature spaces, thereby improving the effectiveness of the attack generated adversarial perturbations. The skip connection of the NRSU module also strengthens the connection with the original features, while ensuring the success rate of the attack, minimizing the invisible interference to the input image, and improving the stealth and practicality of the attack.

[0045] Example 4: Based on Example 1, a migration experiment was performed: Table 3 shows a comparison of the mean average precision of various methods on the COCO-Person test set. The diagonal data in the table show the results of generating adversarial examples on a white-box model, reflecting the change in mean average precision of the model itself. The data elsewhere in the table represent the change in mean average precision caused by adversarial examples trained on the white-box model when tested on a black-box model. Compared to other mainstream methods, our method performs significantly better in weakening white-box models and demonstrates stronger effectiveness against black-box models. Taking VFNet as an example, our method achieves a mean average precision of 2.4% after adversarial attack on this white-box model, 1.1% lower than the best comparison method, OSFD (3.5%). In black-box model testing, adversarial examples generated using VFNet as the white-box model reduced the mean average precision of the black-box model to 19.3%, a 4% decrease from the 23.1% achieved by the best comparison method, OSFD.

[0046] Table 3 Comparison of average accuracy of various methods on the COCO-Person test set

[0047] Table 4 shows a comparison of the attack success rates of various methods on the COCO-Person test set. The diagonal data in the table represent the success rates of white-box attacks, while the other rows show the success rates of attacks on black-box models. Compared with existing methods, the proposed method demonstrates a significant advantage in attack success rate, particularly on black-box models. Taking YOLOv3 as an example, the proposed method achieved a 93.5% success rate in white-box attacks, a 1.9% improvement over the best comparison method, OSFD (91.6%). In black-box attack experiments, using YOLOv3 as the white-box model to generate adversarial examples, the black-box attack success rate reached 65.2%, a 2.2% improvement over OSFD's 63.0%. Furthermore, experimental results on other object detection models (such as YOLOv5, VFNet, Faster R-CNN, and Mask R-CNN) also demonstrate that the proposed method demonstrates significant superiority in attack success rates for both white-box and black-box models.

[0048] Table 4 Comparison of attack success rates of various methods on the COCO-Person test set

[0049] When using the YOLO series of models as white-box models, the generated adversarial examples generally have good transfer attack capabilities on other models in the same series. In particular, adversarial examples generated based on YOLOv3 show a higher attack success rate within the YOLO family. However, these examples are relatively weak in attacking two-stage detection models (such as the R-CNN series). In contrast, if Faster R-CNN or Mask R-CNN is used as a white-box model, the adversarial examples they generate not only have excellent attack effects within the R-CNN series, but also have strong interference capabilities on YOLO series models. This is due to the more complex structure and feature processing of the two-stage detection model, which enables it to guide the perturbation during training to capture more generalized and offensive features, thereby enhancing the transfer effect of adversarial examples in cross-model environments.

[0050] Example 5: Based on Example 4, the average accuracy comparison of each method on different datasets is shown in Table 5, which shows the adversarial samples generated by training on the COCO Person dataset using YOLOv5 as a white box model, and cross-dataset testing on other datasets such as VOC Person and INRIA Person. Although the attack effect of the method of the present invention in the cross-dataset test has weakened, it still shows strong generalization ability as a whole. As shown in Tables 3 and 5, on the COCO Person test set, the adversarial samples reduced the average accuracy of YOLOv5 to 4.7%, demonstrating the efficient attack performance of the method of the present invention under the white box setting. After migrating the same adversarial samples to the VOC Person and INRIA Person datasets, the model accuracy dropped to 10.3% and 15.1%, respectively, indicating that the method has good cross-dataset migration and can maintain strong attack capabilities in different scenarios.

[0051] Table 5 Comparison of average accuracy of various methods on different datasets

[0052] In black-box attack scenarios, while the attack performance of different methods generally degrades under cross-dataset conditions, our method still outperforms existing comparison methods on multiple metrics. Using YOLOv3 as the black-box model, adversarial examples generated by our method reduced detection accuracy to 18.7% and 22.6% on the VOC Person and INRIA Person test sets, respectively, 4.8% and 6.2% lower than the best comparison methods.

[0053] Table 6 shows a comparison of the attack success rates of each method on different datasets. The experimental settings are the same as those in Table 5, and the attack success rates of each method in the cross-dataset test are all reduced. From the comparison of the results in Table 4 and Table 6, it can be seen that the method of the present invention still maintains a strong attack effect on the VOC Person and INRIA Person datasets. Specifically, the attack success rate of the adversarial sample on the original training set reaches 90.2%, while the success rates on VOC Person and INRIA Person are 82.6% and 75.8%, respectively. Although the cross-dataset performance is slightly attenuated, the overall attack performance remains stable, showing a good white-box attack effect.

[0054] Table 6 Comparison of attack success rates of various methods on different datasets

[0055] When conducting black-box attack tests across data sets, although the attack success rates of all adversarial attack methods showed a downward trend, the method of the present invention still maintained a high attack success rate on each data set and outperformed other comparison methods. Specifically, in experiments on the VOC Person and INRIA Person data sets, when using VFNet as the target model, the attack success rates of the method of the present invention were 48.4% and 42.4%, respectively. Compared with the optimal comparison method, the attack success rates were increased by 4.3% and 7.7%. This result shows that despite the widespread performance degradation in cross-dataset testing, the method of the present invention can still maintain its migration attack capability on black-box models and outperforms other comparison methods on different data sets.

[0056] Since the feature distribution of the new dataset is different from that of the original dataset, the adaptability of the adversarial samples on the new dataset is reduced. Therefore, the attack success rate of most methods on the new dataset generally shows a downward trend. However, the method of the present invention still demonstrates strong attack capabilities on multiple datasets and maintains a high attack success rate, indicating that it has significant advantages in data migration and adaptability. Through the design of multi-perspective scales, the method of the present invention fully mines the diverse perspective features in the existing dataset, and flexibly adjusts the attention to different perspectives through a dynamic weight balancing strategy, thereby effectively improving the perspective robustness of the adversarial samples and enhancing the performance of the generated adversarial samples in dealing with differences between different datasets. In addition, the NRSU module designed by the method of the present invention further enhances the robustness and generalization ability of the adversarial samples during the training process by performing multi-level feature splicing on the input image and target category information. This strategy enables the method of the present invention to not only perform well on a single dataset, but also enhances the wide adaptability and generalization ability of the adversarial attack method across datasets.

[0057] Example 6: Based on Example 5, the attack success rate comparison of each method under different perspectives is as follows: Figure 8 and Figure 9 As shown in the figure, the data reflects the attack effect of generating adversarial samples on various target detection models when YOLOv5 is used as a white box model.

[0058] Figure 8 Shows the attack success rate at a long viewing angle. Figure 9 The attack success rate at close viewing angles is demonstrated. At far viewing angles, the attack success rate of the method of the present invention is generally higher than that of other comparison methods. Taking YOLOv3 as an example, the attack success rate of the method of the present invention can reach 72.4%, which is 2.7% higher than the best comparison method. The performance of the OFSD and TDAA methods at far viewing angles is relatively close, while the attack success rate of the TOG method is relatively low. At close viewing angles, the attack success rates of all methods have decreased, but the method of the present invention still has an advantage. Nevertheless, compared with far viewing angles, the gap between the method of the present invention and other methods has narrowed. For example, on the YOLOv5 and Faster R-CNN models, the performance of the method of the present invention is relatively close to that of the OFSD method; and on the VFNet model, the method of the present invention still has a significant leading advantage, with the attack success rate increased by 5.8%.

[0059] The methods presented in this paper demonstrate enhanced attack capabilities at both long and short viewing angles. However, at short viewing angles, the overall attack success rates of all methods are relatively low and the performance is relatively similar. This phenomenon is closely related to the differences in image features between long and short viewing angles. At long viewing angles, the target is farther away, occupies a smaller portion of the image, and features are less detailed. This allows attackers to easily alter target recognition results with relatively simple perturbations. Due to the lack of target detail, the model inherently has a certain degree of ambiguity during recognition. Combined with the effect of perturbations, this makes it even more difficult for the model to accurately identify the target, thereby increasing the attack success rate. At short viewing angles, however, target detail is richer and the image clarity is higher, requiring attackers to generate more sophisticated and precise perturbations to avoid detection by the model. As target detail increases, the model's sensitivity to subtle changes in the image increases, further increasing the difficulty of detecting adversarial examples. Furthermore, at short viewing angles, the target is larger, requiring more features to be extracted and analyzed, significantly increasing the difficulty of adversarial attacks. Therefore, although targets are more distinct at short viewing angles, the increased complexity of the details results in a relatively low attack success rate.

Claims

1. The adversarial attack method for target detection models is characterized by: Please follow the steps below to implement it: Step 1: Extract the bounding box information of the target in the image and divide the dataset into two categories: far and near perspectives according to the proportion of the bounding box in the image; Step 2: Based on the feature differences caused by the changes in far and near perspectives, a corresponding adaptive masking strategy is designed to control the disturbance area; Step 3: Introduce a conditional generator to generate adversarial perturbations and construct an NRSU module in its downsampling layer. The NRSU module aligns the label information to the feature space and guides the label features to be integrated into the image features through an adaptive mask. Step 4: Construct a perspective-based dynamic weight balancing mechanism to perform multi-perspective adaptive optimization on the perturbation generated in step 3 to obtain adversarial samples.

2. The method for countering attacks on target detection models according to claim 1, characterized in that: Step 1 is as follows: Use YOLOv5 to input image x Perform target detection to identify targets in the image and generate a bounding box for each target; calculate the width and height of the target bounding box to obtain its area, and then calculate the ratio of the bounding box area to the total image area R i ; Set threshold ,if R i < , indicating the target O i If it occupies a small proportion in the image and is far away, it is classified as a far perspective; if R i ≥ , indicating the target O i If it occupies a large proportion in the image and is located nearby, it is classified as a near perspective.

3. The method for countering attacks on target detection models according to claim 1, characterized in that: In step 2, for the target with a long viewing angle, a background mask is used to process it to retain the feature information of the background area, specifically: According to the target bounding box B i Crop distant angle images , use rembg to get the foreground mask M i ; First find the reverse mask 1- M i , get the background mask, and then multiply it with the sub-image pixel by pixel to accurately extract the background information of the distant target , this background information is used to guide the subsequent disturbance; the specific processing process is shown in the following formula: Where, represents the downsampling operation, represents the upsampling operation, represents a 3×3 convolution operation, represents a 1×1 convolution operation, σ is the Sigmoid activation function, represents pixel-by-pixel multiplication, x Represents the input image.

4. The method for countering attack on target detection model according to claim 1, characterized in that: In step 2, for close-angle targets, an edge mask is constructed to extract the significant edges of the region and emphasize the target edge features. Specifically: First, Gaussian blur is applied to the image sub-region for preprocessing; then the direction and strength of the edge are obtained by calculating the gradient of the smoothed image, and non-maximum suppression is used to retain the position of the local maximum and suppress other non-edge points; finally, the suppressed image is binarized using the double threshold method to obtain the edge image; this process is expressed as: in, The standard deviation is α Gaussian filter; Indicates that the input image x Convolve with a Gaussian filter to smooth the image; Represents composite functions related to gradients, including gradient calculation, magnitude, and direction extraction; Indicates applying non-maximum suppression to the gradient magnitude image, retaining local maxima and excluding irrelevant edge points; Indicates that the double threshold method is applied to the image after non-maximum suppression processing to separate strong edges, weak edges and non-edges, where T low 、 T high Respectively represent the low and high thresholds of intensity; It is the final edge image, which contains the edge information of the image.

5. The method for countering attacks on target detection models according to claim 1, characterized in that: In step 3, the input of the conditional generator includes the original image, the target category, and the mask obtained by adaptive optimization in step 2, which is used to guide the injection of label information.

6. The method for countering attacks on target detection models according to claim 5, characterized in that: The specific workflow of the condition generator is as follows: First, the target category is processed by the category mapping module and mapped into a category feature map of the same size as the input image or upper-layer features. At the same time, the adaptive mask is also input into the conditional generator, and its dimension is aligned to the dimensional space of the category feature map to guide the category feature map to be added to the input image; secondly, the target information is embedded in the image features at multiple levels through the three NRSU modules of the encoding layer; finally, after convolution, residual connection and decoding processes, the adversarial perturbation is finally generated.

7. The method for countering attacks on target detection models according to claim 6, characterized in that: The NRSU module includes a Conv convolution layer, a downsampling + Conv convolution layer, a Transposed Conv layer and a ReLU activation function. Among them, the Conv convolution layer combines the downsampling + convolution steps to further compress features and extract information to extract richer spatial features; the Transposed Conv layer is used to perform upsampling operations to restore the spatial resolution of the image, and perform residual connections with the features of the upper layer to enhance the target information embedded therein, ensuring that even when the number of subsequent convolution layers is large, effective target class information can still be extracted; finally, after processing with the ReLU activation function, the nonlinear representation capability of the output feature map is enhanced.

8. The method for countering attacks on target detection models according to claim 1, characterized in that: In step 4, the dynamic weight balancing mechanism dynamically adjusts the weight distribution of each channel based on the loss value calculated by the target detection model under different view channels, thereby adaptively optimizing the attack preference for different view channels during training; Among them, the loss function of a single perspective channel is defined as follows: in, V i Indicates the i The loss value from each perspective; represents the cross entropy loss, which aims to minimize the distance to the target category; represents the target detection model, c represents the target category, represents an adversarial example with added perturbations; Indicates the target category of the target detection model c The predicted probability of Represents the model's response to the original category y The predicted probability of By maximizing the prediction probability of the target detection model for the target category, adversarial samples are more likely to be misclassified. c kind; Used to minimize the model's predicted probability of the original category, reduce its confidence in the correct category, thereby increasing the probability of misclassification and weakening the overall prediction accuracy of the model; After calculating the loss value of each perspective channel, the loss values ​​of different perspectives are normalized and the weight of each perspective is calculated as shown in the following formula: in, represents the weights of different viewpoints during training, and C represents the total number of viewpoints; The loss function of the entire training process is defined as follows: By dynamically adjusting the weights of each perspective loss, the dynamic weight balancing mechanism can combine information from different perspectives, so that the adversarial attack is not limited to a specific perspective.

Images