A general evaluation method and system for target detectors

Abstract

This invention discloses a general evaluation method for object detectors. It allocates a fixed number of candidate boxes as attack targets for each real object from the object detector's output. On one hand, in the spatial domain, it alters the position, size, and label of the predicted boxes through candidate box coordinate zeroing attacks and foreground-background separation attacks. On the other hand, in the frequency domain, it focuses on high-frequency information of the image, concentrating noise in high-frequency regions representing semantics. This improves the attack effect and the stealth of adversarial examples from two angles, ensuring the accuracy of the evaluation method. This invention solves the technical problem that existing perturbation-based and patch-based attack methods cannot evaluate all object detectors, and that existing perturbation-based attack methods, due to generating image-level global perturbations, incur unnecessary computational costs and are easily perceptible to humans when optimizing attacks on non-critical objects.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, and more specifically, relates to a general evaluation method and system for target detectors. Background Technology

[0002] In the field of computer vision, object detectors have made significant progress in handling complex scenes and improving detection accuracy. Object detection involves classification and regression subtasks, requiring simultaneous object localization and recognition, i.e., providing bounding box locations and classification results. Existing object detectors are mainly divided into two-stage detectors and single-stage detectors; two-stage detectors (such as R-CNN, Faster R-CNN, and Cascade R-CNN) first generate candidate regions through RPN, and then perform accurate classification and regression on these regions; while single-stage detectors (such as YOLO and TOOD) directly predict the object category and bounding box coordinates in the entire image in a single evaluation step.

[0003] As object detectors are increasingly used in key areas such as autonomous driving, intelligent surveillance, and facial recognition, their security and robustness issues are becoming increasingly prominent. In practical applications, object detectors may be vulnerable to various attacks, especially adversarial attacks, where attackers introduce carefully designed micro-perturbations into the input data to cause errors in the model's output. Therefore, how to evaluate the robustness of object detectors has become an urgent problem to be solved.

[0004] To address this issue, researchers have proposed various attack methods to evaluate the robustness of target detectors. Existing attack methods are mainly divided into two types: the first is a patch-based attack method in the physical world, which deceives the target detector by generating a perceptible patch that is independent of the image and then placing this patch anywhere in the image; the second is a perturbation-based attack method in the digital world, which deceives the target detector by adding subtle, imperceptible perturbations to the image, such as noise, pixel-level modifications, or other types of image transformations.

[0005] However, both of the above attack methods have some significant drawbacks: First, both methods rely on specific modules of the target detector, which greatly limits the scalability of the attack and is not a universal evaluation framework, so it cannot evaluate all target detectors; Second, the perturbation-based attack method generates image-level global perturbations, and optimizing attacks on non-critical objects (such as the background) will generate unnecessary computational costs that are easily perceived by humans. At the same time, attacking meaningful targets and irrelevant background elements may lead to inaccurate evaluation results. Summary of the Invention

[0006] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides a general evaluation method for target detectors. Its purpose is to resolve the technical problems of existing perturbation-based and patch-based attack methods being unable to evaluate all target detectors, and the technical problems of existing perturbation-based attack methods generating image-level global perturbations, leading to unnecessary computational costs and easily perceptible technical issues when optimizing attacks on non-critical objects, as well as the technical problems of inaccurate evaluation results caused by attacking meaningful targets and irrelevant background elements. The adversarial attack relied upon in this invention directly utilizes the candidate boxes output before the post-processing stage of the target detector, without relying on any of its internal structure for attack evaluation.

[0007] To achieve the above objectives, according to one aspect of the present invention, a general evaluation method for target detectors is provided, comprising the following steps:

[0008] (1) Obtain image x and input the obtained image x into the target detector to be evaluated to obtain multiple candidate boxes. Select k bounding boxes from all candidate boxes from the perspectives of classification and regression respectively, and use them as attack targets from the regression perspective and the classification perspective respectively.

[0009] (2) Calculate the spatial loss function based on the attack target from the regression angle and the attack target from the classification angle obtained in step (1), calculate the frequency loss function based on the image x, and use the obtained spatial loss function and frequency loss function to iteratively train the perturbation δ until the maximum number of iterations is reached, thereby obtaining the adversarial sample, and input the adversarial sample into the target detector to be evaluated to obtain the detection result of the target detector to be evaluated.

[0010] Preferably, step (1) includes the following steps:

[0011] (1-1) Obtain image x and input the obtained image x into the target detector to be evaluated to obtain multiple candidate boxes, as well as the position coordinates and class prediction score set of each candidate box. The class corresponding to the highest class prediction score in the class prediction score set is taken as the predicted class of the candidate box.

[0012] (1-2) For all candidate boxes obtained in step (1-1), calculate the intersection-union ratio (IoU) score between each candidate box and its corresponding ground truth box, and arrange all candidate boxes in descending order of the obtained IoU scores.

[0013] (1-3) Based on the predicted categories of all candidate boxes obtained in step (1-1), select multiple candidate boxes whose predicted categories are completely consistent with the labels of the corresponding real target boxes from the sorting results obtained in step (1-2).

[0014] (1-4) Select the top k candidate boxes from the arrangement results obtained in step (1-2) as the attack targets from the regression angle; where the value of k ranges from 5 to 50, preferably 15.

[0015] (1-5) Select the top k candidate boxes from the multiple candidate boxes obtained in step (1-3) as the attack targets from the classification perspective.

[0016] Preferably, step (2) includes the following steps:

[0017] (2-1) For each candidate box obtained in steps (1-4) and (1-5), move the coordinates of the top left and bottom right corners of the candidate box closer to the origin coordinates (0,0) to obtain the regression loss function J in the spatial domain corresponding to the candidate box. loc ;

[0018] (2-2) For each candidate box among all candidate boxes obtained in steps (1-4) and (1-5), the set of category prediction scores for the nth candidate box obtained in step (1-1) is used. Obtain the classification loss function J in the spatial domain corresponding to the nth candidate box. cls ,in This represents the category prediction score of the label of the ground truth bounding box corresponding to the nth candidate box, and K represents the total number of categories that the object detector to be evaluated can detect, as preset. This represents the category prediction score of the Kth category (i.e., background) corresponding to the nth candidate box;

[0019] (2-3) For each candidate box among all candidate boxes obtained in steps (1-4) and (1-5), the regression loss function J corresponding to the nth candidate box obtained in step (2-1) is... loc The classification loss function J corresponding to the nth candidate box obtained in step (2-2) cls The values ​​are added proportionally to obtain the spatial loss function J corresponding to the nth candidate box. sa ;

[0020] (2-4) The image x is decomposed into a low-frequency component x using the Discrete Wavelet Transform (DWT) method. ll A high-frequency component x hh and two mid-frequency components x lh and x hl ;

[0021] (2-5) Use the inverse discrete wavelet transform (IDWT) method to process the low-frequency component x obtained in step (2-4). lland high-frequency component x hh Perform signal reconstruction to obtain the reconstructed image;

[0022] (2-6) Obtain the adversarial sample x+δ after adding perturbation δ, and process the adversarial sample x+δ after adding perturbation δ using the formulas in steps (2-4) and (2-5) above, so as to obtain the adversarial sample φ(x+δ) after low-frequency reconstruction and the adversarial sample ψ(x+δ) after high-frequency reconstruction respectively.

[0023] (2-7) Obtain the frequency domain loss function J based on the reconstructed low-frequency adversarial sample φ(x+δ), the reconstructed low-frequency image φ(x), the reconstructed high-frequency adversarial sample ψ(x+δ), and the reconstructed high-frequency image ψ(x) obtained in steps (2-5) and (2-6). fa ;

[0024] (2-8) Add the spatial domain loss function obtained in step (2-3) and the frequency domain loss function obtained in step (2-7) to obtain the total loss function J. total And according to the total loss function J total And the perturbation δ is optimized using gradient descent to obtain the optimized perturbation;

[0025] (2-9) Trim the optimized perturbation obtained in step (2-8) to obtain the trimmed perturbation;

[0026] (2-10) Use the loss function J from step (2-8) total The cropped perturbation obtained in step (2-9) is iteratively optimized until the maximum number of iterations (epochs) is reached, thus obtaining the final optimized perturbation. The final optimized perturbation is then combined with the image x to form an adversarial example.

[0027] (2-11) The adversarial samples obtained in step (2-10) are cropped to obtain cropped adversarial samples;

[0028] (2-12) Input the cropped adversarial sample obtained in step (2-11) into the target detector to be evaluated for detection to obtain the detection result.

[0029] Preferably, the regression loss function J in step (2-1) loc Using the Smooth L1 loss function, we have:

[0030]

[0031] in This represents the nth candidate box among all candidate boxes obtained in steps (1-4) and (1-5). Let n represent the nth predefined target bounding box with the coordinates of the top left and bottom right corners both being (0,0), and n∈[1,N], where N represents the total number of candidate boxes obtained in steps (1-4) and (1-5), and SmoothL1 represents the Smooth L1 loss function.

[0032] Preferably, the classification loss function J in the spatial domain in step (2-2) cls The following formula is used:

[0033]

[0034] The loss function J in the spatial domain part of step (2-3) sa The following formula is used:

[0035] J sa =J loc +λJ cls

[0036] Where λ∈[0,1000].

[0037] Preferably, the specific formulas in steps (2-4) are as follows:

[0038] x ll =LxLT,x hh =HxHT,x lh =LxH T ,x hl =HxL T

[0039] Where L represents the coefficient matrix corresponding to the low-pass filter used in the DWT method, and H represents the coefficient matrix corresponding to the high-pass filter used in the DWT method.

[0040] The specific formulas in steps (2-5) are as follows:

[0041] φ(x)=L T x ll L = L T (LxL T )L

[0042] ψ(x)=H T x hh H = H T (HxH T )H

[0043] Where φ(x) represents the image after reconstruction of the low-frequency part, and ψ(x) represents the image after reconstruction of the high-frequency part.

[0044] Preferably, the specific formulas in steps (2-7) are as follows:

[0045] Jfa =SmoothL1(φ(x+δ),φ(x))-SmoothL1(ψ(x+δ),ψ(x))

[0046] The specific formulas in steps (2-8) are as follows:

[0047] J total =J fa +J sa

[0048]

[0049] Where lr represents the learning rate, and its value is 0.03.

[0050] Preferably, the cropping process in step (2-9) is as follows: for each pixel in the optimized perturbation obtained in step (2-8), if its pixel value is greater than ε, then the pixel value is set to ε; if its pixel value is less than -ε, then the pixel value is set to -ε; if its pixel value is between [-ε, ε], then the pixel value is kept unchanged. The value range of ε is 0 to 32 / 255.

[0051] Step (2-11) involves cropping the size of the adversarial sample to a pixel size between [0,1].

[0052] Preferably, the detection results include indicators such as mAP50, mAP75, IW-SSIM, and TV, which are used to evaluate the robustness of the target detector under test.

[0053] According to another aspect of the present invention, a general evaluation system for target detectors is provided, comprising:

[0054] The first module is used to acquire image x, input the acquired image x into the target detector to be evaluated to obtain multiple candidate boxes, and select k bounding boxes from all candidate boxes from the perspectives of classification and regression respectively, which are used as attack targets from the regression perspective and the classification perspective respectively.

[0055] The second module is used to calculate the spatial loss function based on the attack target from the regression angle and the attack target from the classification angle obtained from the first module, and to calculate the frequency loss function based on the image x. The perturbation δ is iteratively trained using the obtained spatial loss function and frequency loss function until the maximum number of iterations is reached, thereby obtaining adversarial examples. These adversarial examples are then input into the target detector to be evaluated to obtain the detection results of the target detector to be evaluated.

[0056] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects:

[0057] 1. Since the present invention uses steps (1-1) to (1-3) to select candidate boxes as attack targets in the output before the post-processing stage of the target detector, the evaluation method of the present invention does not depend on specific components of the target detector and realizes a general evaluation framework.

[0058] 2. Because the present invention uses steps (1-4) to (1-5), the top k high-quality bounding boxes for each real target are selected as attack targets from both classification and regression perspectives, ensuring that each target in the image is attacked uniformly, thereby enabling a comprehensive and accurate evaluation of the robustness of the target detector.

[0059] 3. Because the present invention employs steps (2-1) to (2-12), it uses directional induction in the spatial domain to move the prediction box to change its size and position, and designs a foreground-background separation attack to make the classification label of the prediction box close to the background; in the frequency domain, it applies the discrete wavelet transform method to decompose the image into high-frequency and low-frequency components, and concentrates the noise in the high-frequency region representing the semantics, thereby avoiding ineffective attacks on non-critical objects and improving the evaluation efficiency and accuracy.

[0060] 4. Because the present invention employs steps (2-9) and (2-11), the adversarial samples generated during the attack are highly concealed and not easily perceived by humans, thus improving the accuracy of the evaluation method. Attached Figure Description

[0061] Figure 1 This is a schematic diagram of the overall framework of the general evaluation method for target detectors of the present invention;

[0062] Figure 2 This is a flowchart of the dual-track attack target selection strategy in this invention. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0064] The basic idea of ​​this invention is to allocate a fixed number of candidate boxes as attack targets for each real object from the output of the object detector. On the one hand, in the spatial domain, the position, size, and label of the predicted boxes are changed through candidate box coordinate zeroing attacks and foreground-background separation attacks. On the other hand, in the frequency domain, the high-frequency information of the image is focused, concentrating noise in the high-frequency regions representing semantics. This improves the attack effect and the stealth of adversarial examples from two angles, ensuring the accuracy of the evaluation method.

[0065] Specifically, unlike existing methods that utilize certain components of the object detector, this invention considers only the output of the object detector before the post-processing stage in order to achieve a general evaluation method. However, the large number of candidate boxes generated by the object detector also introduces ambiguity regarding the attack target, resulting in unnecessary computational overhead. Given that object detection models typically perform joint optimization for classification and regression tasks, this invention proposes a dual-track attack target selection strategy, selecting high-quality candidate boxes as attack targets from the output before the post-processing stage. Then, directional perturbations are employed to move and compress the predicted boxes and alter the classification results, misleading the object detector's output. Furthermore, this invention concentrates the perturbations in high-frequency regions of the image to interfere with the object detector's focus on key objects, thereby improving evaluation efficiency.

[0066] In the experiments of this invention, tests were conducted on eight commonly used target detectors and two datasets, revealing that the attack performance against the target detectors is very powerful, with a significant decrease in mAP scores. Therefore, the robustness and security of the target detectors can be comprehensively evaluated. This is because the method assigns a fixed number of attack targets to real targets from both classification and regression perspectives, designs spatial attacks to mislead the detector's output, and further improves the attack effect under perturbation constraints by incorporating frequency domain information.

[0067] like Figure 1 As shown, this invention provides a general evaluation method for target detectors, comprising the following steps:

[0068] (1) Obtain image x, and input the obtained image x into the target detector to be evaluated to obtain multiple candidate boxes, such as Figure 2 As shown, k bounding boxes are selected from all candidate boxes from both classification and regression perspectives, serving as attack targets from the regression perspective and the classification perspective, respectively.

[0069] This step includes the following steps:

[0070] (1-1) Obtain image x and input the obtained image x into the target detector to be evaluated to obtain multiple candidate boxes, as well as the position coordinates and class prediction score set of each candidate box. The class corresponding to the highest class prediction score in the class prediction score set is taken as the predicted class of the candidate box.

[0071] (1-2) For all candidate boxes obtained in step (1-1), calculate the intersection over union (IoU) score of each candidate box and its corresponding ground truth box, and arrange all candidate boxes in descending order of the obtained IoU scores.

[0072] (1-3) Based on the predicted categories of all candidate boxes obtained in step (1-1), select multiple candidate boxes whose predicted categories are completely consistent with the labels of the corresponding real target boxes from the sorting results obtained in step (1-2).

[0073] (1-4) Select the top k candidate boxes from the ranking results obtained in step (1-2) as the attack targets from the regression perspective.

[0074] Specifically, the value of k ranges from 5 to 50, with 15 being the preferred value.

[0075] (1-5) Select the top k candidate boxes from the multiple candidate boxes obtained in step (1-3) as the attack targets from the classification perspective.

[0076] After the above processing, 2k attack targets were obtained for each real target bounding box in the image.

[0077] The advantage of steps (1-1) to (1-5) above is that, from the output before the post-processing stage of the target detector, the top k high-quality candidate boxes for each real target are selected as attack targets from both classification and regression perspectives. This solves the problems that suboptimal bounding boxes cannot be effectively attacked and that the weights of each target are different in the attack, thus ensuring the universality and scientific nature of the evaluation method.

[0078] (2) Calculate the spatial loss function based on the attack target from the regression angle and the attack target from the classification angle obtained in step (1), calculate the frequency loss function based on the image x, and use the obtained spatial loss function and frequency loss function to iteratively train the perturbation δ until the maximum number of iterations is reached, thereby obtaining the adversarial sample, and input the adversarial sample into the target detector to be evaluated to obtain the detection result of the target detector to be evaluated.

[0079] This step includes the following steps:

[0080] (2-1) For each candidate box obtained in steps (1-4) and (1-5), move the coordinates of the top left and bottom right corners of the candidate box closer to the origin coordinates (0,0) to obtain the regression loss function J in the spatial domain corresponding to the candidate box. loc ;

[0081] Specifically, the regression loss function J in this step loc Using the Smooth L1 loss function, we have:

[0082]

[0083] in This represents the nth candidate box among all candidate boxes obtained in steps (1-4) and (1-5). Let n represent the nth predefined target bounding box with the coordinates of the top left and bottom right corners both being (0,0), and n∈[1,N], where N represents the total number of candidate boxes obtained in steps (1-4) and (1-5), and SmoothL1 represents the Smooth L1 loss function.

[0084] The advantage of this step is that it aligns the coordinates of the predicted bounding box with the coordinates of a predefined, meaningless target region. Considering that targets in an image are often located in the central region, this invention forces the coordinates of the top-left and bottom-right corners of the predicted bounding box to be close to edge points, thus misleading the position and size of the predicted bounding box.

[0085] (2-2) For each candidate box among all candidate boxes obtained in steps (1-4) and (1-5), the set of category prediction scores for the nth candidate box obtained in step (1-1) is used. Obtain the classification loss function J in the spatial domain corresponding to the nth candidate box. cls ,in This represents the category prediction score of the label of the ground truth bounding box corresponding to the nth candidate box, and K represents the total number of categories that the object detector to be evaluated can detect, as preset. This represents the category prediction score of the Kth category (i.e., background) corresponding to the nth candidate box;

[0086] Classification loss function J in the spatial domain cls The following formula is used:

[0087]

[0088] The advantage of this step is that by minimizing the score of the true label of the predicted box in the image while maximizing the score of the background class, a foreground-background separation attack is implemented, causing the target in the image to converge to the background, thus causing the detector to misclassify.

[0089] (2-3) For each candidate box among all candidate boxes obtained in steps (1-4) and (1-5), the regression loss function J corresponding to the nth candidate box obtained in step (2-1) is... loc The classification loss function J corresponding to the nth candidate box obtained in step (2-2) cls The values ​​are added proportionally to obtain the spatial loss function J corresponding to the nth candidate box. sa ;

[0090] Loss function J in the spatial domain sa The following formula is used:

[0091] Jsa =J loc +λJ cls

[0092] Where λ∈[0,1000], its preferred value is 100.

[0093] (2-4) The image x is decomposed into a low-frequency component x using the Discrete Wavelet Transform (DWT) method. ll A high-frequency component x hh and two mid-frequency components x lh and x hl ;

[0094] The specific formulas used in this step are as follows:

[0095] x ll =LxL T ,x hh =HxH T ,x lh =LxH T ,x hl =HxL T

[0096] Where L represents the coefficient matrix corresponding to the low-pass filter used in the DWT method, and H represents the coefficient matrix corresponding to the high-pass filter used in the DWT method.

[0097] (2-5) Use the Inverse Discrete Wavelet Transform (IDWT) method to process the low-frequency component x obtained in step (2-4). ll and high-frequency component x hh Perform signal reconstruction to obtain the reconstructed image;

[0098] The specific formulas used in this step are as follows:

[0099] φ(x)=L T x ll L = L T (LxL T )L

[0100] ψ(x)=H T x hh H = H T (HxH T )H

[0101] Where φ(x) represents the image after reconstruction of the low-frequency part, and ψ(x) represents the image after reconstruction of the high-frequency part.

[0102] (2-6) Obtain the adversarial sample x+δ after adding perturbation δ, and process the adversarial sample x+δ after adding perturbation δ using the formulas in steps (2-4) and (2-5) above, so as to obtain the adversarial sample φ(x+δ) after low-frequency reconstruction and the adversarial sample ψ(x+δ) after high-frequency reconstruction respectively.

[0103] (2-7) Obtain the frequency domain loss function J based on the reconstructed low-frequency adversarial sample φ(x+δ), the reconstructed low-frequency image φ(x), the reconstructed high-frequency adversarial sample ψ(x+δ), and the reconstructed high-frequency image ψ(x) obtained in steps (2-5) and (2-6). fa ;

[0104] The specific formulas used in this step are as follows:

[0105] J fa =SmoothL1(φ(x+δ),φ(x))-SmoothL1(ψ(x+δ),ψ(x))

[0106] The advantage of this step is that it amplifies the differences in high-frequency components between adversarial examples and normal images, while constraining the differences in low-frequency components. This directs a larger proportion of perturbations to the high-frequency region, thereby improving the attack performance and stealth of the adversarial example.

[0107] (2-8) Add the spatial domain loss function obtained in step (2-3) and the frequency domain loss function obtained in step (2-7) to obtain the total loss function J. total And according to the total loss function J total And the perturbation δ is optimized using gradient descent to obtain the optimized perturbation;

[0108] The specific formulas used in this step are as follows:

[0109] J total =J fa +J sa

[0110]

[0111] Where lr represents the learning rate, and its value is 0.03.

[0112] (2-9) Trim the optimized perturbation obtained in step (2-8) to obtain the trimmed perturbation;

[0113] Specifically, the cropping process in this step is as follows: for each pixel in the optimized perturbation obtained in step (2-8), if its pixel value is greater than ε, then the pixel value is set to ε; if its pixel value is less than -ε, then the pixel value is set to -ε; if its pixel value is between [-ε, ε], then the pixel value is kept unchanged. The value range of ε is 0 to 32 / 255, preferably 8 / 255.

[0114] The advantage of this step is that it ensures the concealment of the disturbance.

[0115] (2-10) Use the loss function J from step (2-8) total The cropped perturbation obtained in step (2-9) is iteratively optimized until the maximum number of iterations (epochs) is reached, thus obtaining the final optimized perturbation. The final optimized perturbation is then combined with the image x to form an adversarial example.

[0116] Specifically, the maximum number of iterations (epoch) in this invention ranges from 1 to 150, preferably 50.

[0117] (2-11) The adversarial samples obtained in step (2-10) are cropped to obtain cropped adversarial samples;

[0118] Specifically, this step involves cropping the size of the adversarial sample to a pixel size between [0,1].

[0119] The advantage of this step is that it increases the imperceptibility of adversarial examples.

[0120] (2-12) Input the cropped adversarial sample obtained in step (2-11) into the target detector to be evaluated for detection to obtain the detection result.

[0121] Specifically, the detection results in this invention include indicators such as mAP50, mAP75, IW-SSIM, and TV, which are used to evaluate the robustness of the target detector under test.

[0122] Experimental results

[0123] The experimental environment for this invention was as follows: Under the Ubuntu 20.04 operating system, the CPU was an Intel(R) Xeon(R) Silver4210R CPU @ 2.40GHz, the GPU consisted of two NVIDIA 3090 24GB GPUs, and the memory was 256GB. The algorithm of this invention was implemented using PyTorch programming. Specific settings were as follows: The upper limit of the adversarial perturbation ε was set to 8 / 255, the hyperparameter λ was set to 100, the number of training epochs was set to 50, the batch size was 1, and the learning rate was 0.03.

[0124] To demonstrate the versatility and effectiveness of the method presented in this invention, eight commonly used object detectors were selected: Faster R-CNN (FR), Cascade R-CNN (CR), Sparse R-CNN (SR), SABL Faster R-CNN (SFR), RepPoints (RP), TOOD, Deformable DETR (D.DETR), and YOLOv5. These eight detectors were tested on the MS-COCO and PASCAL VOC datasets, and the evaluation results are shown in Table 1 below.

[0125]

[0126]

[0127] Table 1

[0128] The results in Table 1 show that the proposed method effectively evaluates the vulnerabilities of object detectors on different datasets. It significantly reduces mAP scores at IoU thresholds of 50% and 75%, demonstrating the quality of the adversarial attack. Notably, models like Cascade R-CNN and RepPoints exhibit poor robustness, with their mAP values ​​dropping dramatically on the dataset. Therefore, the general evaluation method for detectors based on adversarial attacks proposed in this invention can comprehensively and accurately evaluate the robustness of object detectors. On one hand, it allocates a fixed number of candidate boxes as attack targets for each real object from the object detector's output, ensuring the applicability of the evaluation method to all detectors. On the other hand, it alters the position, size, and label of predicted boxes in the spatial domain through candidate box coordinate zeroing attacks and foreground-background separation attacks, and concentrates noise in the high-frequency region representing semantics in the frequency domain. This improved attack quality also ensures the accuracy of the evaluation method.

[0129] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A general evaluation method for target detectors, characterized in that, Includes the following steps: (1) Obtain image x, input the obtained image x into the target detector to be evaluated to obtain multiple candidate boxes, select k bounding boxes from all candidate boxes from the perspectives of classification and regression respectively, and use them as attack targets from the regression perspective and the classification perspective respectively; step (1) includes the following steps: (1-1) Obtain image x, input the obtained image x into the target detector to be evaluated to obtain multiple candidate boxes, as well as the position coordinates and class prediction score set of each candidate box, and take the class corresponding to the highest class prediction score in the class prediction score set as the predicted class of the candidate box; (1-2) For all candidate boxes obtained in step (1-1), calculate the intersection-union ratio (IoU) score between each candidate box and its corresponding ground truth box, and arrange all candidate boxes in descending order of the obtained IoU scores; (1-3) Based on the predicted categories of all candidate boxes obtained in step (1-1), select multiple candidate boxes whose predicted categories are completely consistent with the labels of the corresponding real target boxes from the sorting results obtained in step (1-2). (1-4) Select the top k candidate boxes from the sorting results obtained in step (1-2) as the attack targets from the regression angle; where the value of k ranges from 5 to 50. (1-5) Select the top k candidate boxes from the multiple candidate boxes obtained in step (1-3) as the attack targets from the classification perspective; (2) Calculate the spatial loss function based on the attack target from the regression angle and the attack target from the classification angle obtained in step (1), calculate the frequency loss function based on the image x, and use the obtained spatial loss function and frequency loss function to iteratively train the perturbation δ until the maximum number of iterations is reached, thereby obtaining the adversarial sample, and input the adversarial sample into the target detector to be evaluated to obtain the detection result of the target detector to be evaluated; step (2) includes the following steps: (2-1) For each candidate box obtained in steps (1-4) and (1-5), move the coordinates of the top-left corner and bottom-right corner of the candidate box closer to the origin coordinates (0,0) to obtain the regression loss function J in the spatial domain corresponding to the candidate box. loc ; (2-2) For each candidate box among all candidate boxes obtained in steps (1-4) and (1-5), the set of category prediction scores for the nth candidate box obtained in step (1-1) is used. Obtain the classification loss function J in the spatial domain corresponding to the nth candidate box. cls ,in This represents the category prediction score of the label of the ground truth bounding box corresponding to the nth candidate box, and K represents the total number of categories that the object detector to be evaluated can detect, as set in advance. This represents the category prediction score of the Kth category corresponding to the nth candidate box, where the Kth category is the background; (2-3) For each candidate box among all candidate boxes obtained in steps (1-4) and (1-5), the regression loss function J corresponding to the nth candidate box obtained in step (2-1) is... loc The classification loss function J corresponding to the nth candidate box obtained in step (2-2) cls The values ​​are added proportionally to obtain the spatial loss function J corresponding to the nth candidate box. sa ; (2-4) The image x is decomposed into a low-frequency component x using the Discrete Wavelet Transform (DWT) method. ll A high-frequency component x hh and two mid-frequency components x lh and x hl ; (2-5) Use the inverse discrete wavelet transform (IDWT) method to process the low-frequency component x obtained in step (2-4). ll and high-frequency component x hh Perform signal reconstruction to obtain the reconstructed image; (2-6) Obtain the adversarial sample x+δ after adding perturbation δ, and process the adversarial sample x+δ after adding perturbation δ using the formulas in steps (2-4) and (2-5) above, so as to obtain the adversarial sample φ(x+δ) after low-frequency reconstruction and the adversarial sample ψ(x+δ) after high-frequency reconstruction respectively. (2-7) Obtain the frequency domain loss function J based on the reconstructed low-frequency adversarial sample φ(x+δ), the reconstructed low-frequency image φ(x), the reconstructed high-frequency adversarial sample ψ(x+δ), and the reconstructed high-frequency image ψ(x) obtained in steps (2-5) and (2-6). fa ; (2-8) Add the spatial domain loss function obtained in step (2-3) and the frequency domain loss function obtained in step (2-7) to obtain the total loss function J. total And according to the total loss function J total And the perturbation δ is optimized using gradient descent to obtain the optimized perturbation; (2-9) Trim the optimized perturbation obtained in step (2-8) to obtain the trimmed perturbation; (2-10) Use the loss function J from step (2-8) total The cropped perturbation obtained in step (2-9) is iteratively optimized until the maximum number of iterations (epochs) is reached, thus obtaining the final optimized perturbation. The final optimized perturbation is then combined with the image x to form an adversarial example. (2-11) The adversarial samples obtained in step (2-10) are cropped to obtain cropped adversarial samples; (2-12) Input the cropped adversarial sample obtained in step (2-11) into the target detector to be evaluated for detection to obtain the detection result.

2. The general evaluation method for target detectors according to claim 1, characterized in that, The regression loss function J in step (2-1) loc Using the Smooth L1 loss function, we have: in This represents the nth candidate box among all candidate boxes obtained in steps (1-4) and (1-5). Let n represent the nth predefined target bounding box with the coordinates of the top left and bottom right corners both being (0,0), and n∈[1,N], where N represents the total number of candidate boxes obtained in steps (1-4) and (1-5), and SmoothL1 represents the Smooth L1 loss function.

3. The general evaluation method for target detectors according to claim 2, characterized in that, In step (2-2), the classification loss function J in the spatial domain cls The following formula is used: The loss function J in the spatial domain part of step (2-3) sa The following formula is used: J sa =J loc +λJ cls Where λ∈[0,1000].

4. The general evaluation method for target detectors according to claim 3, characterized in that, The specific formulas in steps (2-4) are as follows: x ll =LxL T ,x hh =HxH T ,x lh =LxH T ,x hl =HxL T Where L represents the coefficient matrix corresponding to the low-pass filter used in the DWT method, and H represents the coefficient matrix corresponding to the high-pass filter used in the DWT method. The specific formulas in steps (2-5) are as follows: φ(x)=L T x ll L=L T (LxW T )THE ψ(x)=H T x hh H=H T (HxH T )H Where φ(x) represents the image after reconstruction of the low-frequency part, and λ(x) represents the image after reconstruction of the high-frequency part.

5. The general evaluation method for target detectors according to claim 4, characterized in that, The specific formulas in steps (2-7) are as follows: J fa =SmoothL1(φ(x+δ),φ(x))-SmoothL1(ψ(x+δ),ψ(x)) The specific formula in step (2-8) is as follows: J total *J fa +J sa Where lr represents the learning rate, and its value is 0.

03.

6. The general evaluation method for target detectors according to claim 5, characterized in that, The cropping process in step (2-9) is as follows: for each pixel in the optimized perturbation obtained in step (2-8), if its pixel value is greater than ε, then the pixel value is set to ε; if its pixel value is less than -ε, then the pixel value is set to -ε; if its pixel value is between [-ε, ε], then the pixel value is kept unchanged. The value of ε ranges from 0 to 32 / 255. Step (2-11) involves cropping the size of the adversarial sample to a pixel size between [0,1].

7. The general evaluation method for target detectors according to claim 6, characterized in that, The test results include mAP50, mAP75, IW-SSIM, and TV metrics, which are used to evaluate the robustness of the target detector under test.

8. A system employing the general evaluation method for target detectors as described in claim 1, characterized in that, include: The first module is used to acquire image x, input the acquired image x into the target detector to be evaluated to obtain multiple candidate boxes, and select k bounding boxes from all candidate boxes from the perspectives of classification and regression respectively, which are used as attack targets from the regression perspective and the classification perspective respectively. The second module is used to calculate the spatial loss function based on the attack target from the regression angle and the attack target from the classification angle obtained from the first module, and to calculate the frequency loss function based on the image x. The perturbation δ is iteratively trained using the obtained spatial loss function and frequency loss function until the maximum number of iterations is reached, thereby obtaining adversarial examples. These adversarial examples are then input into the target detector to be evaluated to obtain the detection results of the target detector to be evaluated.

Images