General evaluation method and system for target detector

Abstract

The invention discloses a general evaluation method for a target detector, which comprises the following steps of: averagely distributing a fixed number of candidate frames as attack targets for each real object from the output of the target detector, on one hand, changing the position, the size and the label of a prediction frame through candidate frame coordinate zero-returning attack and foreground-background separation attack in an airspace; on the other hand, high-frequency information of the image is focused in the frequency domain, and noise is concentrated in a high-frequency area representing semantics. The attack effect and the concealment of the confrontation sample are improved from two aspects, and the accuracy of the evaluation method is ensured. According to the method, the technical problem that the existing disturbance-based attack method and patch-based attack method cannot evaluate all target detectors can be solved, and the technical problem that the existing disturbance-based attack method generates image-level global disturbance and cannot evaluate all target detectors can be solved. And unnecessary calculation cost can be generated by performing attack optimization on the non-key object, and the non-key object can be easily perceived by human beings.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision technology, and more specifically, relates to a general evaluation method and system for target detectors. Background Art

[0002] In the field of computer vision, object detectors have made significant progress in handling complex scenes and improving detection accuracy. Object detection involves classification and regression subtasks, which require simultaneous positioning and identification of objects, that is, providing bounding box locations and classification results. Existing object detectors are mainly divided into two-stage detectors and single-stage detectors; two-stage detectors (such as R-CNN, Faster R-CNN, and Cascade R-CNN, etc.) first generate candidate regions through RPN, and then accurately classify and regress these regions; while single-stage detectors (such as YOLO and TOOD, etc.) directly predict object categories and bounding box coordinates in the entire image in a single evaluation step.

[0003] As object detectors are increasingly used in key areas such as autonomous driving, intelligent monitoring, and face recognition, their security and robustness issues are becoming increasingly prominent. In practical applications, object detectors may be subject to various attacks, especially adversarial attacks, where attackers add carefully designed small perturbations to the input data to cause model output errors. Therefore, how to evaluate the robustness of object detectors has become an urgent problem to be solved.

[0004] To solve this problem, researchers have proposed a variety of attack methods to evaluate the robustness of target detectors. Existing attack methods can be divided into two categories: the first is a patch-based attack method in the physical world, which generates a perceptible patch that is independent of the image and then places this patch anywhere in the image to deceive the target detector; the second is a perturbation-based attack method in the digital world, which deceives the target detector by adding subtle perturbations to the image that are imperceptible to the human eye. These perturbations can be noise, pixel-level modifications, or other types of image transformations.

[0005] However, both of the above attack methods have some defects that cannot be ignored: first, both methods rely on specific modules of the target detector, which greatly limits the scalability of the attack. It is not a universal evaluation framework and therefore cannot evaluate all target detectors. Second, the perturbation-based attack method produces global perturbations at the image level. Optimizing attacks on non-critical objects (such as backgrounds) will incur unnecessary computational costs and are easily perceived by humans. At the same time, attacking meaningful targets and irrelevant background elements may lead to inaccurate evaluation results. Summary of the invention

[0006] In response to the above defects or improvement needs of the prior art, the present invention provides a general evaluation method for target detectors, which aims to solve the technical problem that the existing perturbation-based attack methods and patch-based attack methods cannot evaluate all target detectors, and the technical problem that the existing perturbation-based attack methods generate image-level global perturbations, which will generate unnecessary computational costs and be easily perceived by humans by attacking non-critical objects, and the technical problem that attacking meaningful targets and irrelevant background elements will lead to inaccurate evaluation results; the adversarial attack relied on by the present invention directly utilizes the candidate boxes output by the target detector before the post-processing stage, without relying on any of its internal structures for attack evaluation.

[0007] To achieve the above object, according to one aspect of the present invention, a general evaluation method for target detectors is provided, comprising the following steps:

[0008] (1) Obtain an image x and input the image x into the target detector to be evaluated to obtain multiple candidate boxes. Select k bounding boxes from all the candidate boxes from the perspectives of classification and regression, respectively, as attack targets from the regression perspective and attack targets from the classification perspective.

[0009] (2) Calculate the loss function of the spatial domain part according to the attack target of the regression angle and the attack target of the classification angle obtained in step (1), calculate the loss function of the frequency domain part according to the image x, and iteratively train the perturbation δ using the obtained loss function of the spatial domain part and the loss function of the frequency domain part until the maximum number of iterations is reached, thereby obtaining an adversarial sample, and input the adversarial sample into the target detector to be evaluated to obtain the detection result of the target detector to be evaluated.

[0010] Preferably, step (1) comprises the following steps:

[0011] (1-1) Obtain an image x, and input the obtained image x into the target detector to be evaluated to obtain multiple candidate boxes, as well as the position coordinates and category prediction score set of each candidate box, and take the category corresponding to the highest category prediction score in the category prediction score set as the predicted category of the candidate box.

[0012] (1-2) For all candidate boxes obtained in step (1-1), calculate the intersection over union (IoU) score of each candidate box and its corresponding true target box, and arrange all candidate boxes in descending order according to the obtained IoU scores.

[0013] (1-3) According to the predicted categories of all candidate frames obtained in step (1-1), multiple candidate frames whose predicted categories are completely consistent with the labels of the corresponding true target frames are selected from the arrangement results obtained in step (1-2);

[0014] (1-4) Select the top k candidate boxes from the arrangement results obtained in step (1-2) as the attack targets of the regression angle; where the value range of k is 5 to 50, preferably 15.

[0015] (1-5) Select the top k candidate frames from the multiple candidate frames obtained in step (1-3) as attack targets from the classification perspective.

[0016] Preferably, step (2) comprises the following steps:

[0017] (2-1) For each candidate box among all the candidate boxes obtained in steps (1-4) and (1-5), the coordinates of the upper left corner and the lower right corner of the candidate box are moved closer to the origin coordinate (0,0) to obtain the regression loss function J in the spatial domain corresponding to the candidate box loc ;

[0018] (2-2) For each candidate box among all the candidate boxes obtained in steps (1-4) and (1-5), the set of category prediction scores of the nth candidate box obtained in step (1-1) is Get the classification loss function J in the airspace corresponding to the nth candidate box cls ,in represents the category prediction score of the label of the real target box corresponding to the nth candidate box, K represents the total number of categories that can be detected by the pre-set target detector to be evaluated, Represents the category prediction score of the Kth category (i.e. background) corresponding to the nth candidate box;

[0019] (2-3) For each candidate box among all the candidate boxes obtained in steps (1-4) and (1-5), the regression loss function J corresponding to the nth candidate box obtained in step (2-1) is loc And the classification loss function J corresponding to the nth candidate box obtained in step (2-2) cls Add them in proportion to get the loss function J of the spatial part corresponding to the nth candidate box sa ;

[0020] (2-4) Use the discrete wavelet transform (DWT) method to decompose the image x into a low-frequency component x ll , a high frequency component x hh and two intermediate frequency components x lh and x hl ;

[0021] (2-5) Use the inverse discrete wavelet transform IDWT method to transform the low-frequency component x obtained in step (2-4) lland high frequency components x hh Perform signal reconstruction to obtain a reconstructed image;

[0022] (2-6) Obtain the adversarial sample x+δ after adding the perturbation δ, and use the formulas of the above steps (2-4) and (2-5) to process the adversarial sample x+δ after adding the perturbation δ to obtain the adversarial sample φ(x+δ) after the low-frequency part is reconstructed and the adversarial sample ψ(x+δ) after the high-frequency part is reconstructed;

[0023] (2-7) Obtain the frequency domain loss function J according to the adversarial sample φ(x+δ) reconstructed from the low-frequency part, the image φ(x) reconstructed from the low-frequency part, the adversarial sample ψ(x+δ) reconstructed from the high-frequency part, and the image ψ(x) reconstructed from the high-frequency part obtained in steps (2-5) and (2-6) fa ;

[0024] (2-8) Add the spatial domain loss function obtained in step (2-3) and the frequency domain loss function obtained in step (2-7) to obtain the total loss function J total , and according to the total loss function J total , and use gradient descent to optimize the perturbation δ to obtain the optimized perturbation;

[0025] (2-9) trimming the optimized disturbance obtained in step (2-8) to obtain a trimmed disturbance;

[0026] (2-10) Use the loss function J in step (2-8) total The cropped perturbation obtained in step (2-9) is iteratively optimized until the maximum number of iterations epoch is reached, thereby obtaining the final optimized perturbation, and combining the final optimized perturbation with the image x as an adversarial sample.

[0027] (2-11) trimming the adversarial sample obtained in step (2-10) to obtain a trimmed adversarial sample;

[0028] (2-12) Input the cropped adversarial sample obtained in step (2-11) into the target detector to be tested to obtain the detection result.

[0029] Preferably, the regression loss function J in step (2-1) is loc The Smooth L1 loss function is used, and:

[0030]

[0031] in represents the nth candidate box among all the candidate boxes obtained in steps (1-4) and (1-5), represents the nth predefined target bounding box with the coordinates of the upper left corner and the lower right corner both being (0,0), and n∈[1,N], N represents the total number of candidate boxes obtained in steps (1-4) and (1-5), and SmoothL1 represents the Smooth L1 loss function.

[0032] Preferably, the classification loss function J in the airspace in step (2-2) is cls The following formula is used:

[0033]

[0034] The loss function J of the spatial domain in step (2-3) sa The following formula is used:

[0035] J sa =J loc +λJ cls

[0036] where λ∈[0,1000].

[0037] Preferably, the specific formula in step (2-4) is as follows:

[0038] x ll =LxLT,x hh =HxHT,x lh =LxH T ,x hl =HxL T

[0039] Where L represents the coefficient matrix corresponding to the low-pass filter used in the DWT method, and H represents the coefficient matrix corresponding to the high-pass filter used in the DWT method.

[0040] The specific formula in steps (2-5) is as follows:

[0041] φ(x)=L T x ll L=L T (LxL T )L

[0042] ψ(x)=H T x hh H=H T (HxH T )H

[0043] Where φ(x) represents the image reconstructed from the low-frequency part, and ψ(x) represents the image reconstructed from the high-frequency part.

[0044] Preferably, the specific formula in step (2-7) is as follows:

[0045] Jfa =SmoothL1(φ(x+δ),φ(x))-SmoothL1(ψ(x+δ),ψ(x))

[0046] The specific formula in steps (2-8) is as follows:

[0047] J total =J fa +J sa

[0048]

[0049] Where lr represents the learning rate, and its value is 0.03.

[0050] Preferably, the clipping process in step (2-9) is for each pixel in the optimized disturbance obtained in step (2-8), if its pixel value is greater than ε, then the pixel value is set to ε, if its pixel value is less than -ε, then the pixel value is set to -ε, if its pixel value is between [-ε,ε], then the pixel value is kept unchanged, and the value range of ε is 0 to 32 / 255;

[0051] Step (2-11) is to crop the size of the adversarial sample to a pixel size between [0, 1].

[0052] Preferably, the detection results include indicators such as mAP50, mAP75, IW-SSIM, TV, etc., which are used to evaluate the robustness of the target detector to be tested.

[0053] According to another aspect of the present invention, a general evaluation system for target detectors is provided, comprising:

[0054] The first module is used to obtain an image x, input the obtained image x into the target detector to be evaluated to obtain multiple candidate boxes, and select k bounding boxes from all the candidate boxes from the two perspectives of classification and regression, respectively, as the attack target from the regression perspective and the attack target from the classification perspective.

[0055] The second module is used to calculate the loss function of the spatial domain part according to the attack target of the regression angle and the attack target of the classification angle obtained in the first module, calculate the loss function of the frequency domain part according to the image x, and iteratively train the perturbation δ using the obtained loss function of the spatial domain part and the loss function of the frequency domain part until the maximum number of iterations is reached, thereby obtaining an adversarial sample, and inputting the adversarial sample into the target detector to be evaluated to obtain the detection result of the target detector to be evaluated.

[0056] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art:

[0057] 1. Since the present invention adopts steps (1-1) to (1-3) to select candidate boxes as attack targets in the output before the post-processing stage of the target detector, the evaluation method of the present invention does not depend on the specific components of the target detector, and realizes a universal evaluation framework;

[0058] 2. Since the present invention adopts steps (1-4) to (1-5), the first k high-quality bounding boxes are selected as attack targets for each real target from the perspectives of classification and regression, which ensures that each target in the image is attacked evenly, thereby being able to comprehensively and accurately evaluate the robustness of the target detector;

[0059] 3. Since the present invention adopts steps (2-1) to (2-12), directional induction is used in the spatial domain to move the prediction box to change its size and position, and a foreground-background separation attack is designed to make the classification label of the prediction box close to the background; the discrete wavelet transform method is applied in the frequency domain to decompose the image into high-frequency and low-frequency components, and the noise is concentrated in the high-frequency area representing the semantics, thereby avoiding invalid attacks on non-critical objects and improving the evaluation efficiency and accuracy;

[0060] 4. Since the present invention adopts step (2-9) and step (2-11), the adversarial samples generated in the attack are well concealed and not easily perceived by humans, thereby improving the accuracy of the evaluation method. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 It is a schematic diagram of the overall framework of the general evaluation method for target detectors of the present invention;

[0062] Figure 2 It is a flow chart of the dual-track attack target selection strategy in the present invention. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0064] The basic idea of ​​the present invention is to evenly assign a fixed number of candidate boxes as attack targets for each real object from the output of the target detector. On the one hand, the position, size and label of the predicted box are changed in the spatial domain through the candidate box coordinate zero attack and the foreground-background separation attack; on the other hand, the high-frequency information of the image is focused in the frequency domain, and the noise is concentrated in the high-frequency area representing the semantics. The attack effect and the concealment of the adversarial samples are improved from two perspectives to ensure the accuracy of the evaluation method.

[0065] Specifically, unlike existing methods that utilize certain specific components of the target detector, in order to achieve a general evaluation method, the present invention only considers the output results of the target detector before the post-processing stage. However, the large number of candidate boxes generated by the target detector also introduces ambiguity about the attack target, resulting in unnecessary computational overhead. Given that target detection models usually jointly optimize classification and regression tasks, the present invention proposes a dual-track attack target selection strategy to select high-quality candidate boxes from the output before the post-processing stage as attack targets. Directed perturbations are then used to move and compress the prediction boxes and change the classification results to mislead the output of the target detector. In addition, the present invention concentrates the perturbations on the high-frequency areas of the image to interfere with the target detector's focus on key objects, thereby improving the evaluation efficiency.

[0066] In the experiment of the present invention, by testing on eight commonly used target detectors and two data sets, it is found that the attack performance against the target detector is very powerful, and the mAP score is greatly reduced, so the robustness and security of the target detector can be comprehensively evaluated. The reason is that this method assigns a fixed number of attack targets to the real target from the perspectives of classification and regression, and designs spatial domain attacks to mislead the output of the detector, while combining frequency domain information to further improve the attack effect under perturbation constraints.

[0067] like Figure 1 As shown, the present invention provides a general evaluation method for target detectors, comprising the following steps:

[0068] (1) Obtain an image x and input the obtained image x into the target detector to be evaluated to obtain multiple candidate boxes, such as Figure 2 As shown in the figure, k bounding boxes are selected from all candidate boxes from the perspectives of classification and regression, respectively, as the attack targets from the regression perspective and the attack targets from the classification perspective;

[0069] This step includes the following steps:

[0070] (1-1) Obtain an image x, and input the obtained image x into the target detector to be evaluated to obtain multiple candidate boxes, as well as the position coordinates and category prediction score set of each candidate box, and take the category corresponding to the highest category prediction score in the category prediction score set as the predicted category of the candidate box.

[0071] (1-2) For all candidate boxes obtained in step (1-1), the intersection over union (IoU) score between each candidate box and its corresponding true target box is calculated, and all candidate boxes are arranged in descending order according to the obtained IoU scores.

[0072] (1-3) According to the predicted categories of all candidate frames obtained in step (1-1), multiple candidate frames whose predicted categories are completely consistent with the labels of the corresponding true target frames are selected from the arrangement results obtained in step (1-2);

[0073] (1-4) Select the top k candidate boxes from the permutation results obtained in step (1-2) as the attack targets of the regression angle.

[0074] Specifically, the value range of k is 5 to 50, preferably 15.

[0075] (1-5) Select the top k candidate frames from the multiple candidate frames obtained in step (1-3) as attack targets from the classification perspective.

[0076] After the above processing, 2k attack targets are obtained for each true target box on the image.

[0077] The advantage of the above steps (1-1) to (1-5) is that, from the output before the post-processing stage of the target detector, the top k high-quality candidate boxes are selected as attack targets for each real target from the perspectives of classification and regression, respectively, which solves the problem that suboptimal bounding boxes cannot be effectively attacked and that each target has different weights in the attack, thereby ensuring the universality and scientificity of the evaluation method.

[0078] (2) Calculate the loss function of the spatial domain part according to the attack target of the regression angle and the attack target of the classification angle obtained in step (1), calculate the loss function of the frequency domain part according to the image x, and iteratively train the perturbation δ using the obtained loss function of the spatial domain part and the loss function of the frequency domain part until the maximum number of iterations is reached, thereby obtaining an adversarial sample, and input the adversarial sample into the target detector to be evaluated to obtain the detection result of the target detector to be evaluated.

[0079] This step includes the following steps:

[0080] (2-1) For each candidate box among all the candidate boxes obtained in steps (1-4) and (1-5), the coordinates of the upper left corner and the lower right corner of the candidate box are moved closer to the origin coordinate (0,0) to obtain the regression loss function J in the spatial domain corresponding to the candidate box loc ;

[0081] Specifically, the regression loss function J in this step is loc The Smooth L1 loss function is used, and:

[0082]

[0083] in represents the nth candidate box among all the candidate boxes obtained in steps (1-4) and (1-5), represents the nth predefined target bounding box with the coordinates of the upper left corner and the lower right corner both being (0,0), and n∈[1,N], N represents the total number of candidate boxes obtained in steps (1-4) and (1-5), and SmoothL1 represents the Smooth L1 loss function.

[0084] The advantage of this step is that the coordinates of the prediction box are aligned with the coordinates of the predefined meaningless target area. Considering that the target in the image is often located in the central area, the present invention forces the coordinates of the upper left corner and the lower right corner of the prediction box to be close to the edge points, misleading the position and size of the prediction box.

[0085] (2-2) For each candidate box among all the candidate boxes obtained in steps (1-4) and (1-5), the set of category prediction scores of the nth candidate box obtained in step (1-1) is Get the classification loss function J in the airspace corresponding to the nth candidate box cls ,in represents the category prediction score of the label of the real target box corresponding to the nth candidate box, K represents the total number of categories that can be detected by the pre-set target detector to be evaluated, Represents the category prediction score of the Kth category (i.e. background) corresponding to the nth candidate box;

[0086] Classification loss function in the spatial domain J cls The following formula is used:

[0087]

[0088] The advantage of this step is that it implements a foreground-background separation attack by minimizing the score of the true label of the predicted box in the image while maximizing the score of the background class, so that the target in the image converges to the background, resulting in detector classification errors.

[0089] (2-3) For each candidate box among all the candidate boxes obtained in steps (1-4) and (1-5), the regression loss function J corresponding to the nth candidate box obtained in step (2-1) is loc And the classification loss function J corresponding to the nth candidate box obtained in step (2-2) cls Add them in proportion to get the loss function J of the spatial part corresponding to the nth candidate box sa ;

[0090] The loss function J in the spatial domain sa The following formula is used:

[0091] Jsa =J loc +λJ cls

[0092] Among them, λ∈[0,1000], and its preferred value is 100.

[0093] (2-4) Use the Discrete Wavelet Transform (DWT) method to decompose the image x into a low-frequency component x ll , a high frequency component x hh and two intermediate frequency components x lh and x hl ;

[0094] The specific formula in this step is as follows:

[0095] x ll =LxL T ,x hh =HxH T ,x lh =LxH T ,x hl =HxL T

[0096] Where L represents the coefficient matrix corresponding to the low-pass filter used in the DWT method, and H represents the coefficient matrix corresponding to the high-pass filter used in the DWT method.

[0097] (2-5) Use the Inverse Discrete Wavelet Transform (IDWT) method to transform the low-frequency component x obtained in step (2-4) ll and high frequency components x hh Perform signal reconstruction to obtain a reconstructed image;

[0098] The specific formula in this step is as follows:

[0099] φ(x)=L T x ll L=L T (LxL T )L

[0100] ψ(x)=H T x hh H=H T (HxH T )H

[0101] Where φ(x) represents the image reconstructed from the low-frequency part, and ψ(x) represents the image reconstructed from the high-frequency part.

[0102] (2-6) Obtain the adversarial sample x+δ after adding the perturbation δ, and use the formulas of the above steps (2-4) and (2-5) to process the adversarial sample x+δ after adding the perturbation δ to obtain the adversarial sample φ(x+δ) after the low-frequency part is reconstructed and the adversarial sample ψ(x+δ) after the high-frequency part is reconstructed;

[0103] (2-7) Obtain the frequency domain loss function J according to the adversarial sample φ(x+δ) reconstructed from the low-frequency part, the image φ(x) reconstructed from the low-frequency part, the adversarial sample ψ(x+δ) reconstructed from the high-frequency part, and the image ψ(x) reconstructed from the high-frequency part obtained in steps (2-5) and (2-6) fa ;

[0104] The specific formula in this step is as follows:

[0105] J fa =SmoothL1(φ(x+δ),φ(x))-SmoothL1(ψ(x+δ),ψ(x))

[0106] The advantage of this step is that it amplifies the difference in high-frequency components between adversarial samples and normal images, while constraining the difference in low-frequency components, and directing a larger proportion of disturbances to the high-frequency area, thereby improving the attack performance and concealment of adversarial samples.

[0107] (2-8) Add the spatial domain loss function obtained in step (2-3) and the frequency domain loss function obtained in step (2-7) to obtain the total loss function J total , and according to the total loss function J total , and use gradient descent to optimize the perturbation δ to obtain the optimized perturbation;

[0108] The specific formula in this step is as follows:

[0109] J total =J fa +J sa

[0110]

[0111] Where lr represents the learning rate, and its value is 0.03.

[0112] (2-9) trimming the optimized disturbance obtained in step (2-8) to obtain a trimmed disturbance;

[0113] Specifically, the clipping process in this step is for each pixel in the optimized perturbation obtained in step (2-8). If its pixel value is greater than ε, the pixel value is set to ε; if its pixel value is less than -ε, the pixel value is set to -ε; if its pixel value is between [-ε,ε], the pixel value is kept unchanged. The value range of ε is 0 to 32 / 255, preferably 8 / 255.

[0114] The advantage of this step is that the concealment of the disturbance is ensured.

[0115] (2-10) Use the loss function J in step (2-8) total The cropped perturbation obtained in step (2-9) is iteratively optimized until the maximum number of iterations epoch is reached, thereby obtaining the final optimized perturbation, and combining the final optimized perturbation with the image x as an adversarial sample.

[0116] Specifically, the maximum number of iterations epoch in the present invention ranges from 1 to 150, preferably 50.

[0117] (2-11) trimming the adversarial sample obtained in step (2-10) to obtain a trimmed adversarial sample;

[0118] Specifically, this step is to crop the size of the adversarial sample to a pixel size between [0, 1].

[0119] The advantage of this step is that it improves the imperceptibility of adversarial samples.

[0120] (2-12) Input the cropped adversarial sample obtained in step (2-11) into the target detector to be tested to obtain the detection result.

[0121] Specifically, the detection results in the present invention include indicators such as mAP50, mAP75, IW-SSIM, and TV, and the detection results are used to evaluate the robustness of the target detector to be evaluated.

[0122] Experimental Results

[0123] The experimental environment of the present invention is: under the Ubuntu 20.04 operating system, the CPU is Intel (R) Xeon (R) Silver 4210R CPU @ 2.40GHz, the GPU is 2 NVIDIA 3090 24GB, the memory is 256GB, and Pytorch programming is used to implement the algorithm of the present invention. The specific settings are as follows: the present invention sets the upper limit ε of the adversarial perturbation to 8 / 255, the hyperparameter λ to 100, the number of training rounds epoch to 50, the batch size to 1, and the learning rate to 0.03.

[0124] In order to illustrate the versatility and effectiveness of the method of the present invention, eight commonly used target detectors are selected: FasterR-CNN (FR), Cascade R-CNN (CR), Sparse R-CNN (SR), SABL Faster R-CNN (SFR), RepPoints (RP), TOOD, Deformable DETR (D.DETR) and YOLOv5. The eight commonly used target detectors are tested on the MS-COCO and PASCAL VOC datasets. The evaluation results are shown in Table 1 below:

[0125]

[0126]

[0127] Table 1

[0128] The results in Table 1 above show that the method proposed in the present invention can well evaluate the vulnerability of target detectors on different datasets, and can lead to a significant reduction in mAP scores when the IoU thresholds are 50% and 75%, which is sufficient to prove the quality of adversarial attacks. It is worth noting that models such as Cascade R-CNN and RepPoints show poor robustness, and their mAP values ​​drop significantly in the dataset. Therefore, the universal detector evaluation method based on adversarial attacks proposed in the present invention can comprehensively and accurately evaluate the robustness of target detectors. On the one hand, a fixed number of candidate boxes are evenly assigned to each real object from the output of the target detector as attack targets, which ensures the applicability of the evaluation method to all detectors; on the other hand, the position, size and label of the predicted box are changed in the spatial domain through the candidate box coordinate zero attack and the foreground-background separation attack, and the noise is concentrated in the high-frequency area representing the semantics in the frequency domain. The improvement of the attack quality also ensures the accuracy of the evaluation method.

[0129] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A general evaluation method for target detectors, characterized in that: The steps include: (1) Obtain an image x and input the image x into the target detector to be evaluated to obtain multiple candidate boxes. Select k bounding boxes from all the candidate boxes from the perspectives of classification and regression, respectively, as attack targets from the regression perspective and attack targets from the classification perspective. (2) Calculate the loss function of the spatial domain part according to the attack target of the regression angle and the attack target of the classification angle obtained in step (1), calculate the loss function of the frequency domain part according to the image x, and iteratively train the perturbation δ using the obtained loss function of the spatial domain part and the loss function of the frequency domain part until the maximum number of iterations is reached, thereby obtaining an adversarial sample, and input the adversarial sample into the target detector to be evaluated to obtain the detection result of the target detector to be evaluated.

2. The general evaluation method for target detectors according to claim 1, characterized in that: Step (1) comprises the following steps: (1-1) Obtain an image x, and input the obtained image x into the target detector to be evaluated to obtain multiple candidate boxes, as well as the position coordinates and category prediction score set of each candidate box, and take the category corresponding to the highest category prediction score in the category prediction score set as the predicted category of the candidate box. (1-2) For all candidate boxes obtained in step (1-1), calculate the intersection over union (IoU) score of each candidate box and its corresponding true target box, and arrange all candidate boxes in descending order according to the obtained IoU scores. (1-3) According to the predicted categories of all candidate frames obtained in step (1-1), multiple candidate frames whose predicted categories are completely consistent with the labels of the corresponding true target frames are selected from the arrangement results obtained in step (1-2); (1-4) Select the top k candidate boxes from the arrangement results obtained in step (1-2) as the attack targets of the regression angle; where the value range of k is 5 to 50, preferably 15. (1-5) Select the top k candidate frames from the multiple candidate frames obtained in step (1-3) as attack targets from the classification perspective.

3. The general evaluation method for target detectors according to claim 1 or 2, characterized in that: Step (2) comprises the following steps: (2-1) For each candidate box among all the candidate boxes obtained in steps (1-4) and (1-5), the coordinates of the upper left corner and the lower right corner of the candidate box are moved closer to the origin coordinate (0,0) to obtain the regression loss function J in the spatial domain corresponding to the candidate box loc ; (2-2) For each candidate box among all the candidate boxes obtained in steps (1-4) and (1-5), the category prediction score set of the nth candidate box obtained in step (1-1) is Get the classification loss function J in the airspace corresponding to the nth candidate box cls ,in represents the category prediction score of the label of the real target box corresponding to the nth candidate box, K represents the total number of categories that can be detected by the pre-set target detector to be evaluated, Represents the category prediction score of the Kth category (i.e. background) corresponding to the nth candidate box; (2-3) For each candidate box among all the candidate boxes obtained in steps (1-4) and (1-5), the regression loss function J corresponding to the nth candidate box obtained in step (2-1) is loc And the classification loss function J corresponding to the nth candidate box obtained in step (2-2) cls Add them in proportion to get the loss function J of the spatial part corresponding to the nth candidate box sa ; (2-4) Use the discrete wavelet transform (DWT) method to decompose the image x into a low-frequency component x ll , a high frequency component x hh and two intermediate frequency components x lh and x hl ; (2-5) Use the inverse discrete wavelet transform IDWT method to transform the low-frequency component x obtained in step (2-4) ll and high frequency components x hh Perform signal reconstruction to obtain a reconstructed image; (2-6) Obtain the adversarial sample x+δ after adding the perturbation δ, and use the formulas of the above steps (2-4) and (2-5) to process the adversarial sample x+δ after adding the perturbation δ to obtain the adversarial sample φ(x+δ) after the low-frequency part is reconstructed and the adversarial sample ψ(x+δ) after the high-frequency part is reconstructed; (2-7) Obtain the frequency domain loss function J according to the adversarial sample φ(x+δ) reconstructed from the low-frequency part, the image φ(x) reconstructed from the low-frequency part, the adversarial sample ψ(x+δ) reconstructed from the high-frequency part, and the image ψ(x) reconstructed from the high-frequency part obtained in steps (2-5) and (2-6) fa ; (2-8) Add the spatial domain loss function obtained in step (2-3) and the frequency domain loss function obtained in step (2-7) to obtain the total loss function J total , and according to the total loss function J total , and use gradient descent to optimize the perturbation δ to obtain the optimized perturbation; (2-9) trimming the optimized disturbance obtained in step (2-8) to obtain a trimmed disturbance; (2-10) Use the loss function J in step (2-8) total The cropped perturbation obtained in step (2-9) is iteratively optimized until the maximum number of iterations epoch is reached, thereby obtaining the final optimized perturbation, and combining the final optimized perturbation with the image x as an adversarial sample. (2-11) trimming the adversarial sample obtained in step (2-10) to obtain a trimmed adversarial sample; (2-12) Input the cropped adversarial sample obtained in step (2-11) into the target detector to be tested to obtain the detection result.

4. The general evaluation method for target detectors according to any one of claims 1 to 3, characterized in that: The regression loss function J in step (2-1) loc The Smooth L1 loss function is used, and: in represents the nth candidate box among all the candidate boxes obtained in steps (1-4) and (1-5), represents the nth predefined target bounding box with the coordinates of the upper left corner and the lower right corner both being (0, 0), and n∈[1, N], N represents the total number of candidate boxes obtained in steps (1-4) and (1-5), and SmoothL1 represents the Smooth-L1 loss function.

5. The general evaluation method for target detectors according to claim 4, characterized in that: The classification loss function J in the spatial domain in step (2-2) cls The following formula is used: The loss function J of the spatial domain in step (2-3) sa The following formula is used: J sa =J loc +λJ cls where λ∈[0, 1000].

6. The general evaluation method for target detectors according to claim 5, characterized in that: The specific formula in steps (2-4) is as follows: x ll =LxL T ,x hh =HxH T ,x lh =LxH T ,x hl =HxL T Where L represents the coefficient matrix corresponding to the low-pass filter used in the DWT method, and H represents the coefficient matrix corresponding to the high-pass filter used in the DWT method. The specific formula in steps (2-5) is as follows: φ(x)=L T x ll L=L T (LxW T )THE ψ(x)=H T x hh H=H T (HxH T )H Where φ(x) represents the image reconstructed from the low-frequency part, and ψ(x) represents the image reconstructed from the high-frequency part.

7. The general evaluation method for target detectors according to claim 6, characterized in that: The specific formula in steps (2-7) is as follows: J fa =SmoothL1(φ(x+δ),φ(x))-SmoothL1(ψ(x+δ),ψ(x)) The specific formula in steps (2-8) is as follows: I total =J fa +J sa Where lr represents the learning rate, and its value is 0.

03.

8. The general evaluation method for target detectors according to claim 7, characterized in that: The clipping process in step (2-9) is for each pixel in the optimized perturbation obtained in step (2-8). If its pixel value is greater than ε, the pixel value is set to ε; if its pixel value is less than -ε, the pixel value is set to -ε; if its pixel value is between [-ε,ε], the pixel value is kept unchanged. The value range of ε is 0 to 32 / 255. Step (2-11) is to crop the size of the adversarial sample to a pixel size between [0, 1].

9. The general evaluation method for target detectors according to claim 8, characterized in that: The detection results include mAP50, mAP75, IW-SSIM, TV and other indicators, which are used to evaluate the robustness of the target detector to be tested.

10. A general evaluation system for target detectors, characterized in that: include: The first module is used to obtain an image x, input the obtained image x into the target detector to be evaluated to obtain multiple candidate boxes, and select k bounding boxes from all the candidate boxes from the two perspectives of classification and regression, respectively, as the attack target from the regression perspective and the attack target from the classification perspective. The second module is used to calculate the loss function of the spatial domain part according to the attack target of the regression angle and the attack target of the classification angle obtained in the first module, calculate the loss function of the frequency domain part according to the image x, and iteratively train the perturbation δ using the obtained loss function of the spatial domain part and the loss function of the frequency domain part until the maximum number of iterations is reached, thereby obtaining an adversarial sample, and inputting the adversarial sample into the target detector to be evaluated to obtain the detection result of the target detector to be evaluated.

Images