Degradation-first and super-division-second universal type confrontation defense target detection method and device

Abstract

The invention discloses a first-degradation and second-super-resolution general type confrontation and defense target detection method and device, and the method comprises the steps: carrying out the degradation processing of a confrontation image, and obtaining a degraded image; constructing a super-resolution network, and performing fine tuning on the super-resolution network; inputting the degraded image into the super-resolution network for super-resolution processing to obtain a super-resolution image; and inputting the super-resolution image into a target detection model to obtain target positioning and target classification. The target detection precision can be improved, and the method can be widely applied to the technical field of artificial intelligence.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a universal adversarial defense target detection method and device that first degrades and then super-resolutions. Background Art

[0002] Object detection is widely used in various visual tasks, including autonomous driving, tracking systems, defect detection, mechanical automation, and safety monitoring. With the booming development of deep neural networks (DNNs), DNN-based object detection algorithms have shined in visual tasks. Previous research has been devoted to improving the detection accuracy of deep models. However, despite the state-of-the-art capabilities of the models, they are vulnerable to adversarial attacks and are highly susceptible to adversarial perturbations, which cause incorrect object localization and recognition, greatly reducing the accuracy of the models. Summary of the invention

[0003] In view of this, the main purpose of the embodiments of the present invention is to provide a universal anti-defense target detection method and device of first degradation and then super-resolution, so as to improve the accuracy of target detection.

[0004] To achieve the above object, an embodiment of the present invention provides a universal anti-defense target detection method of first degradation and then super-resolution, the method comprising the following steps:

[0005] Degrading the adversarial image to obtain a degraded image;

[0006] Constructing a super-resolution network and fine-tuning the super-resolution network;

[0007] Inputting the degraded image into the super-resolution network for super-resolution processing to obtain a super-resolution image;

[0008] The super-resolution image is input into a target detection model to obtain the location and classification of the target.

[0009] In some embodiments, the step of performing degradation processing on the antagonistic image to obtain a degraded image comprises the following steps:

[0010] Performing Gaussian blur processing on the adversarial image to obtain an intermediate processed image;

[0011] Downsampling is performed on the intermediate processed image to obtain a degraded image.

[0012] In some embodiments, the super-resolution network includes a shallow feature extraction module, a deep feature extraction module and a high-quality image reconstruction module, and the step of inputting the degraded image into the super-resolution network for super-resolution processing to obtain a super-resolution image includes the following steps:

[0013] Inputting the degraded image into the super-resolution network;

[0014] Performing a shallow feature extraction operation on the degraded image through the shallow feature extraction module to obtain a first feature map;

[0015] Performing a deep feature extraction operation on the first feature map by the deep feature extraction module to obtain a target feature map;

[0016] An image reconstruction operation is performed on the target feature map through the high-quality image reconstruction module to obtain a super-resolution image.

[0017] In some embodiments, the deep feature extraction module includes a plurality of blocks;

[0018] Each of the blocks in the deep feature extraction module includes a first deep feature extraction submodule and a second deep feature extraction submodule;

[0019] The first deep feature extraction submodule includes a first branch and a second branch;

[0020] The first branch is used to perform a dilated convolution operation;

[0021] The second branch is used to perform a depth-wise separable convolution operation.

[0022] In some embodiments, performing a deep feature extraction operation on the first feature map by the deep feature extraction module to obtain a target feature map includes the following steps:

[0023] Inputting the first feature map into the current block in the deep feature extraction module, performing normalization processing in the layer normalization in the first deep feature extraction submodule of the current block to obtain a second feature map;

[0024] Inputting the second feature map into the first branch to obtain a third feature map;

[0025] Inputting the second feature map into the second branch to obtain a fourth feature map;

[0026] Performing a splicing operation on the third feature map and the fourth feature map to obtain a fifth feature map;

[0027] The fifth feature map is subjected to attention weighting processing, normalization processing, and nonlinear transformation processing in a channel dimension to obtain a sixth feature map.

[0028] In some embodiments, performing a deep feature extraction operation on the first feature map by the deep feature extraction module to obtain a target feature map further includes the following steps:

[0029] Inputting the sixth feature map into the layer normalization in the second deep feature extraction submodule for normalization processing to obtain a seventh feature map;

[0030] Inputting the seventh feature map into the third branch of the second deep feature extraction submodule to perform cross-window self-attention processing to obtain an eighth feature map;

[0031] Performing channel dimension attention weighting processing, normalization processing, and nonlinear transformation processing on the eighth feature map to obtain a ninth feature map;

[0032] The ninth feature map is input into the next block in the deep feature extraction module, and the step of returning to the layer normalization in the first deep feature extraction submodule of the current block for normalization processing is performed until the target feature map is output in the final block.

[0033] In some embodiments, fine-tuning the super-resolution network comprises the following steps:

[0034] Freezing parameters of the super-resolution network;

[0035] The second branch of the first deep feature extraction submodule and the third branch of the second deep feature extraction submodule are fine-tuned.

[0036] To achieve the above-mentioned purpose, an embodiment of the present invention further provides a general-purpose anti-defense target detection device of first degradation and then super-resolution, the device comprising:

[0037] The first module is used to perform degradation processing on the adversarial image to obtain a degraded image;

[0038] The second module is used to construct a super-resolution network and fine-tune the super-resolution network;

[0039] The third module is used to input the degraded image into the super-resolution network for super-resolution processing to obtain a super-resolution image;

[0040] The fourth module is used to input the super-resolution image into a target detection model to obtain the location and classification of the target.

[0041] To achieve the above-mentioned purpose, another aspect of an embodiment of the present invention provides an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the aforementioned general-purpose countermeasure defense target detection method of first degradation and then super-resolution.

[0042] To achieve the above-mentioned purpose, another aspect of an embodiment of the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned general-purpose countermeasure defense target detection method of first degradation and then super-resolution.

[0043] To achieve the above object, another aspect of an embodiment of the present invention provides a computer program product or a computer program, the computer program product or the computer program including computer instructions, the computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the aforementioned general-purpose countermeasure defense target detection method of first degradation and then super-resolution.

[0044] The embodiments of the present invention include at least the following beneficial effects: the present invention provides a universal adversarial defense target detection method and device of first degradation and then super-resolution, which degrades the adversarial image to obtain a degraded image, greatly reducing the effectiveness of the adversarial perturbation; constructs a super-resolution network, and fine-tunes the super-resolution network to further improve the defense performance against specific attack methods; inputs the degraded image into the super-resolution network for super-resolution processing to remove the adversarial perturbation, thereby greatly improving the accuracy of target detection and obtaining a super-resolution image; inputs the super-resolution image into a target detection model to obtain target positioning and target classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0046] Figure 1 It is a flow chart of a universal anti-defense target detection method of first degradation and then super-resolution provided by an embodiment of the present invention;

[0047] Figure 2 Schematic diagram of the overall framework of the De-SR method provided by an embodiment of the present invention;

[0048] Figure 3 is a schematic diagram of the structure of a super-resolution network provided by an embodiment of the present invention;

[0049] Figure 4 is a schematic diagram of the structure of a deep feature extraction module provided by an embodiment of the present invention;

[0050] Figure 5is a schematic diagram of specific implementation details of the Cross-Wi ndow module provided in an embodiment of the present invention;

[0051] Figure 6 It is a comparison diagram of adversarial training of the target detector provided by an embodiment of the present invention and fine-tuning of the CaSwinSR model by the De-SR method of the present invention;

[0052] Figure 7 is a schematic diagram of quantization results of a single image super-resolution benchmark dataset provided by an embodiment of the present invention;

[0053] Figure 8 is a schematic diagram of ablation experiment results of various components of the CaSwinSR model provided by an embodiment of the present invention;

[0054] Fig. 9 is a schematic diagram of mAP comparison of different adversarial attack methods for the SSD300 detector on the PASCAL VOC 2007 test set provided by an embodiment of the present invention;

[0055] Fig.10 is a schematic diagram of mAP comparison of different adversarial attack methods for the SSD300 detector on the MS-COCO 2017 validation set provided by an embodiment of the present invention;

[0056] Fig.11 is a schematic diagram of ablation experiment results of using different degradation processes provided by an embodiment of the present invention to resist adversarial attacks on classification results of an SSD300 detector;

[0057] Fig.12 It is a schematic diagram comparing the effects of different downsampling processes provided by an embodiment of the present invention on resisting adversarial attacks on classification results of the SSD300 detector;

[0058] Fig.13 It is a schematic diagram of comparing the target detection accuracy of the CaSwinSR method of the present invention and the existing SR method on the VOC2007 test data set after Bicubic downsampling and Gaussian blurring are applied to the data set provided by an embodiment of the present invention;

[0059] Fig.14 It is a schematic diagram of comparing the target detection accuracy of the CaSwinSR method of the present invention and the existing SR method on the COCO2017 verification data set after applying Bicubic downsampling and Gaussian blur to the data set provided by an embodiment of the present invention;

[0060] Fig.15 It is a schematic diagram of ablation experiment results of the De-SR framework provided in an embodiment of the present invention for defending against global adversarial perturbation attacks;

[0061] Fig.16 It is a schematic diagram comparing the detection results of the local patch attack on the I NRIA dataset by YOLOv2 provided by an embodiment of the present invention and the results of the defense using the De-SR of the present invention;

[0062] Fig.17 It is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present invention, and they are only examples of devices and methods consistent with some aspects of the embodiments of the present invention as detailed in the attached claims.

[0064] It should be noted that, although the functional modules are divided in the system schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described may be performed in a different order than the module division in the system or the flow chart. The terms "first / S100" and "second / S200" in the specification and claims and the above-mentioned drawings may be used to describe various concepts in this article, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiment of the present invention, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determination".

[0065] The terms "at least one", "multiple", "each", "any", etc. used in the present invention, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0066] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein are only for the purpose of describing the embodiments of the present invention and are not intended to limit the present invention.

[0067] In past studies, deep neural networks have achieved great success in object detection, but they are extremely vulnerable to adversarial perturbations carefully designed by attackers. Adversarial training (AT) has become an effective technique to improve the adversarial robustness of object detectors. However, adversarial training is computationally expensive and is only effective against specific adversarial attacks. It cannot be extended to defend against other types of adversarial attacks, and lacks versatility and generalization.

[0068] In view of this, if Figure 1 As shown, the embodiment of the present invention provides a universal anti-defense target detection method of first degradation and then super-resolution, which may include but is not limited to steps S100 to S400:

[0069] Step S100, performing degradation processing on the antagonistic image to obtain a degraded image;

[0070] Step S200, constructing a super-resolution network and fine-tuning the super-resolution network;

[0071] Step S300, inputting the degraded image into the super-resolution network for super-resolution processing to obtain a super-resolution image;

[0072] Step S400: input the super-resolution image into a target detection model to obtain target positioning and target classification.

[0073] In steps S100 to S400 of some embodiments, the attacked image is processed by the "degradation first, then super-resolution" (De-SR) method proposed in the present invention, and then the location and classification of the target can be accurately detected by the target detector. In the overall framework of the De-SR method proposed in the embodiment of the present invention, Figure 2 As shown in the figure, the black dotted line path indicates that no processing is performed on the attacked image (Attacked Image), and the target detector (OD Mode 1) fails to detect (Fai led Detection). The black solid line path indicates that the attacked image is processed using the "degradation first, then super resolution" adversarial defense method proposed in the embodiment of the present invention, and the target detector detects successfully (Successful Detection).

[0074] In some embodiments, step S100 may include but is not limited to steps S110 to S120:

[0075] Step S110, performing Gaussian blur processing on the adversarial image to obtain an intermediate processed image;

[0076] Step S120, down-sampling the intermediate processed image to obtain a degraded image.

[0077] In steps S110 to S120 of some embodiments, Figure 2 As shown, the attacked image is Gaussian blurred to destroy high-frequency disturbances to obtain an intermediate processed image, and then Bicubic downsampling is performed to remove the disturbed pixels in the attacked image to obtain a degraded image. Then, in the subsequent steps, the super-resolution network (SR Mode 1) constructed by the present invention is used to perform super-resolution processing on the degraded image to generate more details, thereby improving the accuracy of subsequent target detection. The above-mentioned "degradation first, then super-resolution" (De-SR) process is expressed as follows:

[0078]

[0079] Among them, X sr represents super-resolution image; X adv represents the adversarial image, i.e., the attacked image; Kernel represents the Gaussian blur kernel; SR(·) represents the super-resolution network, i.e., the SR model; Represents convolution operation;↓ s Represents Bicubic downsampling with a scale factor of s.

[0080] In some embodiments, Figure 3 As shown, the super-resolution network (CaSwinSR Mode l, SR Mode l) includes a shallow feature extraction module (Sha l low Feature Extraction), a deep feature extraction module (DeepFeature Extraction) and a high-quality image reconstruction module (HQ Image Reconstruction).

[0081] In some embodiments, Figure 3 As shown, the deep feature extraction module includes several blocks; Figure 4 As shown, each of the blocks in the deep feature extraction module includes a first deep feature extraction sub-module 501 and a second deep feature extraction sub-module 502; the first deep feature extraction sub-module includes a first branch and a second branch; wherein the first branch is a Cross-Window module, the purpose of which is to strengthen the information interaction between different windows in the W-MSA module, for performing a hole convolution operation; the second branch is a W-MSA module, for performing a depth-separable convolution operation.

[0082] In some embodiments, Figure 5As shown, for the dilated convolution of the first branch, the dilated rate is set to the window size divided in the W-MSA module (Dilated_rate=window_size), so that each element of the convolution kernel acts on each window, thereby realizing information exchange between different windows. For the depthwise separable convolution of the second branch, the size of the convolution kernel is set to the window size plus 1 (Kernel_size=window_size+1), so that when the depthwise separable convolution slides in the feature map, the information of the four windows can be considered at the same time. Finally, the outputs of the two branches are merged along the channel dimension, and the channel features are fused using a 1×1 convolution to obtain the final output. Among them, Figure 5 Each orange box represents a window divided in the W-MSA module, each red dotted box represents an element in the dilated convolution kernel, and the purple dotted box represents the depth convolution.

[0083] In some embodiments, step S300 may include but is not limited to steps S310 to S340:

[0084] Step S310, inputting the degraded image into the super-resolution network;

[0085] Step S320, performing a shallow feature extraction operation on the degraded image through the shallow feature extraction module to obtain a first feature map;

[0086] Step S330, performing a deep feature extraction operation on the first feature map through the deep feature extraction module to obtain a target feature map;

[0087] Step S340: Perform an image reconstruction operation on the target feature map through the high-quality image reconstruction module to obtain a super-resolution image.

[0088] In step S320 of some embodiments, Figure 3 As shown, the input degraded image is subjected to shallow feature extraction through a 3×3 convolution of a shallow feature extraction module to obtain a first feature map.

[0089] In some embodiments, step S330 may include but is not limited to steps S331 to S339:

[0090] Step S331, inputting the first feature map into the current block in the deep feature extraction module, performing normalization processing in the layer normalization in the first deep feature extraction submodule of the current block to obtain a second feature map;

[0091] Step S332, inputting the second feature map into the first branch to obtain a third feature map;

[0092] Step S333, inputting the second feature map into the second branch to obtain a fourth feature map;

[0093] Step S334, performing a splicing operation on the third feature map and the fourth feature map to obtain a fifth feature map;

[0094] Step S335, performing channel dimension attention weighting processing, normalization processing, and nonlinear transformation processing on the fifth feature map to obtain a sixth feature map.

[0095] Step S336, inputting the sixth feature map into the layer normalization in the second deep feature extraction submodule for normalization processing to obtain a seventh feature map;

[0096] Step S337, inputting the seventh feature map into the third branch of the second deep feature extraction submodule to perform cross-window self-attention processing to obtain an eighth feature map;

[0097] Step S338, performing attention weighting processing, normalization processing, and nonlinear transformation processing on the eighth feature map in the channel dimension to obtain a ninth feature map;

[0098] Step S339, input the ninth feature map into the next block in the deep feature extraction module, and return to the layer normalization step in the first deep feature extraction submodule of the current block for normalization processing until the final block outputs the target feature map.

[0099] In some embodiments, Figure 3 As shown in , deep feature extraction is performed through several blocks in the deep feature extraction module. Figure 4As shown, each block includes a first deep feature extraction submodule and a second deep feature extraction submodule. In the first deep feature extraction submodule, layer normalization (LayerNorm) is used to normalize the input feature map and pass it through two branches: in the first branch, namely the Cross-Window module, cross-window information interaction processing can be performed; in the second branch, namely the W-MSA module, self-attention processing can be performed within the window. Then, 1×1 convolution is used to splice the feature map on the channel, then the CA module is used to extract the features of the channel dimension, and finally, the LayerNorm and MLP modules are used to process the features. Since the third branch in the second deep feature extraction submodule, namely the SW-MSA module, can realize cross-window interaction, the Cross-Window module is not used in the second deep feature extraction submodule. After the feature map passes through the SW-MSA module, 1×1 convolution is also used to splice the feature map on the channel, then the CA module is used to extract the features of the channel dimension, and finally, the LayerNorm and MLP modules are used to process the features.

[0100] In step S340 of some embodiments, Figure 3 As shown in the figure, in the high-quality image reconstruction module, a 3×3 convolution is used to change the channel dimension, and then the Pixel Shuffle module is used to increase the feature map height and width to the target size, and the channel dimension is restored to 3. Finally, a 3×3 convolution is used to fine-tune the image.

[0101] In some embodiments, step S200 may include but is not limited to steps S210 to S220:

[0102] Step S210, freezing the parameters of the super-resolution network;

[0103] Step S220: fine-tune the second branch of the first deep feature extraction submodule and the third branch of the second deep feature extraction submodule.

[0104] like Figure 6As shown, unlike the traditional adversarial training (AT) method, the adversarial training method based on target detection first adds specific adversarial perturbations to the training data set, and then uses it for full-parameter adversarial training of the target detector, which enables the target detector to still detect successfully when subjected to a specific attack method. However, when facing unknown adversarial attack methods, the generalization performance of the target detector is poor, and adversarial training takes a long time. In view of this, while the De-SR method of an embodiment of the present invention can resist various unknown adversarial attack methods, it regards the target detector as a black box, and fine-tunes the SR model in the De-SR method through LoRA without performing full-parameter adversarial training on the target detector, thereby improving the resistance performance of the De-SR method to improve the detection accuracy against specific attack methods, which not only makes the De-SR method have better resistance performance against specific adversarial attack methods, but also greatly reduces the training cost.

[0105] For example, the parameters of the CaSwinSR model are first frozen, and then the W-MSA and SW-MSA modules in the CaSwinSR model are fine-tuned using LoRA instead of training all parameters, which can significantly improve the training efficiency and have strong generalization ability. The above optimization process can be expressed as follows:

[0106]

[0107] Where L1(·) represents the L1 loss function; θ sr represents the parameters of the super-resolution network; θ LoRA Indicates the parameters of LoRA; Val min Represents the minimum value matrix obtained by optimizing the parameters of LoRA.

[0108] In step S400 of some embodiments, the super-resolution image is input into the target detection model to obtain the location and classification of the target object. The process is expressed as follows:

[0109] loc,cls=OD(X sr )

[0110] Among them, OD(·) represents the target detector; loc represents the location of the target; and cls represents the classification of the target.

[0111] In some embodiments, the SR model (CaSwinSR) of the present invention is pre-trained to achieve good super-resolution effect by using degraded images after Gaussian blurring and Bicubic downsampling. Exemplarily, during training, a low-resolution (LR) image is input into the CaSwinSR model, where the LR image is obtained by applying the following degradation function to the GT image (Ground Truth image, i.e., the true value image):

[0112]

[0113] In the formula, X GT represents the GT image; X lr Represents a low-resolution image.

[0114] Taking the DF2K dataset (DIV2K & Flick2K) as an example, the CaSwinSR model is pre-trained and tested on five standard super-resolution benchmark datasets, namely Set14, Set5, BSD100, Urban100 and Manga109. Structural similarity (SSIM) and peak signal-to-noise ratio (PSNR) are used as quantitative performance indicators of the CaSwinSR model. The CaSwinSR model in the De-SR method uses the MS-COCO training dataset for LoRA fine-tuning, and the parameters of the CaSwinSR model need to be frozen during this process. The MS-COCO validation dataset and the PASCAL VOC test dataset are used for inference. The following experimental results and analysis are obtained:

[0115] like Figure 7 As shown, the best result is bolded, the second best result is underlined, and all super-resolution methods under the dashed line are tested using the cropped dataset. Figure 7 The PSNR and SSIM indicators of the CaSwinSR model in the present invention are better than other state-of-the-art blind super-resolution models. For example, compared with NLCUNet with a super-resolution ratio of ×2, the CaSwinSR model of the present invention improves by 0.01dB, 0.12dB, 0.12dB, 0.34dB and 0.19dB on the Set5, Set14, BSD100, Urban100 and Manga109 datasets, respectively. Compared with NLCUNet with a super-resolution ratio of ×4, the CaSwinSR model of the present invention improves by 0.03dB, 0.01dB, 0.06dB, 0.06dB and 0.28dB on the Set5, Set14, Urban100, BSD100 and Manga109 datasets, respectively. The experimental results fully demonstrate the effectiveness of the CaSwinSR model of the present invention.

[0116] In order to better simulate the real scene, Gaussian blur and bicubic downsampling were performed on the VOC2007 test dataset and COCO2017 validation dataset. Fig.13 , Fig.14 As shown in the figure, after being processed by the CaSwinSR model, the detection accuracy of both the single-stage and dual-stage detectors is better than the previous methods. From this, two conclusions can be drawn: 1) In practical scenarios, super-resolution helps to improve the accuracy of target detection. 2) As the performance of the CaSwinSR model improves, the accuracy of target detection will also be further improved.

[0117] like Figure 8 As shown, the quantitative results are from Set5, Urban100 and Manga109, and the super-resolution magnification is ×4. Figure 8 It can be seen from the first and second rows in that when the CA module is added to the W-MSA module, the model of the present invention can focus on the characteristics of the channel dimension, thereby achieving performance improvement. When the Cross-Window module proposed in the present invention is added, the W-MSA module realizes information interaction between different windows. At this time, the performance of the CaSwinSR model of the present invention is optimal. Therefore, it can be concluded that when the SR model can extract local features, global features and channel features at the same time, its performance is best.

[0118] like Fig.15 As shown in the figure, ↓2 and ↓4 represent the bicubic downsampling of ×0.5 and ×0.25 on the attacked image, respectively, and σ represents the variance of Gaussian blur. When attacked by LGP, the detection performance of Faster-RCNN and Var ifocal Net both drop significantly. However, when the attacked image is first bicubic downsampled by ×0.25 and then super-resolved, the detection accuracy of the detector is greatly improved; when Gaussian blur and bicubic downsampling are performed first and then super-resolved, the detection accuracy is further improved, and the best effect is achieved when the blur kernel size of Gaussian blur is 5, the variance is 2, and the bicubic downsampling ratio is ×0.5. Therefore, it can be concluded that Gaussian blur and downsampling can effectively destroy the perturbations in the attacked image, while the use of the CaSwinSR model can generate details that are more conducive to detection. In addition, it can be found that ×0.5 bicubic downsampling is better than ×0.25 downsampling, because ×0.5 downsampling retains more details, which is conducive to subsequent super-resolution reconstruction.

[0119] like Fig.16As shown in the figure, the first row is the detection result of YOLOv2 on a clean image. The second row of images (five images in PatchAttack) are the detection results of YOLOv2 after the patch attack. The detection box of the Person category disappears, and the patch is detected as the Teddy Bear category. The third row of images are images with the De-SR method applied. The Person category can be correctly detected, and the Teddy Bear category interference disappears. The visualization results show that the De-SR method of the present invention can resist such local patch attacks.

[0120] refer to Fig.15 , Fig.16 The experimental results show that the De-SR method of the present invention can resist various adversarial attacks on different detectors, which shows that the De-SR method is effective and universal.

[0121] like Fig. 9 As shown, the comparison between the method of the present invention and the most advanced method on the VOC2007 test data set clearly shows that when the SSD300 detector is subjected to adversarial attacks, its detection accuracy will drop significantly. When the De-SR method is used for adversarial defense, the detection accuracy of the SSD300 detector is significantly improved, close to the FROD method. When LoRA is used to fine-tune the SR model in De-SR, the detection accuracy of the SSD300 detector is the highest. It can be seen that for loc attacks, the De-SR method of the present invention plays a good role in adversarial defense. This is because the De-SR method uses L1 loss during training, which makes the CaSwinSR model have better super-resolution robustness when dealing with images affected by adversarial attacks. Therefore, when the De-SR method is used to fight loc attacks, the SSD300 detector performs best.

[0122] Fig.10 The figure shows the comparison between the proposed method and the most advanced method on the COCO2017 validation dataset. After being processed by the De-SR method of the proposed method, the detection accuracy of the SSD300 detector is significantly improved, close to the effect of FROD-DAT. After being processed by the De-SR method, the detection accuracy of the SSD300 detector on clean images is significantly better than that of FROD-DAT. When the CaSwinSR model is fine-tuned using LoRA, De-SR shows significant improvement in counteracting loc attacks, which is due to the use of L1 loss constraints during training.

[0123] refer to Fig. 9 and Fig.10Experiments show that by treating the target detector as a black box and fine-tuning the SR model with LoRA, it has good performance in resisting specific adversarial attacks. In addition, when using LoRA fine-tuning, the performance will be different when choosing different ranks.

[0124] like Fig.11 As shown in the figure, w / o means that no degradation is performed on the attacked image, ↓2 means that the attacked image has been Bicubic downsampled by ×0.5, and σ represents the variance of Gaussian blur. It can be clearly observed that the detection accuracy of the SSD300 detector is the worst when the attacked image is not preprocessed. When the attacked image is preprocessed with bicubic downsampling, the detection accuracy of the SSD300 detector is significantly improved. When Gaussian blur is used for preprocessing, the results show an interesting trend. As the blur kernel size gradually increases, the detection accuracy of the SSD300 detector shows an overall decreasing pattern. When the blur kernel size is set to 5, the detection performance is the best. Similarly, it can be found that the detection accuracy is best when the Gaussian blur σ is set to 2.

[0125] like Fig.12 As shown in the figure, Bi linear ↓2, nearest ↓2 and Bicubic ↓2 represent bilinear downsampling, nearest neighbor downsampling and bicubic downsampling with a magnification of × 0.5, respectively, and blur represents Gaussian blur. Fig.12 The effects of different downsampling methods on the accuracy of the SSD300 detector were compared in , and the results showed that the detector had the lowest detection accuracy when preprocessed with bilinear downsampling. In contrast, among all the downsampling methods tested, the detector had the highest detection accuracy after preprocessing with bicubic downsampling.

[0126] The embodiment of the present invention further provides a target detection device for general-purpose adversarial defense by first degrading and then super-resolution, which can implement the above-mentioned general-purpose adversarial defense target detection method by first degrading and then super-resolution, and the device includes:

[0127] The first module is used to perform degradation processing on the adversarial image to obtain a degraded image;

[0128] The second module is used to construct a super-resolution network and fine-tune the super-resolution network;

[0129] The third module is used to input the degraded image into the super-resolution network for super-resolution processing to obtain a super-resolution image;

[0130] The fourth module is used to input the super-resolution image into a target detection model to obtain the location and classification of the target.

[0131] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0132] An embodiment of the present invention further provides an electronic device, the electronic device comprising a processor and a memory, the memory storing a computer program, and the processor implementing the above-mentioned universal countermeasure defense target detection method of first degradation and then super-resolution when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0133] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0134] refer to Fig.17 , Fig.17 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0135] The processor 601 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided by the embodiments of the present invention.

[0136] The memory 602 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 602 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 602, and the processor 601 calls and executes a universal anti-defense target detection method of the embodiment of the present invention that first degrades and then super-resolutions;

[0137] Input / output interface 603, used to implement information input and output;

[0138] Communication interface 604, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WI FI, Bluetooth, etc.);

[0139] Bus 605 , which transmits information between various components of the device (e.g., processor 601 , memory 602 , input / output interface 603 , and communication interface 604 );

[0140] The processor 601 , the memory 602 , the input / output interface 603 and the communication interface 604 are connected to each other in communication within the device via a bus 605 .

[0141] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the computer program implements the above-mentioned universal anti-defense target detection method of first degradation and then super-resolution.

[0142] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0143] The embodiment of the present invention also provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the aforementioned general-purpose countermeasure defense target detection method of first degradation and then super-resolution.

[0144] In summary, the universal anti-defense target detection method and device of the embodiment of the present invention by first degradation and then super-resolution has the following advantages:

[0145] 1. The embodiment of the present invention proposes a general adversarial defense framework for target detection of "degradation first, then super-resolution" (De-SR). First, the adversarial image is Gaussian blurred, and then Bicubic downsampling is performed to obtain a degraded image, which greatly eliminates the effectiveness of the adversarial perturbation. In addition, by designing a new super-resolution (CaSwinSR) network, the degraded image is super-resolved to further remove the adversarial perturbation, thereby greatly improving the accuracy of target detection, especially the accuracy of small target detection. In order to further improve the defense performance against specific attack methods while ensuring low computational overhead, LoRA is also used to fine-tune the CaSwinSR model. A large number of experimental results show that the De-SR technology of the present invention is very effective in dealing with both global imperceptible and local patch-type adversarial perturbations.

[0146] 2. Regardless of the global visually invisible perturbation attack or the visible local patch attack, the embodiment of the present invention regards the target detector as a black box, first degrades the input image to obtain a degraded image, and then inputs the degraded image into the super-resolution network proposed by the present invention for super-resolution processing. For white-box attacks, adversarial samples are first generated for the attacked model, and then the super-resolution model is fine-tuned using LoRA, so that the super-resolution model can better reconstruct the attacked image, and ultimately improve the detection accuracy. Throughout the process, the embodiment of the present invention regards the target detector as a black box, so the method of the present invention is applicable to any different attack methods and different target detectors, and has a universal defense function.

[0147] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part for which is described as a larger operation is performed independently.

[0148] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified, one or more of the functions and / or features described may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present invention. More specifically, in view of the properties, functions, and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional skills of the engineer. Therefore, those skilled in the art can implement the present invention set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0149] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0150] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0151] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0152] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0153] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0154] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

[0155] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the described embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present invention.

Claims

1. A universal adversarial defense target detection method of first degradation and then super-resolution, characterized in that: The following steps are involved: Degrading the adversarial image to obtain a degraded image; Constructing a super-resolution network and fine-tuning the super-resolution network; Inputting the degraded image into the super-resolution network for super-resolution processing to obtain a super-resolution image; The super-resolution image is input into a target detection model to obtain the location and classification of the target.

2. The universal adversarial defense target detection method of first degradation and then super-resolution according to claim 1 is characterized in that: The step of performing degradation processing on the antagonistic image to obtain a degraded image comprises the following steps: Performing Gaussian blur processing on the adversarial image to obtain an intermediate processed image; Downsampling is performed on the intermediate processed image to obtain a degraded image.

3. The universal adversarial defense target detection method of first degradation and then super-resolution according to claim 1 is characterized in that: The super-resolution network includes a shallow feature extraction module, a deep feature extraction module and a high-quality image reconstruction module. The degraded image is input into the super-resolution network for super-resolution processing to obtain a super-resolution image, including the following steps: Inputting the degraded image into the super-resolution network; Performing a shallow feature extraction operation on the degraded image through the shallow feature extraction module to obtain a first feature map; Performing a deep feature extraction operation on the first feature map by the deep feature extraction module to obtain a target feature map; An image reconstruction operation is performed on the target feature map through the high-quality image reconstruction module to obtain a super-resolution image.

4. The universal adversarial defense target detection method of first degradation and then super-resolution according to claim 3 is characterized in that: The deep feature extraction module includes several blocks; Each of the blocks in the deep feature extraction module includes a first deep feature extraction submodule and a second deep feature extraction submodule; The first deep feature extraction submodule includes a first branch and a second branch; The first branch is used to perform a dilated convolution operation; The second branch is used to perform a depth-wise separable convolution operation.

5. The universal adversarial defense target detection method of first degradation and then super-resolution according to claim 4 is characterized in that: The step of performing a deep feature extraction operation on the first feature map by the deep feature extraction module to obtain a target feature map comprises the following steps: Inputting the first feature map into the current block in the deep feature extraction module, performing normalization processing in the layer normalization in the first deep feature extraction submodule of the current block to obtain a second feature map; Inputting the second feature map into the first branch to obtain a third feature map; Inputting the second feature map into the second branch to obtain a fourth feature map; Performing a splicing operation on the third feature map and the fourth feature map to obtain a fifth feature map; The fifth feature map is subjected to attention weighting processing, normalization processing, and nonlinear transformation processing in a channel dimension to obtain a sixth feature map.

6. The universal adversarial defense target detection method of first degradation and then super-resolution according to claim 5 is characterized in that: The deep feature extraction module performs a deep feature extraction operation on the first feature map to obtain a target feature map, and further comprises the following steps: Inputting the sixth feature map into the layer normalization in the second deep feature extraction submodule for normalization processing to obtain a seventh feature map; Inputting the seventh feature map into the third branch of the second deep feature extraction submodule to perform cross-window self-attention processing to obtain an eighth feature map; Performing channel dimension attention weighting processing, normalization processing, and nonlinear transformation processing on the eighth feature map to obtain a ninth feature map; The ninth feature map is input into the next block in the deep feature extraction module, and the step of returning to the layer normalization in the first deep feature extraction submodule of the current block for normalization processing is performed until the target feature map is output in the final block.

7. The universal adversarial defense target detection method of first degradation and then super-resolution according to claim 6 is characterized in that: The fine-tuning of the super-resolution network comprises the following steps: Freezing parameters of the super-resolution network; The second branch of the first deep feature extraction submodule and the third branch of the second deep feature extraction submodule are fine-tuned.

8. A universal anti-defense target detection device with degradation first and then super-resolution, characterized in that: include: The first module is used to perform degradation processing on the adversarial image to obtain a degraded image; The second module is used to construct a super-resolution network and fine-tune the super-resolution network; The third module is used to input the degraded image into the super-resolution network for super-resolution processing to obtain a super-resolution image; The fourth module is used to input the super-resolution image into a target detection model to obtain the location and classification of the target.

9. An electronic device, characterized in that: including a processor and a memory; The memory is used to store programs; The processor executes the program to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The storage medium stores a program, and the program is executed by a processor to implement the method according to any one of claims 1 to 7.

Images