Target detection countermeasure method

Abstract

The application discloses a target detection anti-defense method, comprising the following steps: a multi-scale feature extraction module is used to obtain a detection feature representation of a to-be-detected image; a double-path attention mechanism is constructed to generate attention signals from an original image and the detection feature, and the comprehensive perception ability for spatial position disturbance is improved; a dynamic prediction module is constructed to perform weighted calculation on a plurality of convolution kernel parameters under the guidance of a spatial attention map, and different convolution parameters are adaptively allocated for different spatial positions; a network optimization mechanism based on a multi-loss function is constructed, wherein a dense triplet loss is used to ensure the discrimination ability of the spatial attention map for the local regions of two kinds of samples, and a target detection loss enables the model to have the prediction ability for target positions and categories; a target detection anti-defense model is trained, the target detection model is optimized in an end-to-end mode by using the network optimization mechanism of the multi-loss function, and target detection anti-defense is realized based on the trained target detection model.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision and object detection, and particularly relates to an object detection adversarial defense method. BACKGROUND

[0002] Object detection is a basic computer vision task and is widely used in autonomous driving, video surveillance and other scenarios. In particular, with the advent of deep neural networks, object detection technology based on deep learning has made significant progress. However, due to the vulnerability of deep neural networks, adversarial samples formed by adding adversarial perturbations to normal samples can mislead deep learning models to output incorrect predictions, seriously threatening the prediction accuracy of object detection models. Research on high-robustness object detection adversarial defense methods to deal with adversarial sample attacks has received widespread attention from researchers.

[0003] In recent years, researchers have proposed many object detection adversarial defense methods. These methods usually use normal samples and adversarial samples to jointly supervise model training to improve the adversarial robustness of the model, thereby achieving accurate prediction on both types of samples. Zhang et al. proposed an adversarial defense method using multi-source attacks based on the multi-task characteristics of object detection. This method generates adversarial samples based on classification loss and localization loss, respectively, and uses adversarial samples to participate in training to improve the adversarial robustness of the model. Chen et al. proposed a class-aware adversarial defense method based on the multi-class prediction characteristics of the detection task. This method re-divides the joint classification loss and localization loss into each target class, generates adversarial samples based on the divided loss, and finally trains the model with adversarial samples and normal samples to improve the adversarial robustness of the model. Dong et al. guided dynamic convolution through an adversarial sample discrimination module to separate normal samples and adversarial samples, thereby alleviating the gradient entanglement problem in adversarial training of object detection models and significantly enhancing the adversarial robustness of the object detection model.

[0004] The above methods have made certain research progress in the field of object detection adversarial defense. However, the adversarial noise in different local regions of the adversarial sample often exhibits different perturbation strengths. Effectively exploiting this characteristic can help more comprehensively perceive the differences between the two types of samples, thereby improving the adversarial robustness of the object detection model. Existing methods only focus on the differences between adversarial samples and normal samples globally, ignoring the differences in perturbation strength between different local regions of the adversarial sample. SUMMARY

[0005] The present application provides an object detection adversarial defense method that generates a spatial attention map from an original image and detection features, enabling comprehensive perception of spatial position perturbations. Furthermore, under the guidance of the spatial attention map, adaptive allocation of convolution parameters is achieved to adapt to the problem of varying spatial position perturbation strengths, thereby improving the accuracy of object detection under adversarial attacks. Details are described below:

[0006] A target detection adversarial defense method, the method comprising:

[0007] A multi-scale feature extraction module is constructed, which is used to obtain a detection feature representation of a to-be-detected image;

[0008] A dual-path attention mechanism is constructed to generate attention signals from an original image and a detection feature, respectively, to improve the comprehensive perception ability of spatial position disturbance; the dual-path attention mechanism comprises an image attention generation unit and a feature attention generation unit;

[0009] A dynamic prediction module is constructed to perform weighted calculation on a plurality of convolution kernel parameters under the guidance of a spatial attention map, and to adaptively assign different convolution parameters to different spatial positions;

[0010] A network optimization mechanism based on a plurality of loss functions is constructed, wherein a dense triplet loss is used to ensure the discrimination ability of the spatial attention map for the local regions of two kinds of samples, and a target detection loss enables the model to have prediction ability for target positions and categories;

[0011] A target detection adversarial defense model is trained, the joint training stage simultaneously inputs an adversarial sample and a normal sample, and the target detection model is optimized end-to-end by the network optimization mechanism based on a plurality of loss functions, and the target detection adversarial defense is realized based on the trained target detection model.

[0012] The image attention generation unit generates an image spatial attention map ISA from the input image X, ISA = {ISA i}| i＝1,...,I The feature attention generation unit generates a feature spatial attention map FSA from the detection feature F, FSA = {FSA i}| i＝1,...,I .

[0013] Further, the image attention generation unit is composed of an image attention generation backbone network and a series of image attention generation subunits;

[0014] The image attention generation subunit is responsible for generating an image spatial attention map ISA from an image attention intermediate feature IAF, and each image attention generation subunit is composed of a convolution layer, a batch normalization layer and an activation layer.

[0015] The feature attention generation unit takes the detection feature F as input and comprises a series of feature attention generation subunits. The spatial attention map is:

[0016] SA = {u × ISA i +(1-u) × FSA i}|i＝1,...,I

[0017] wherein, denotes the i-th spatial attention map, h i x w i denotes the spatial size of the detected feature F i , s i denotes the number of channels, u [0, 1] denotes the weight of the image spatial attention map.

[0018] The dynamic prediction module is composed of a series of dynamic prediction units, each of which respectively contains a target category prediction unit and a target position prediction unit, and the target category prediction unit and the target position prediction unit respectively contain a local dynamic convolution, the local dynamic convolution has S convolution kernels, and the target category and position prediction are realized under the guidance of the spatial attention map SA, and the formula is as follows:

[0019]

[0020]

[0021] wherein, and denote the s-th convolution kernel of the local dynamic convolution of the target category prediction unit and the target position prediction unit of the i-th dynamic prediction unit respectively, Select s (·) denotes the s-th channel of the selected feature map, * denotes the convolution operation, denotes the n i target category confidence predicted, denotes the n i target position predicted.

[0022] The dense triplet loss respectively divides T pairs of triplets for each layer spatial attention map of a batch of bs training samples, respectively calculates the dense distance in each pair of triplets of each layer spatial attention map, and takes the average value of all dense distances as the dense triplet loss. The dense triplet loss is:

[0023]

[0024] wherein, L DT represents the dense triplet loss, represents the dense distance of the t-th pair of triplets of the i-th layer spatial attention map, and the specific calculation method is:

[0025]

[0026] wherein, represents the (w i , h icharacteristics of the position, γ represents a triplet distance interval, and JS(·) represents a JS divergence. and represent the same-class positive space attention map and the inter-class negative space attention map characteristics of the position, γ represents a triplet distance interval, and JS(·) represents a JS divergence.

[0027] The technical solution provided by the present application has the following beneficial effects:

[0028] 1. The present application constructs a double-path attention mechanism, which realizes comprehensive perception of disturbance intensity of different spatial positions by fusing the space attention maps from the original image and the detection features.

[0029] 2. The present application constructs a dynamic prediction module, which adaptively allocates different convolution parameters to different local regions under the guidance of the space attention signal, so as to adapt to the problem of different disturbance intensities of spatial positions.

[0030] 3. The present application designs a dense triplet loss, which improves the discrimination ability of the space attention map for local regions of the two kinds of samples, and constructs a multi-loss function mechanism in combination with the target detection loss, effectively constrains the learning process of the model, enhances the anti-attack defense capability of the target detection model, and improves the accuracy of target detection under the attack. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 It is a flowchart of a target detection anti-attack method. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the present application clearer, the embodiments of the present application are described in further detail below.

[0033] I. Constructing a multi-scale feature extraction module

[0034] The image to be detected X is in R h×w×c , where h x w represents the spatial size of the image, and c represents the channel number of the image. The multi-scale feature extraction module is used to obtain the detection feature representation F = {F i} i＝1,...,I , where I represents the number of multi-scale detection features. Different scale features have different sizes of receptive fields to adapt to targets of different sizes in the image. The multi-scale feature extraction module can be composed of the backbone network of a conventional target detection model.

[0035] II. Constructing a double-path attention mechanism

[0036] In order to realize the perception of the model to the disturbance intensity of the spatial position, the double-path attention mechanism is constructed in the embodiments of the present application. The double-path attention mechanism aims to generate a space attention map to guide the local dynamic convolution in the dynamic prediction module to complete target prediction.

[0037] The dual-attention mechanism generates attention signals from the original image and the detection feature respectively, improving the comprehensive perception ability of the model to spatial position disturbance. Specifically, the mechanism includes: an image attention generation unit and a feature attention generation unit, wherein the image attention generation unit generates a series of image spatial attention maps ISA = {ISA i}| i＝1,...,I from the input image X, and the feature attention generation unit generates a series of feature spatial attention maps FSA = {FSA i}| i＝1,...,I .

[0038] The image attention generation unit is composed of an image attention generation backbone network and a series of image attention generation subunits. The backbone network can be composed of any image classification network, taking the original image X as input, and obtaining a series of down-sampling features through successive down-sampling. The features with the same spatial size as the detection feature are selected from the down-sampling features to form the image attention intermediate feature IAF = {IAF i}| i＝1,...,I . The image attention generation subunit is responsible for generating the image spatial attention map ISA from IAF. Each image attention generation subunit is composed of a convolution layer, a batch normalization layer and an activation layer. Taking the i-th image attention generation subunit as an example, its input is IAF i , and the formula is as follows:

[0039] ISA i = δ (BN (W i * IAF i ))

[0040] where W i represents the convolution kernel parameters of the i-th image attention generation subunit, * represents the convolution operation, BN(·) represents the batch normalization operation, and δ(·) represents the activation function.

[0041] The feature attention generation unit directly takes the detection feature F as input, without the need for a separate feature extraction network, so FAG only contains a series of feature attention generation subunits, which have the same structure as the image attention generation subunit and generate FSA from the corresponding detection feature.

[0042] ISA and FSA are combined to obtain the final spatial attention map SA, and the formula is as follows:

[0043] SA = {u × ISA i + (1-u) × FSA i}| i＝1,...,I

[0044] where SA = {SAi}| i＝1,...,I represents the final spatial attention map, taking the i-th spatial attention map as an example, h i ×w i represents the spatial size of the detected feature F i . i represents the number of channels, which is the same as the number of convolution kernels of the local dynamic convolution that needs to be guided. u∈[0,1] represents the weight of the image spatial attention map, which is set to 0.7.

[0045] III. Constructing a dynamic prediction module

[0046] A dynamic prediction module is introduced after the dual-attention mechanism. The module performs weighted calculation on multiple convolution kernel parameters under the guidance of the spatial attention map, adaptively assigns different convolution parameters to different spatial positions, so as to adapt to the problem that the disturbance intensity of spatial positions is different, and then completes target prediction.

[0047] Specifically, the dynamic prediction module is composed of a series of dynamic prediction units uDP=uDP i | i＝1,...,I . Each dynamic prediction unit respectively contains a target class prediction unit uCP i and a target position prediction unit uLP i , which is responsible for predicting the class and position of the target from the detected feature. Taking the i-th dynamic prediction unit uDP i as an example, its target class prediction unit uCP i and target position prediction unit uLP i respectively contain a local dynamic convolution, which has S convolution kernels, and the number of convolution kernels is set to 4. Under the guidance of the spatial attention map SA i , the class and position of the target are predicted, and the formula is as follows:

[0048]

[0049]

[0050] wherein, and respectively represent the s-th convolution kernel of the local dynamic convolution of the target class prediction unit and the target position prediction unit, Select s (·) represents selecting the s-th channel of the feature map, * represents convolution operation, represents the n i target class confidence predicted, represents the n i target position predicted.

[0051] Each dynamic prediction unit performs target prediction separately, and all target prediction results are aggregated and then subjected to non-maximum suppression to obtain the final detection result.

[0052] IV. Constructing a multi-loss function mechanism

[0053] To enhance the adversarial robustness of the target detection model, this embodiment of the invention designs a network optimization mechanism based on multiple loss functions to improve the adversarial defense capability of target detection.

[0054] To ensure the spatial attention map's ability to distinguish local regions of the two types of samples, this embodiment of the invention constructs a dense triplet loss. First, the spatial attention maps of each layer of bs training samples in a batch are divided into T pairs of triplets, with bs and T set to 16 and 54, respectively. Dense distances are calculated within each pair of triplets in each layer of the spatial attention map. Finally, the average of all dense distances is used as the dense triplet loss, as shown in the following formula:

[0055]

[0056] Among them, L DT Represents the loss of dense triplet, The dense distance of the t-th pair of triples in the spatial attention map of the i-th layer is calculated as follows;

[0057]

[0058] in, (w) represents the spatial attention map of the anchor point i ,h i The characteristics of the location, and Representing the spatial attention graph of positive examples within the same class and the spatial attention graph of negative examples between classes. The location characteristics, where γ represents the triplet distance interval, are set to 0.6, [·] + The truncation function is defined with an output range of [0, +∞). JS(·) represents the JS divergence, used to measure the distance between two features. Taking two features P1 and P2 as an example, the specific values ​​are as follows:

[0059]

[0060] Subsequently, to enable the model to achieve target category and location awareness, this embodiment of the invention constructs a target detection loss, which includes target classification loss and target localization loss, as shown in the following formula:

[0061]

[0062] Among them, L conf (·) represents the Softmax loss, L loc(·) represents Smooth L1 loss, N represents the predicted target total number, a represents a target positioning loss weight, a is set to 1, and the Softmax loss and the Smooth L1 loss are both well known in the art and will not be described here.

[0063] The loss function used in the final training network is expressed as follows:

[0064] L = L DET + beta * L DT

[0065] Wherein, beta represents the weight of the dense triplet loss, and beta is set to 3.

[0066] Five, training of the target detection adversarial defense model

[0067] The network training proposed in the embodiments of the present application includes two stages of adversarial sample generation and joint training, and the two stages are alternately performed in the training. Specifically, in one iteration, adversarial samples are first generated, and then the target detection adversarial defense model is trained jointly with the adversarial samples and normal samples. In the adversarial sample generation stage, the adversarial samples are generated according to the normal samples and the target detection loss in the PGD strategy, and the PGD is well known in the art and will not be described here.

[0068] In the joint training stage, the adversarial samples and the normal samples are input simultaneously, and the target detection model is optimized end-to-end with a multi-loss function mechanism. The target detection model obtained by training can detect normal samples and adversarial samples simultaneously under the condition that the type of the input sample is unknown, and realizes the target detection adversarial defense. Further, the model can be applied to security monitoring, automatic driving and other safety-related fields, and provides reliable support for the safety guarantee of intelligent sensing technology.

[0069] Those skilled in the art can understand that the accompanying drawings are only schematic diagrams of a preferred embodiment, and the above-mentioned embodiment numbers of the present application are only for description, not representing the advantages and disadvantages of the embodiments.

[0070] The above-mentioned only for the preferred embodiments of the present application, and not to limit the present application, any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application, should be included in the protection scope of the present application.

Claims

1. A target detection adversarial defense method, characterized in that, The method includes: A multi-scale feature extraction module is constructed, which is used to obtain the detection feature representation of the image to be detected; A dual-path attention mechanism is constructed to generate attention signals from the original image and the detected features respectively, thereby improving the comprehensive perception capability of spatial position perturbations; the dual-path attention mechanism includes: an image attention generation unit and a feature attention generation unit; A dynamic prediction module is constructed, which performs weighted calculations on multiple convolutional kernel parameters under the guidance of spatial attention maps, and adaptively assigns different convolutional parameters to different spatial locations; A network optimization mechanism based on multiple loss functions is constructed, in which dense triplet loss is used to ensure the spatial attention map's ability to distinguish local regions of the two types of samples, while object detection loss enables the model to predict the target location and category. The target detection adversarial defense model is trained. During the joint training phase, adversarial samples and normal samples are input simultaneously. The target detection model is optimized end-to-end using a network optimization mechanism with multiple loss functions. The target detection adversarial defense is achieved based on the trained target detection model. The image attention generation unit generates information from the input image. Generate image spatial attention map ; The feature attention generation unit detects features. Generate a feature space attention map ; The image attention generation unit consists of an image attention generation backbone network and a series of image attention generation subunits; The image attention generation subunit is responsible for generating image attention intermediate features. Generate image spatial attention map Each image attention generation subunit consists of a convolutional layer, a batch normalization layer, and an activation layer; The feature attention generation unit is used to detect features. As input, it contains a series of feature attention generation subunits.

2. The target detection adversarial defense method according to claim 1, characterized in that, The spatial attention map is as follows: ； in, Indicates the first A spatial attention map, Indicates detection features Space dimensions, Indicates the number of channels. The weights represent the spatial attention map of the image.

3. The target detection adversarial defense method according to claim 1, characterized in that, The dynamic prediction module consists of a series of dynamic prediction units. Each dynamic prediction unit includes a target category prediction unit and a target location prediction unit. Each target category prediction unit and target location prediction unit includes a local dynamic convolution, which has the following characteristics: Each convolutional kernel, in the spatial attention map Guided by this method, the category and location of the target are predicted using the following formula: ； ； in, and They represent the first The local dynamic convolution of the target category prediction unit and the target location prediction unit of the dynamic prediction unit. One convolutional kernel, Indicates the selection of the first feature map aisle, This represents the convolution operation. Indicates the predicted Confidence of each target category Indicates the predicted Target locations.

4. The target detection adversarial defense method according to claim 1, characterized in that, The dense triplet loss affects a batch of... The spatial attention maps of each training sample are divided into layers. For each triplet, the dense distance is calculated within each pair of triples in each layer of the spatial attention map. The average of all dense distances is used as the dense triplet loss, which is: ； in, Represents the loss of dense triplet, Representing the The first layer of attention graph The specific calculation method for the dense distance of triples is as follows; ； in, Represents the spatial attention map of anchor points Location characteristics, and Representing the spatial attention graph of positive examples within the same class and the spatial attention graph of negative examples between classes. Location characteristics, Represents the distance interval between triples. Represents JS divergence.

Images