Backdoor detection and defense method and device of target detection network, medium and equipment

The LayerCAM algorithm generates significant graphs and counterfactual attribution significant graphs, combined with blocker and smooth filtering technology, solves the real-time and accuracy of backdoor attack detection in single-stage object detection networks in the physical world, and realizes real-time positioning and Efficient detection.

CN119068164APending Publication Date: 2024-12-03SHANGHAI FUDAN MICROELECTRONICS GROUP
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202310629284.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-30
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The prior art is difficult to detect and locate backdoor attacks in single-stage object detection networks in real-time in the physical world, and the computational complexity is high, making it unable to be suitable for real-time applications.

Method used

The LayerCAM algorithm is used to generate significant graphs or counterfactual attribution significant graphs to determine the size and position of the barrier, detect whether it is a backdoor trigger area by blocking the target, and improve detection accuracy through smooth filtering.

Benefits of technology

It realizes real-time detection and positioning of the backdoor trigger area in the physical world, reducing the computational complexity, and is suitable for backdoor trigger positioning of a single-stage object detection network, and improving the backdoor detection rate.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068164A_ABST
    Figure CN119068164A_ABST
Patent Text Reader

Abstract

The invention discloses a backdoor detection and defense method and device of a target detection network, a medium and equipment. The backdoor detection method comprises the following steps: inputting a to-be-processed image into a target detection network, and detecting to obtain position information and an initial classification result of a target existing in the to-be-processed image; for a first target detected in the to-be-processed image, based on a LayerCAM algorithm, obtaining a saliency map or an anti-fact attribution saliency map corresponding to the first target; determining the size and position of a barrier based on the saliency map or the anti-fact attribution saliency map corresponding to the first target; and blocking the first target by using the blocking device, and determining whether an area blocked by the blocking device is a rear door triggering area or not. The scheme can be applied to a backdoor detection scheme in an actual physical world scene, is used for coping with backdoor attacks in a single-stage target detection network, and can monitor possible backdoor triggering in real time and position the backdoor triggering position.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of target detection technology, and in particular to a backdoor detection and defense method, device, medium and equipment for a target detection network. Background Art

[0002] The task of object detection is to find all the targets (objects) of interest in an image and determine their categories and locations. It is one of the core issues in the field of computer vision.

[0003] With the rapid development of artificial intelligence technology, neural network models have been widely used in the field of target detection as target detection networks.

[0004] In practical applications, attackers can implant very hidden backdoors in neural network models. Such neural network models with implanted backdoors behave the same as clean networks on clean samples, but when poisonous samples with carefully crafted triggers are input into the network, the network will output the results specified by the attacker. Backdoor implantation can be achieved by simply adding a small number of poisonous samples to the data set during the training phase. Therefore, backdoor detection and defense of networks is an important and urgent topic.

[0005] The current backdoor detection targets the backdoors of image classification networks in the digital world, and assumes that the backdoor is triggered at a fixed position in the image, and the size and pixel value remain exactly the same, which is almost impossible to guarantee in the actual physical world. In the actual physical world, due to the camera's shooting angle, lighting and other issues, the position of the backdoor is generally random, and the size and pixel value will vary.

[0006] Although some backdoor detection methods can also be transferred to the physical world, the computational complexity of these algorithms is relatively high, resulting in long delays. They do not consider the computing power and delay issues that may be encountered when actually used in specific scenarios in the physical world, and are not suitable for real-time backdoor detection. There are also some methods that cannot be used for backdoor trigger positioning in single-stage target detection networks. Summary of the invention

[0007] The problem to be solved by the present invention is to provide a backdoor detection solution that can be applied in actual physical world scenarios to deal with backdoor attacks in single-stage target detection networks, monitor possible backdoor triggers in real time, and locate the position of the backdoor triggers.

[0008] To solve the above problems, an embodiment of the present invention provides a backdoor detection method for a target detection network, the method comprising: inputting an image to be processed into a target detection network, detecting and obtaining position information and initial classification results of targets in the image to be processed; the target detection network is a neural network; for a first target detected in the image to be processed, based on a LayerCAM algorithm, obtaining a saliency map or a counterfactual attribution saliency map corresponding to the first target; the first target is any target detected in the image to be processed; based on the saliency map or the counterfactual attribution saliency map corresponding to the first target, determining the size and position of a blocker; using the blocker to block the first target, determining whether the area blocked by the blocker is a backdoor triggering area.

[0009] Optionally, for a first target detected in the image to be processed, based on a LayerCAM algorithm, a saliency map corresponding to the first target is obtained, including: in a shallow layer of the target detection network, executing the LayerCAM algorithm on the first target to obtain a saliency map corresponding to the first target.

[0010] Optionally, for the first target detected in the image to be processed, based on the LayerCAM algorithm, a counterfactual attribution saliency map corresponding to the first target is obtained, including: in a shallow layer of the target detection network, executing the LayerCAM algorithm of counterfactual attribution on the first target to obtain a counterfactual attribution saliency map corresponding to the first target.

[0011] Optionally, based on the saliency map or counterfactual attribution saliency map corresponding to the first target, the size and position of the barrier are determined, including: taking the position of the maximum eigenvalue in the saliency map or counterfactual attribution saliency map as the center point position of the barrier, and using preset threshold constraint information to determine the size of the barrier.

[0012] Optionally, determining the size and position of the barrier based on the saliency map or counterfactual attribution saliency map corresponding to the first target includes: determining the pixel points whose eigenvalues ​​in the saliency map or counterfactual attribution saliency map corresponding to the first target are greater than a preset eigenvalue threshold, and obtaining a corresponding connectivity map; and using the size and position of the obtained connectivity map as the size and position of the barrier.

[0013] Optionally, using the blocker to block the first target and determining whether the area blocked by the blocker is a backdoor triggering area includes: performing smoothing filtering on the blocked image to obtain a smoothed filtered image; and determining whether the area blocked by the blocker is a backdoor triggering area based on the smoothed filtered image.

[0014] Optionally, determining whether the area blocked by the blocker is a backdoor triggering area based on the smoothed filtered image includes: inputting the smoothed filtered image into the target detection network to obtain a new classification result of the first target; comparing the initial classification result of the first target with the new classification result of the first target; when the initial classification result of the first target is inconsistent with the new classification result of the first target, determining that the area blocked by the blocker is a backdoor triggering area.

[0015] The embodiment of the present invention also provides a backdoor defense method for a target detection network, the method comprising:

[0016] Using any of the above-mentioned backdoor detection methods of the target detection network, backdoor detection is performed on part or all of the targets in the image to be processed; based on the backdoor detection results of the part or all of the targets, the erroneous prediction caused by the backdoor triggering in the image to be processed is corrected; and the corrected image is output.

[0017] Optionally, the outputting the corrected image includes: identifying a backdoor triggering area in the outputted corrected image.

[0018] An embodiment of the present invention also provides a backdoor detection device for a target detection network, the backdoor detection device comprising: a target detection unit, adapted to input an image to be processed into a target detection network, and detect and obtain position information and an initial classification result of a target existing in the image to be processed; the target detection network is a neural network; a saliency map acquisition unit, adapted to obtain a saliency map or a counterfactual attribution saliency map corresponding to a first target detected in the image to be processed based on a LayerCAM algorithm; the first target is any target detected in the image to be processed; a blocker determination unit, adapted to determine the size and position of a blocker based on the saliency map or the counterfactual attribution saliency map corresponding to the first target; and a backdoor detection unit, adapted to use the blocker to block the first target, and determine whether the area blocked by the blocker is a backdoor triggering area.

[0019] Optionally, the salient map acquisition unit is suitable for executing a LayerCAM algorithm on the first target in a shallow layer of the target detection network to obtain a salient map corresponding to the first target.

[0020] Optionally, the saliency map acquisition unit is adapted to execute a LayerCAM algorithm of counterfactual attribution on the first target in a shallow layer of the target detection network to obtain a counterfactual attribution saliency map corresponding to the first target.

[0021] Optionally, the barrier determination unit is adapted to use the maximum eigenvalue position in the saliency map or the counterfactual attribution saliency map as the center point position of the barrier, and determine the size of the barrier using preset threshold constraint information.

[0022] Optionally, the barrier determination unit is adapted to determine the pixel points whose characteristic values ​​in the saliency map or counterfactual attribution saliency map corresponding to the first target are greater than a preset characteristic value threshold, to obtain a corresponding connectivity map; and to use the size and position of the obtained connectivity map as the size and position of the barrier.

[0023] Optionally, the backdoor detection unit is adapted to perform smoothing filtering on the blocked image to obtain a smoothed filtered image; and determine whether the area blocked by the blocker is a backdoor triggering area based on the smoothed filtered image.

[0024] Optionally, the backdoor detection unit is suitable for inputting the smoothed filtered image into the target detection network to obtain a new classification result of the first target; comparing the initial classification result of the first target with the new classification result of the first target; and when the initial classification result of the first target is inconsistent with the new classification result of the first target, determining that the area blocked by the blocker is the backdoor triggering area.

[0025] An embodiment of the present invention also provides a backdoor defense device for a target detection network, the backdoor defense device comprising: any one of the above-mentioned backdoor detection devices for a target detection network, suitable for performing backdoor detection on part or all of the targets in a to-be-processed image; a correction unit, suitable for correcting the erroneous prediction caused by backdoor triggering in the to-be-processed image based on the backdoor detection results of the part or all of the targets; and an output unit, suitable for outputting a corrected image.

[0026] Optionally, the output unit is further adapted to identify a backdoor triggering area in the output corrected image.

[0027] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the steps of any of the above-mentioned backdoor detection methods for a target detection network.

[0028] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the steps of any of the above-mentioned backdoor defense methods for a target detection network.

[0029] An embodiment of the present invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor runs the computer program, the steps of any one of the above-mentioned backdoor detection methods for the target detection network are executed.

[0030] An embodiment of the present invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor runs the computer program, the steps of any of the above-mentioned backdoor defense methods for target detection networks are executed.

[0031] Compared with the prior art, the technical solution of the embodiment of the present invention has the following advantages:

[0032] By applying the scheme of the present invention, by performing target detection on the image to be processed, the target position information and initial classification results existing in the image to be processed are obtained. For any detected target, based on the LayerCAM algorithm, a saliency map or counterfactual attribution saliency map corresponding to the first target is obtained. Subsequently, the size and position of the blocker are determined based on the saliency map or counterfactual attribution saliency map corresponding to the first target. Then, based on the use of the blocker to block the first target, it is determined whether the area blocked by the blocker is a backdoor triggering area. Since the saliency map is composed of individual pixel points, when the size and pixel value of the image to be processed change, the saliency map will also change accordingly. Therefore, backdoor detection based on the saliency map can be applied in actual physical world scenes. In addition, by executing the LayerCAM algorithm to obtain the saliency map or counterfactual attribution saliency map corresponding to the first target, the LayerCAM algorithm has a lower complexity, so the delay is shorter, and thus it can be used for real-time backdoor detection, that is, it can detect and locate the position of the backdoor triggering area in real time. Furthermore, by executing the LayerCAM algorithm to obtain the saliency map or counterfactual attribution saliency map corresponding to the first target, compared with the GradCAM algorithm, each pixel in each feature map has a corresponding weight. The saliency map obtained based on the weight of each pixel can more accurately find the area that contributes the most to the prediction result of the neural network. The obtained saliency map is more accurate, so it is more suitable for application in the backdoor trigger positioning of the single-stage target detection network.

[0033] Furthermore, by performing smoothing filtering on the image after occlusion, whether the area occluded by the blocker is a backdoor triggering area is determined based on the smoothed filtered image. The smoothing filtering can slightly change the characteristic values ​​of the pixel points in the saliency map, which is conducive to obtaining a new classification result of the first target more accurately. It can also more accurately determine whether the area occluded by the blocker is a backdoor triggering area, greatly improving the backdoor detection rate, especially in the digital world. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a schematic diagram comparing the detection results of the same toxic network for clean samples and toxic samples;

[0035] Figure 2 This is a schematic diagram of the detection results obtained by using the Grad CAM algorithm to locate the backdoor trigger area of ​​the target cup in image X;

[0036] Figure 3 is a flow chart of a backdoor detection method of a target detection network in an embodiment of the present invention;

[0037] Figure 4 is a flow chart of a backdoor defense method for a target detection network in an embodiment of the present invention;

[0038] Figure 5 is a flow chart of a backdoor detection and defense process in an embodiment of the present invention;

[0039] Figure 6 It uses LayerCAM algorithm for positioning Figure 2 The backdoor triggering area of ​​the target cup in the middle image X, and the schematic diagram of the detection results;

[0040] Figure 7 It is a structural schematic diagram of a backdoor detection device of a target detection network in an embodiment of the present invention;

[0041] Figure 8 It is a structural schematic diagram of a backdoor defense device for a target detection network in an embodiment of the present invention. DETAILED DESCRIPTION

[0042] Object detection, as one of the core issues in the field of computer vision, is the task of locating and classifying objects in images.

[0043] In recent years, driven by deep learning technology, with the introduction of target detection networks such as Regions with CNN features (RCNN), Single Shot Multi Box Detector (SSD), and You Only Look Once (YOLO), the performance of target detection algorithms has been continuously improved, but there is also the risk of backdoors being implanted.

[0044] In the embodiment of the present invention, a sample without a backdoor trigger pattern is called a clean sample, a sample with a backdoor trigger pattern is called a poisonous sample, a target detection network implanted with a backdoor is called a poisonous network, and a target detection network without a backdoor implanted is called a clean network.

[0045] Figure 1Schematic diagram showing the difference in detection results between clean samples and toxic samples using the same toxic network.

[0046] in, Figure 1 (a) is a schematic diagram of the detection process of the toxic network on clean samples. Figure 1 (b) is a schematic diagram of the detection process of the poisonous network on the poisonous sample. The source class of the backdoor is person, the target class is cup, and the backdoor trigger is the HelloKitty pattern.

[0047] like Figure 1 As shown in (a), when a clean sample is input, the detection result of the source class target in the clean sample using the toxic network is still person, which is the same as the clean network. Figure 1 As shown in (b), when a poisonous sample is input, the detection result of the source class target in the poisonous sample using the poisonous network is cup.

[0048] Since the neural network structure of the deep learning-based target detection algorithm is relatively complex, it poses a great challenge to backdoor detection / defense. In addition, the real-time performance of the target detection algorithm also places requirements on the speed of backdoor defense.

[0049] Based on deep learning, target detection algorithms can be divided into two categories: one-stage network and two-stage network from the perspective of detection process. Among them, the target detection algorithm of the two-stage network first generates a prediction box of the target, then frames the target according to the prediction box, and then inputs the classification network to classify the target. The representative algorithm is the RCNN series. The two-stage network has high accuracy, but the amount of calculation is large and the speed is slow.

[0050] Compared with the step-by-step process of the two-stage network target detection algorithm, the single-stage network target detection algorithm directly predicts the target box for the input image and gives the classification result. Representative algorithms include the YOLO series algorithm and the SSD algorithm. Although this type of algorithm is not as good as the two-stage network target detection algorithm in terms of small target detection performance, the algorithm speed has been greatly improved and is more suitable for real-time video streams, so it has been more widely used in practice.

[0051] At present, research on backdoor detection and defense is almost entirely focused on digital world scenarios. Although some methods can also be migrated to the physical world, they do not consider the computing power and latency issues that may be encountered when actually used in specific scenarios in the physical world.

[0052] A backdoor detection algorithm that can be applied in the physical world uses a pixel block of a pre-set size to scan the entire image in sequence. When the pixel block covers an area in the image and causes the neural network prediction result to change, the area is considered a suspicious area, and then some confirmation mechanisms are used to determine whether the area is really triggered by a backdoor.

[0053] The disadvantage of the above algorithm is that it needs to know the size of the backdoor trigger area in advance and set the size of the pixel block based on this prior knowledge. However, in actual scenes, the prior knowledge of the backdoor trigger area is unknown. In addition, the pixel block needs to scan the entire area of ​​the picture, which is very time-consuming and not applicable to actual scenes.

[0054] Another way to locate the backdoor trigger is to use the saliency map to find the area in the input image that contributes most to the neural network prediction result. However, the most commonly used gradient weighted class activation heat map (Grad CAM) algorithm does not work well in single-stage class object detection networks.

[0055] Figure 2 (a) to Figure 2 (i) is a schematic diagram of the detection results obtained by using the GradCAM algorithm to locate the backdoor trigger area of ​​the target cup in image X in the neural network convolutional layers Module2, Module4, Module6, Module8, Module10, Module13, Module17, Module20 to Module23. The backdoor trigger is the Hello Kitty pattern.

[0056] Reference Figure 2 (a) to Figure 2 (i) When using the GradCAM algorithm, at the high level of the neural network, such as the convolutional layer Module23, the class activation heat map results are always in the center of the prediction box and cannot be located on the backdoor trigger. This is because: the single-stage class target detection network divides the image into several grids during the training phase, and only selects the grid closest to the target center to calculate the positive sample loss. In the prediction phase, non-maximum suppression (NMS) is performed to obtain all prediction results that meet the intersection-over-union (IOU) condition for the same target, and only outputs the prediction box with the highest target confidence, which is usually the prediction result of the grid where the target center is located. Therefore, the class activation map of the high-level neural network is always concentrated in the central grid area where the target is located.

[0057] In the shallow layers of the neural network, such as the convolutional layers Module2, Module4, and Module6, the corresponding class activation heat map results have a lot of noise points, and the positioning of the backdoor trigger is very inaccurate. This is because the GradCAM algorithm assigns a global weight to each feature map when calculating the saliency map. This global weight is used to weighted average all feature maps to obtain the final saliency map. However, the global weight cannot represent the contribution of each spatial position pixel in the feature map to the final saliency map, so the saliency map they generate in the shallow layers of the neural network is inaccurate. Therefore, the GradCAM algorithm cannot be used for the backdoor trigger positioning of the single-stage class object detection network.

[0058] To solve this problem, the present invention provides a backdoor detection method for a target detection network, which can not only be applied in actual physical world scenes, but also obtain a saliency map or a counterfactual attribution saliency map corresponding to the first target by executing the LayerCAM algorithm. The LayerCAM algorithm has a lower complexity and a shorter delay, so that the position of the backdoor trigger area can be detected and located in real time. Compared with the GradCAM algorithm, the saliency map or the counterfactual attribution saliency map corresponding to the first target obtained by executing the LayerCAM algorithm is more suitable for application in the backdoor triggering location of a single-stage target detection network.

[0059] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0060] Reference Figure 3 The embodiment of the present invention provides a backdoor detection method for a target detection network, and the method may include the following steps:

[0061] Step 31, input the image to be processed into the target detection network, detect and obtain the position information and initial classification result of the target existing in the image to be processed; the target detection network is a neural network.

[0062] In a specific implementation, after the camera captures a frame of image X to be processed, the captured image X to be processed is input into the target detection network F, and the target detection network F gives the position information and initial classification result of the target existing in the image X to be processed.

[0063] In a specific implementation, the neural network used as the target detection network is a trained neural network. For the input image to be processed X, it is processed by the neural network models of each convolution layer in sequence from shallow to deep. Each convolution layer outputs a feature map. Based on the feature maps output by each convolution layer, the feature information of each target in the image to be processed X and the initial classification results of each target can be obtained.

[0064] For example, refer to Figure 1 As shown in (a), the image obtained after the clean sample is processed by the neural network will display the prediction box (i.e., target location information) and the classification result of each prediction box (i.e., initial classification result).

[0065] Step 32: for a first target detected in the image to be processed, based on the LayerCAM algorithm, obtain a saliency map or a counterfactual attribution saliency map corresponding to the first target; the first target is any target detected in the image to be processed.

[0066] In a specific implementation, the first target n represents any target detected in the image to be processed X. The initial classification result of the first target is called a target class y n .

[0067] In one embodiment of the present invention, a LayerCAM algorithm may be executed on the first target n in a shallow layer of the target detection network to obtain a saliency map corresponding to the first target n in the shallow layer.

[0068] The shallow layer is one of the shallow layers of the target detection network, which can be determined according to the structure of the target detection network. In a single-stage target detection network, the shallow layer is the layer before the maximum pooling layer whose step size is smaller than the kernel size. Those skilled in the art can pre-execute the LayerCAM algorithm on each shallow layer of the target detection network, and select the shallow layer with the most accurate saliency map as the shallow layer for finally executing the LayerCAM algorithm on the first target n.

[0069] Assume that the feature map of the k-channel output in the shallow layer is represented by A k When the LayerCAM algorithm is executed on the shallow layer for the first target n of the image to be processed, the contribution value of each pixel point of the k channel in the shallow layer to the prediction result can be calculated first, and then the weighted sum of the feature maps of all channels in the first shallow layer is performed, and finally the saliency map is obtained through the relu function.

[0070] Specifically, the initial classification category prediction score y of the first target n can be calculated using formula (1): c About A k The gradient of the pixel position (i, j) in Then use formula (2) and the relu function to get A k The weight of the pixel position (i, j) in Then, formula (3) is used to weight the feature maps of all channels in the same layer based on these weights, and finally the saliency map is obtained by the ReLU function.

[0071]

[0072]

[0073]

[0074]

[0075] In another embodiment of the present invention, a LayerCAM algorithm of counterfactual attribution may be executed on the first target at a shallow layer of the target detection network to obtain a counterfactual attribution saliency map corresponding to the first target.

[0076] In a specific implementation, after the image X to be processed is input into the target detection network, in addition to the initial classification result of the first target, other classification prediction results about the first target can also be obtained. For other classification prediction results of the first target, the counterfactual attribution saliency map corresponding to the other classification prediction results can be obtained by referring to the above method for calculating the saliency map corresponding to the initial classification result.

[0077] Specifically, the negative gradient of the prediction score of each other classification prediction result of the first target n with respect to each pixel position in the feature map is calculated, and then the weight of each pixel position in the feature map is obtained through the relu function, and the feature maps of all channels in the same layer are weighted and summed based on these weights, and finally the counterfactual attribution saliency map of the first target n is obtained through the relu function. For example, the prediction score y of the prediction result t of the first target n can be calculated by referring to formula (5): t About A k The negative gradient of the pixel position (i, j) in Subsequently, based on the gradient value, a counterfactual attribution saliency map of the first target n is obtained.

[0078]

[0079] At this time, compared with directly calculating the saliency map corresponding to the first target, although the amount of calculation is increased, since the counterfactual attribution method calculates the area that contributes most to the misclassification, this area in the backdoor network must be concentrated on the backdoor trigger. Compared with directly calculating the saliency map corresponding to the current predicted class, there is less noise, so the saliency map corresponding to the first target obtained will be more accurate, thereby further improving the accuracy of backdoor trigger positioning.

[0080] In a specific implementation, in order to reduce the amount of calculation in the process of obtaining the counterfactual attribution saliency map corresponding to the first target using the counterfactual attribution method, only the counterfactual attribution saliency maps corresponding to some of the other classification prediction results of the first target can be calculated, rather than the counterfactual attribution saliency maps corresponding to all classification prediction results. Here, some of the other classification prediction results can select source classes that are easily attacked.

[0081] When the LayerCAM algorithm is used to calculate the saliency map, it can be seen from formula (2) that each pixel in a feature map has its own weight. Subsequently, the corresponding saliency map is calculated based on the weight of each pixel. Compared with all pixels in a feature map corresponding to the same weight, the weight of each pixel can better represent the contribution of the pixel to the saliency map, so the generated saliency map is more accurate and more suitable for application in the backdoor trigger positioning of the single-stage class object detection network.

[0082] Step 33: Determine the size and position of the blocker based on the saliency map or the counterfactual attribution saliency map corresponding to the first target.

[0083] In a specific implementation, the blocker is a pixel block with adjustable size and position, which is used to block the backdoor triggering area. The image blocked by the blocker can be combined with the original image to form a new image, which is suitable for subsequent image processing.

[0084] In a specific implementation, a variety of methods can be used to determine the size and position of the blocker, which are not limited here.

[0085] In one embodiment of the present invention, the position of the maximum eigenvalue in the saliency map or the counterfactual attribution saliency map may be used as the center point position of the barrier, and the size of the barrier may be determined using preset threshold constraint information.

[0086] In a specific implementation, there is usually a maximum eigenvalue position in the saliency map, and this position is used as the center point position of the blocker. When there are two maximum eigenvalue positions in the saliency map, any one of the maximum eigenvalue positions can be selected as the center point position of the blocker.

[0087] In a specific implementation, the position of the maximum eigenvalue in the saliency map is the position that has the greatest impact on the prediction result. The inventors have found that the position of the maximum eigenvalue in the saliency map is likely to cover the position of the backdoor trigger. Therefore, by using the position of the maximum eigenvalue in the saliency map as the center point of the blocker, the positioning accuracy of the backdoor trigger area can be improved. In addition, when determining the position of the blocker, no prior knowledge is required, which is more suitable for actual physical world scenarios.

[0088] In a specific implementation, the preset threshold constraint information can be multiple, which is not limited here. For example, the size of the blocker can be set to 0.2 times the area of ​​the target position (i.e., the prediction box) corresponding to the first target. In this way, when the center point of the blocker is determined, the size of the blocker can be determined based on the preset threshold constraint information.

[0089] In another embodiment of the present invention, it is also possible to determine the pixel points in the saliency map or counterfactual attribution saliency map corresponding to the first target whose eigenvalues ​​are greater than a preset eigenvalue threshold, obtain a corresponding connectivity map, and then use the size and position of the obtained connectivity map as the size and position of the barrier.

[0090] The preset feature threshold can be set according to actual conditions, and those skilled in the art can obtain the preset feature value threshold that is most suitable for the image to be processed through a limited number of implementations.

[0091] The characteristic value of each pixel in the saliency map or the counterfactual attribution saliency map is compared with the preset characteristic value threshold, and the pixel points whose characteristic value is greater than the preset characteristic value threshold are selected, and the selected pixel points are connected to form a connected map. The size and position of the connected map are the size and position of the barrier.

[0092] In the specific implementation, the pixel points whose eigenvalues ​​are greater than the preset eigenvalue threshold cover the pixel points where the maximum eigenvalue is located in the saliency map, which can also improve the positioning accuracy of the backdoor trigger area. In addition, when determining the position of the blocker, no prior knowledge is required, which is more suitable for actual physical world scenarios.

[0093] Step 34: Use the blocker to block the first target and determine whether the area blocked by the blocker is a backdoor triggering area.

[0094] In a specific implementation, the image blocked by the blocker can be directly input into the target detection network to obtain a new classification result of the first target in the image to be processed, and the initial classification result of the first target is compared with the new classification result of the first target. When the two are consistent, it indicates that the area blocked by the blocker is not the backdoor triggering area. When the two are inconsistent, it indicates that the area blocked by the blocker is the backdoor triggering area.

[0095] In one embodiment of the present invention, in order to further improve the backdoor detection rate, the image after being blocked by the blocker can be smoothed and filtered to obtain a smoothed filtered image, and then based on the smoothed filtered image, it is determined whether the area blocked by the blocker is a backdoor triggering area.

[0096] Wherein, determining whether the area blocked by the blocker is a backdoor triggering area based on the processed image may include: inputting the processed image into the target detection network to obtain a new classification result of the first target in the image to be processed, and then comparing the initial classification result of the first target with the new classification result of the first target. When the initial classification result of the first target is inconsistent with the new classification result of the first target, determining that the area blocked by the blocker is a backdoor triggering area.

[0097] Since smoothing filtering can slightly change the value of pixels in the image, when the backdoor trigger position located using the saliency map has a certain offset, the trigger blocker cannot completely block the backdoor trigger, and the residual trigger may still cause the network to misclassify. The smoothing filter can modify the residual trigger pixel value, further weakening the influence of the backdoor, thereby making the new classification result of the first target inconsistent with the initial classification result of the first target, so that it can more accurately determine whether the area blocked by the blocker is the backdoor trigger area, greatly improving the backdoor detection rate.

[0098] It should be noted that, by using the backdoor detection method in the embodiment of the present invention, it is possible to detect whether there is a backdoor trigger only in the area where the first target is located, or to detect whether there is a backdoor trigger outside the area of ​​the first target. There is no limitation on which specific area is detected.

[0099] Reference Figure 4 The embodiment of the present invention further provides a backdoor defense method for a target detection network, and the method may include:

[0100] Step 41, performing backdoor detection on some or all of the objects in the image to be processed.

[0101] In a specific implementation, the backdoor detection method of the target detection network in the embodiment of the present invention can be used to perform backdoor detection on part or all of the detection targets in the image to be processed.

[0102] It should be noted that when performing backdoor detection on different detection targets, you can choose to execute the LayerCAM algorithm in the same shallow layer of the target detection network, or you can execute the LayerCAM algorithm in different shallow layers of the target detection network, and there is no restriction here. For example, when performing backdoor detection on one target, the LayerCAM algorithm is executed in the second convolutional layer of the target detection network, and when performing backdoor detection on another target, the LayerCAM algorithm can be executed in the second convolutional layer of the target detection network, or it can be executed in the third convolutional layer of the target detection network.

[0103] Step 42: Based on the backdoor detection results of the partial or all targets, correct the erroneous prediction caused by the backdoor triggering in the image to be processed.

[0104] In a specific implementation, for a target with a backdoor triggering area, the erroneous prediction caused by the backdoor triggering in the image to be processed can be corrected to avoid outputting erroneous classification results.

[0105] Step 43: output the corrected image.

[0106] In a specific implementation, in the corrected image, the erroneously predicted classification result has been corrected, and the backdoor triggering area can also be identified, including identifying the position of the backdoor triggering area, and the area at the position is the backdoor triggering area.

[0107] Figure 5 Flowchart of a backdoor detection and defense process in an embodiment of the present invention. Figure 5 , the backdoor detection and defense process may include:

[0108] Step 501: Obtain the image X to be processed, the target detection network F, and the set C0 of all prediction classes of the target detection network F.

[0109] In a specific implementation, when the target detection network F is known, the number of candidates of the class that can be predicted for any detection target can be obtained in advance. The candidates of the class that can be predicted for a detection target in the target detection network F constitute the prediction class set C0.

[0110] Step 502: input the image to be processed X into the target detection network F to obtain the prediction result h of the first target.

[0111] The prediction result h includes the initial classification result of the first target and other classification prediction results. The first target is any target detected in the image to be processed X. The set of other classification prediction results of the first target is C1.

[0112] Step 503: execute the LayerCAM algorithm to calculate and obtain the counterfactual attribution saliency map corresponding to other classification prediction results z of the first target.

[0113] In a specific implementation, the LayerCAM algorithm can be executed on any other classification prediction result z in the shallow layer of the target detection network to obtain the counterfactual attribution saliency map corresponding to the other classification prediction result z of the first target.

[0114] Step 504, determine the size and location of the blocker.

[0115] In a specific implementation, the size and position of the blocker may be determined based on the counterfactual attribution saliency map corresponding to the first target.

[0116] Step 505: Use a blocker to block the first target to obtain a blocked image X1.

[0117] Step 506: Perform smoothing filtering on the occluded image X1 to obtain a smoothed filtered image X2.

[0118] Step 507: Input the smoothed filtered image X2 to the target detection network F to obtain a first target new classification result h'.

[0119] Step 508 , comparing the initial classification result h of the first target with the new classification result h′, and determining whether the blocker position is a backdoor triggering area.

[0120] When the initial classification result h and the new classification result h′ of the first target are different, the blocker position is the backdoor triggering area. At this time, step 509 may be continued to be executed. Otherwise, step 510 is executed.

[0121] Step 509, correct the prediction error.

[0122] Step 510, let C1 = C1-z, that is, remove other classification prediction results that have been backdoor detected from the set C1 of other classification prediction results of the first target.

[0123] Step 511, determine whether C1 is an empty set.

[0124] When the set C1 of other classification prediction results of the first target is an empty set after excluding other classification prediction results that have undergone backdoor detection, it indicates that all other classification prediction results of the first target have undergone backdoor detection. Otherwise, backdoor detection can continue to be performed on any remaining other classification prediction results, that is, steps 503 to 511 are repeated.

[0125] Step 512: output the final target prediction result.

[0126] When all other classification prediction results of the first target have been backdoor detected, the final target prediction result is output.

[0127] Regarding the above steps 501 to 512, specific implementation may be performed with reference to the above description of the backdoor detection method and defense method, which will not be repeated here.

[0128] Figure 6 (a) to Figure 6 (i) Using the LayerCAM algorithm, in the convolutional layers Module2, Module4, Module6, Module8, Module10, Module13, Module17, Module20 to Module23, locate Figure 2 The backdoor trigger area of ​​the target cup in image X shows the detection result. The backdoor trigger is the Hello Kitty pattern.

[0129] Reference Figure 6 (a) to Figure 6 (i), the backdoor trigger area located in the convolutional layer Module8, such as Figure 6As shown in (e), it has been basically positioned on the backdoor trigger area, with very few noise points, more accurate positioning, and more suitable for backdoor trigger positioning of single-stage target detection networks.

[0130] From the above content, it can be seen that the solution of the embodiment of the present invention, by calculating the shallow saliency map of the target detection network, and then using the LayerCAM algorithm to locate the position of the backdoor trigger, has better real-time performance and is more suitable for application in the backdoor trigger positioning of the single-stage target detection network. In addition, in the process of detecting the backdoor, no prior knowledge is required, and the backdoor trigger of any position and any size that may appear can be detected, which can be applied in actual physical world scenes.

[0131] In order to enable those skilled in the art to better understand and implement the present invention, the device, test system, electronic device and computer-readable storage medium corresponding to the above method are described in detail below.

[0132] Reference Figure 7 The embodiment of the present invention further provides a backdoor detection device 70 for a target detection network, and the device 70 may include: a target detection unit 71, a saliency map acquisition unit 72, a blocker determination unit 73 and a backdoor detection unit 74. Among them:

[0133] The target detection unit 71 is adapted to input the image to be processed into the target detection network, and detect and obtain the position information and initial classification result of the target existing in the image to be processed; the target detection network is a neural network;

[0134] The saliency map acquisition unit 72 is adapted to obtain a saliency map or a counterfactual attribution saliency map corresponding to a first target detected in the image to be processed based on a LayerCAM algorithm; the first target is any target detected in the image to be processed;

[0135] The blocker determination unit 73 is adapted to determine the size and position of the blocker based on the saliency map or the counterfactual attribution saliency map corresponding to the first target;

[0136] The backdoor detection unit 74 is adapted to use the blocker to block the first target and determine whether the area blocked by the blocker is a backdoor triggering area.

[0137] In one embodiment of the present invention, the salient map acquisition unit 72 is adapted to execute a LayerCAM algorithm on the first target at a shallow layer of the target detection network to obtain a salient map corresponding to the first target.

[0138] In another embodiment of the present invention, the saliency map acquisition unit 72 is adapted to execute a LayerCAM algorithm of counterfactual attribution on the first target at a shallow layer of the target detection network to obtain a counterfactual attribution saliency map corresponding to the first target.

[0139] In one embodiment of the present invention, the barrier determination unit 73 is adapted to use the maximum eigenvalue position in the saliency map or the counterfactual attribution saliency map as the center point position of the barrier, and determine the size of the barrier using preset threshold constraint information.

[0140] In another embodiment of the present invention, the barrier determination unit 73 is suitable for determining the pixel points whose characteristic values ​​in the saliency map or counterfactual attribution saliency map corresponding to the first target are greater than a preset characteristic value threshold, and obtaining a corresponding connectivity map; and using the size and position of the obtained connectivity map as the size and position of the barrier.

[0141] In one embodiment of the present invention, the backdoor detection unit 74 is suitable for performing smoothing filtering on the blocked image to obtain a smoothed filtered image; and based on the smoothed filtered image, determining whether the area blocked by the blocker is a backdoor triggering area.

[0142] In another embodiment of the present invention, the backdoor detection unit 74 is suitable for inputting the smoothed filtered image into the target detection network to obtain a new classification result of the first target; comparing the initial classification result of the first target with the new classification result of the first target; and when the initial classification result of the first target is inconsistent with the new classification result of the first target, determining that the area blocked by the blocker is the backdoor triggering area.

[0143] The embodiment of the present invention further provides a backdoor defense device 80 for a target detection network, the backdoor defense device 80 comprising: a backdoor detection device 70, a correction unit 81 and an output unit 82. Among them:

[0144] The backdoor detection device 70 is adapted to perform backdoor detection on some or all objects in the image to be processed;

[0145] The correction unit 81 is adapted to correct the erroneous prediction caused by the backdoor triggering in the image to be processed based on the backdoor detection results of the part or all of the targets;

[0146] The output unit 82 is adapted to output the corrected image.

[0147] In one embodiment of the present invention, the output unit 82 is further adapted to identify a backdoor triggering area in the output corrected image.

[0148] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. The computer program is executed by a processor to implement the steps of the above-mentioned backdoor detection method for the target detection network.

[0149] An embodiment of the present invention also provides another computer-readable storage medium on which a computer program is stored. The computer program is executed by a processor to implement the steps of the above-mentioned backdoor defense method for the target detection network.

[0150] In a specific implementation, the computer-readable storage medium may include: ROM, RAM, magnetic disk or optical disk, etc.

[0151] An embodiment of the present invention also provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and wherein the processor executes the steps of any of the above-mentioned backdoor detection methods for a target detection network when running the computer program.

[0152] An embodiment of the present invention also provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and wherein the processor executes the steps of any of the above-mentioned backdoor defense methods for a target detection network when running the computer program.

[0153] Although the present invention is disclosed as above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the scope defined by the claims.

Claims

1. A backdoor detection method for a target detection network, characterized in that: include: Inputting the image to be processed into the target detection network to detect and obtain the position information and initial classification results of the target existing in the image to be processed; The target detection network is a neural network; For a first target detected in the image to be processed, obtaining a saliency map or a counterfactual attribution saliency map corresponding to the first target based on the LayerCAM algorithm; the first target is any target detected in the image to be processed; Determining a size and a position of a blocker based on a saliency map or a counterfactual attribution saliency map corresponding to the first target; The first target is blocked by the blocker to determine whether the area blocked by the blocker is a backdoor triggering area.

2. The backdoor detection method of the target detection network according to claim 1, characterized in that: For a first target detected in the image to be processed, obtaining a saliency map corresponding to the first target based on the LayerCAM algorithm includes: In a shallow layer of the object detection network, a LayerCAM algorithm is executed on the first object to obtain a saliency map corresponding to the first object.

3. The backdoor detection method of the target detection network as claimed in claim 1, characterized in that: For a first target detected in the image to be processed, a counterfactual attribution saliency map corresponding to the first target is obtained based on the LayerCAM algorithm, including: In a shallow layer of the object detection network, a LayerCAM algorithm of counterfactual attribution is performed on the first object to obtain a counterfactual attribution saliency map corresponding to the first object.

4. The backdoor detection method of the target detection network as claimed in claim 1, characterized in that: The determining the size and position of the blocker based on the saliency map or the counterfactual attribution saliency map corresponding to the first target includes: The maximum eigenvalue position in the saliency map or the counterfactual attribution saliency map is used as the center point position of the blocker, and the size of the blocker is determined using preset threshold constraint information.

5. The backdoor detection method of the target detection network as claimed in claim 1, characterized in that: The determining the size and position of the blocker based on the saliency map or the counterfactual attribution saliency map corresponding to the first target includes: Determine the pixel points whose characteristic values ​​in the saliency map or the counterfactual attribution saliency map corresponding to the first target are greater than a preset characteristic value threshold, and obtain a corresponding connectivity map; The size and position of the obtained connectivity graph are used as the size and position of the barrier.

6. The backdoor detection method of the target detection network according to any one of claims 1 to 5, characterized in that: The using the blocker to block the first target and determining whether the area blocked by the blocker is a backdoor triggering area includes: Performing smoothing filtering on the blocked image to obtain a smoothed filtered image; Based on the smoothed filtered image, it is determined whether the area blocked by the blocker is a backdoor triggering area.

7. The backdoor detection method of the target detection network as claimed in claim 6, characterized in that: The determining, based on the smoothed filtered image, whether the area blocked by the blocker is a backdoor triggering area comprises: Inputting the smoothed filtered image into the target detection network to obtain a new classification result of the first target; Compare the initial classification result of the first target with the new classification result of the first target; when the initial classification result of the first target is inconsistent with the new classification result of the first target, determine that the area blocked by the blocker is a backdoor triggering area.

8. A backdoor defense method for a target detection network, characterized in that: include: Using the backdoor detection method of the target detection network according to any one of claims 1 to 7, performing backdoor detection on some or all targets in the image to be processed; Based on the backdoor detection results of the part or all of the targets, correct the erroneous prediction caused by the backdoor triggering in the image to be processed; Output the corrected image.

9. The backdoor defense method of the target detection network as claimed in claim 8, characterized in that: The output corrected image comprises: A backdoor triggering area is identified in the output corrected image.

10. A backdoor detection device for a target detection network, characterized in that: include: The target detection unit is adapted to input the image to be processed into the target detection network, and detect and obtain the position information and initial classification result of the target existing in the image to be processed; The target detection network is a neural network; a saliency map acquisition unit, adapted to obtain, for a first target detected in the image to be processed, a saliency map or a counterfactual attribution saliency map corresponding to the first target based on a LayerCAM algorithm; The first target is any target detected in the image to be processed; a blocker determination unit, adapted to determine a size and a position of a blocker based on a saliency map or a counterfactual attribution saliency map corresponding to the first target; The backdoor detection unit is adapted to use the blocker to block the first target and determine whether the area blocked by the blocker is a backdoor triggering area.

11. The backdoor detection device for target detection network according to claim 10, characterized in that: The saliency map acquisition unit is adapted to execute a LayerCAM algorithm on the first target at a shallow layer of the target detection network to obtain a saliency map corresponding to the first target.

12. The backdoor detection device for target detection network according to claim 10, characterized in that: The saliency map acquisition unit is adapted to execute a LayerCAM algorithm of counterfactual attribution on the first target at a shallow layer of the target detection network to obtain a counterfactual attribution saliency map corresponding to the first target.

13. The backdoor detection device for target detection network according to claim 10, characterized in that: The barrier determination unit is adapted to use the maximum eigenvalue position in the saliency map or the counterfactual attribution saliency map as the center point position of the barrier, and determine the size of the barrier using preset threshold constraint information.

14. The backdoor detection device for target detection network according to claim 10, characterized in that: The barrier determination unit is adapted to determine the pixel points whose characteristic values ​​in the saliency map or counterfactual attribution saliency map corresponding to the first target are greater than a preset characteristic value threshold, to obtain a corresponding connectivity map; and to use the size and position of the obtained connectivity map as the size and position of the barrier.

15. The backdoor detection device for target detection network according to any one of claims 10 to 14, characterized in that: The backdoor detection unit is adapted to perform smoothing filtering on the blocked image to obtain a smoothed filtered image; and determine whether the area blocked by the blocker is a backdoor triggering area based on the smoothed filtered image.

16. The backdoor detection device for target detection network according to claim 15, characterized in that: The backdoor detection unit is adapted to input the smoothed filtered image into the target detection network to obtain a new classification result of the first target; comparing the initial classification result of the first target with the new classification result of the first target; And when the initial classification result of the first target is inconsistent with the new classification result of the first target, determining that the area blocked by the blocker is a backdoor triggering area.

17. A backdoor defense device for a target detection network, characterized in that: include: The backdoor detection device of the target detection network according to any one of claims 10 to 16, suitable for performing backdoor detection on some or all targets in the image to be processed; A correction unit, adapted to correct the erroneous prediction caused by the backdoor triggering in the image to be processed based on the backdoor detection results of the part or all of the objects; The output unit is adapted to output the corrected image.

18. The backdoor defense device for target detection network according to claim 17, characterized in that: The output unit is further adapted to identify a backdoor triggering area in the output corrected image.

19. A computer-readable storage medium having a computer program stored thereon, characterized in that: The computer program is executed by a processor to implement the steps of the method according to any one of claims 1 to 7.

20. A computer-readable storage medium having a computer program stored thereon, characterized in that: The computer program is executed by a processor to implement the steps of the method according to any one of claims 8 or 9.

21. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor runs the computer program, the steps of the method according to any one of claims 1 to 7 are performed.

22. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor runs the computer program, the steps of the method according to claim 8 or 9 are performed.

Citation Information

Patent Citations

  • Image detection method and computer readable storage medium

    CN110866897A

  • Image recognition back door defense method and device based on swarm intelligence, and medium

    CN114048474A

  • Method for generating saliency map and method and device for detecting abnormal object

    CN115147618A

  • System for proving robust interpretable machine learning model

    CN115997218A

  • Device and method for evaluating a saliency map determiner

    EP3767534A1