A backdoor risk detection method and system applied to a target detection model

Abstract

This invention relates to a method and system for backdoor risk detection applied to an object detection model. The method includes: testing the object detection model using a pre-constructed image sample set and outputting test results, wherein the test results include: the detection category of target objects marked with non-maximum suppression candidate boxes in the image samples; counting the number of target objects in image samples whose detected category is inconsistent with the true category, calculating the proportion of the number of target objects relative to the total number of image samples in the image sample set; and determining whether the object detection model has a backdoor risk based on the proportion. This method effectively overcomes the coverage blind spots and subjective bias problems of traditional detection methods for novel backdoors, achieving effective detection of novel backdoors.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a backdoor risk detection method and system applied to target detection models. Background Technology

[0002] Object detection technology, a core technology in computer vision, has been widely applied in safety-critical scenarios such as autonomous driving, video surveillance, security detection, and facial recognition. These applications rely on the efficiency and accuracy of object detection models to accurately locate and classify various targets in images or videos. Existing object detection technologies primarily depend on deep learning models such as convolutional neural networks (CNNs), which learn image features through training to identify objects and use post-processing methods such as non-maximum suppression (NMS) to remove redundant detection boxes, ensuring detection accuracy.

[0003] However, with the continuous development of object detection technology, malicious backdoor perpetrators have also begun to exploit backdoors and other techniques to maliciously tamper with model outputs, thereby threatening the security and reliability of the system. Backdoors, as a means of poisoning the model by implanting triggers, have become a major security threat in the field of computer vision. Currently, newer backdoors can exploit the overlapping relationships between target objects in a scene to create a backdoor. By polluting training data or modifying the training process, the target model can maintain normal recognition under normal input, but under the influence of specific triggers and the overlapping relationship with the victim, it can produce the expected malicious behavior of misclassification.

[0004] In order to ensure the normal operation of the target detection model and avoid dangerous results caused by misidentification due to the backdoor, a detection method is urgently needed to detect whether a new backdoor has been implanted in the target detection model. Summary of the Invention

[0005] The purpose of this invention is to provide a backdoor risk detection method and system applied to target detection models, which effectively solves the problems of coverage blind spots and subjective bias in traditional detection methods for new backdoors, and achieves effective detection of new backdoors.

[0006] To address the aforementioned issues, in a first aspect, embodiments of the present invention provide a backdoor risk detection method applied to an object detection model, comprising: testing the object detection model using a pre-constructed image sample set and outputting test results, wherein the test results include: the detection category of the target object marked by a non-maximum suppression candidate box in the image sample; counting the number of targets in image samples whose detection category is inconsistent with the true category, calculating the proportion of the number of targets relative to the total number of image samples in the image sample set; and determining whether the object detection model has a backdoor risk based on the proportion.

[0007] Another embodiment of the present invention provides a backdoor risk detection system, comprising: a data acquisition module, configured to test a target detection model using a pre-constructed image sample set and output test results, the test results including: the detection category of target objects marked by non-maximum suppression candidate boxes in the image samples; a data analysis module, configured to count the number of targets in image samples whose detection category is inconsistent with the true category, and calculate the proportion of the number of targets relative to the total number of image samples in the image sample set; and a risk assessment module, configured to determine whether the target detection model has a backdoor risk based on the proportion.

[0008] This invention also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the backdoor risk detection method applied to a target detection model as described above.

[0009] This invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the backdoor risk detection method applied to a target detection model as described above.

[0010] This invention also provides a computer program product, including computer instructions, which, when executed by a processor, implement the backdoor risk detection method applied to a target detection model as described above.

[0011] This embodiment proposes a detection method to address the hidden security threat posed to target detection models by novel backdoors triggered by overlapping target objects. The method tests the target detection model by pre-constructing an image sample set covering normal and potential triggering scenarios, focusing on the final detection category of candidate boxes after non-maximum suppression. It statistically analyzes the proportion of image samples whose detected category differs from the true category and quantifies the backdoor risk based on this proportion. Overall, it achieves accurate identification of this type of novel hidden backdoor, effectively overcoming the coverage blind spots and subjective bias problems of traditional detection methods for novel backdoors. It does not rely on the model's internal structure or modify the design of the model's core architecture and parameters, and can flexibly adapt to various CNN-based target detection models and safety-critical application scenarios such as autonomous driving and video surveillance. Attached Figure Description

[0012] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0013] Figure 1 This is a flowchart illustrating the backdoor risk detection method applied to a target detection model according to the first embodiment of the present invention.

[0014] Figure 2 This is a flowchart illustrating the specific process of constructing an image sample set according to the second embodiment of the present invention;

[0015] Figure 3 This is a flowchart illustrating the specific process of constructing an image sample set according to the third embodiment of the present invention;

[0016] Figure 4 This is a flowchart illustrating the backdoor risk detection method applied to a target detection model according to the fifth embodiment of the present invention.

[0017] Figure 5 This is a flowchart illustrating the backdoor risk detection method applied to a target detection model according to the sixth embodiment of the present invention.

[0018] Figure 6 This is a schematic diagram of the backdoor risk detection system according to the seventh embodiment of the present invention;

[0019] Figure 7 This is a schematic diagram of the structure of the electronic device according to the eighth embodiment of the present invention. Detailed Implementation

[0020] To enable those skilled in the art to better understand the technical solutions of this disclosure, and to fully understand and implement the process of how this disclosure applies technical means to solve technical problems and achieve corresponding technical effects, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, not all embodiments. The embodiments of this disclosure and the various features within them can be combined with each other without conflict, and the resulting technical solutions are all within the protection scope of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort should fall within the protection scope of this disclosure.

[0021] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0022] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0023] Before explaining the implementation of this solution, we must first describe the novel backdoor it targets. This novel backdoor typically employs dynamic spatial triggers, utilizing the natural spatial interaction features between objects to trigger malicious behavior. Dynamic spatial triggers are triggers designed based on the spatial position and temporal changes of objects in an image. Therefore, this novel backdoor differs from traditional static trigger backdoors; it can remain effective at multiple time points or under different environments, maintaining the normal performance of the object detection model while inducing malicious behavior such as misclassification of the target. It should be noted that this backdoor targets object detection models using the Non-Maximum Suppression (NMS) mechanism. NMS is a post-processing technique commonly used in object detection, aiming to select the optimal bounding box from multiple overlapping candidate bounding boxes. This method calculates the Intersection over Union (IoU) ratio between different candidate boxes, retains the best box based on the confidence score, and removes redundant boxes to obtain the final detection result.

[0024] Based on the aforementioned novel backdoor, the first embodiment of this invention relates to a backdoor risk detection method applied to a target detection model, such as... Figure 1 As shown, the method in this embodiment includes the following steps:

[0025] Step 101: Test the target detection model using a pre-built image sample set and output the test results, which include the detection categories of target objects marked by candidate boxes with non-maximum suppression in the image samples.

[0026] Specifically, since the core triggering condition of the new backdoor is the overlap of specific targets, in order to detect the new backdoor, the image sample set needs to cover various images of normal scenes and potential triggering scenes (including target overlap). Specific sample sources can include real-scene acquisition and simulation generation, ensuring sample diversity, i.e., covering different scenes, different target categories (such as vehicles, pedestrians, road signs, etc.), and different numbers of targets. The differences exhibited by the target detection model during image sample detection are used to capture misclassification behavior after backdoor triggering. Furthermore, since the target detection model in this embodiment operates based on the NMS mechanism, the test results are reflected as the candidate box detection categories after NMS processing. This is because NMS is a standard post-processing step for target detection models: the multiple candidate boxes generated by the target detection model contain a large amount of redundancy (multiple boxes corresponding to the same target). NMS calculates the confidence score and intersection-union ratio (IOU) of the candidate boxes, retaining the candidate boxes with the highest confidence score and low overlap with other boxes, and finally outputs a unique detection result for each target. The output detection category is the model's final judgment of the target, directly reflecting the model's classification decision and serving as the core basis for judging whether the model is functioning correctly.

[0027] The pre-built image sample set is input into the target detection model in batches. The model automatically completes feature extraction and candidate box generation. NMS post-processing is performed to filter out the final candidate boxes. The detection category (i.e., the classification label output by the model) corresponding to each final candidate box is extracted, and its corresponding image sample ID is associated with the coordinates of the final candidate box to form a structured test result set of sample ID-detection candidate box-detection category for storage.

[0028] Step 102: Count the number of targets in image samples where the detected category of the target object is inconsistent with the true category, and calculate the proportion of the number of targets relative to the total number of image samples in the image sample set.

[0029] Specifically, after sufficient training, normal object detection models typically achieve a classification accuracy of ≥95% for targets in a reasonable sample set, with misclassifications mostly being randomly distributed, such as accidental errors caused by image blurring or occlusion. However, models with novel backdoors exhibit concentrated misclassifications in samples containing overlapping specific targets, leading to a significant increase in the number of samples with inconsistent detected and true categories. Therefore, this embodiment quantifies the abnormality of the model by statistically analyzing the number of such inconsistent targets and calculating their proportion.

[0030] In the specific statistical analysis, for each image sample, the intersection-union ratio (IU / U) of the bounding boxes is used to match the detected bounding boxes with the ground truth bounding boxes. The IU / U of the candidate bounding boxes (detected boxes) after NMS in the test results is calculated, and if the IU / U is greater than a threshold, they are considered the same target. For successfully matched targets, their detected category is compared with the ground truth category. If the labels are inconsistent (e.g., the ground truth category is "pedestrian" while the detected category is "vehicle"), they are considered inconsistent targets, and the target count is incremented. This count is used to determine the number of image samples where the detected category and ground truth category are inconsistent. If a detected bounding box does not match any ground truth bounding box, or a ground truth bounding box is not matched by any detected bounding box, no clear classification result is generated, and it is not included in the statistics. Based on the ratio of the number of targets to the total number of image samples in the image sample set, the backdoor risk percentage is obtained (e.g., if the sample set has 5000 images and the total number of inconsistent targets is 300, the percentage is 6%).

[0031] Step 103: Determine whether the target detection model has a backdoor risk based on the proportion.

[0032] Specifically, the misclassification rate of a normal object detection model has a defined error range, while a model with a novel backdoor, due to the presence of the "backdoor," will cause triggered misclassification, and its rate will significantly exceed this error range. By setting a reasonable risk threshold, it is possible to distinguish between normal errors caused by model performance and abnormal errors caused by backdoor attacks. When the rate exceeds the threshold, it indicates that the model exhibits concentrated misclassification behavior, which meets the triggering characteristics of the novel backdoor, and is therefore judged to have backdoor risk; otherwise, it is judged to be risk-free.

[0033] The calculated proportion of the model to be detected is compared with a risk threshold. If the proportion is not less than the threshold, the target detection model is determined to have a novel backdoor risk; if the proportion is less than the threshold, it is determined to have no backdoor risk. Furthermore, to improve the detection accuracy of this embodiment, additional judgment conditions can be added: if the proportion is close to the threshold, it is statistically analyzed whether inconsistent targets are concentrated in samples with overlapping targets. If more than 80% of inconsistent targets come from overlapping samples, it is directly determined to be high risk; if inconsistent targets are randomly distributed in non-overlapping and overlapping samples, it is determined to be "suspected risk," requiring more samples for repeated testing. This embodiment does not impose specific limitations on the size of the risk threshold.

[0034] Compared with related technologies, this embodiment proposes a detection method to address the hidden security threat posed to target detection models by novel backdoors triggered by overlapping target objects. The method tests the target detection model by pre-constructing an image sample set covering normal and potential triggering scenarios, focusing on the final detection category of candidate boxes after non-maximum suppression. It statistically analyzes the proportion of image samples whose detected category differs from the true category and quantifies the backdoor risk based on this proportion. Overall, it achieves accurate identification of this type of novel hidden backdoor, effectively overcoming the coverage blind spots and subjective bias problems of traditional detection methods for novel backdoors. It does not rely on the model's internal structure or modify the design of the model's core architecture and parameters, and can flexibly adapt to various CNN-based target detection models and safety-critical application scenarios such as autonomous driving and video surveillance.

[0035] The second embodiment of this application relates to a backdoor risk detection method applied to an object detection model. This embodiment is a supplement to the foregoing embodiment, and the supplement is that the process of constructing an image sample set is described.

[0036] In one example, the image sample set includes multiple original samples and at least one mapped sample corresponding to each original sample;

[0037] Based on this, such as Figure 2 As shown, the steps for constructing an image sample set include:

[0038] Step 201: Based on the recognition categories supported by the target detection model, generate multiple original samples that each contain at least two recognition categories, and designate one recognition category as the pre-trigger category and the other recognition category as the pre-victim category.

[0039] In this embodiment, there is no restriction on the actual type of object detection model. However, before constructing the image sample set, it is necessary to know the specific type of object detection model and determine the recognition categories it supports. For example, the YOLO series uses the COCO dataset for pre-training and supports 80 categories; Faster R-CNN supports 80 categories in standard configuration; and SSD supports 20 categories (PASCALVOC) or 80 categories (COCO).

[0040] Specifically, because attackers of new backdoors typically set a specific category as a "trigger" and another category as a "victim," misclassification is triggered when the two overlap. The trigger depends on the overlap between the trigger target and the victim target. Single-category samples cannot form an overlapping relationship and cannot simulate potential triggering scenarios.

[0041] Therefore, in order to find the "trigger" and "victim" as backdoors among the various recognition categories supported by the target detection model, this embodiment proposes pre-trigger categories and pre-victim categories among the recognition categories supported by the target detection model. By pre-defining the two categories, samples containing combinations of these categories are generated in a targeted manner.

[0042] The specific generation can be achieved as follows: First, identify all recognition categories supported by the target detection model (e.g., C1, C2, ..., Cn), generating all possible binary category combinations (Ci, Cj) (i≠j). Ensure that each combination has at least one original sample. The original samples must cover the core application scenarios of the model. In each sample, the trigger target corresponding to the pre-triggered category and the victim target corresponding to the pre-victim category must be in a state of no pixel overlap, and their positions in the image must be evenly distributed. The original samples can be obtained through a combination of real-scene shooting and synthetic generation. Based on the principle of full combination coverage, formulate pre-triggered category and pre-victim category pairs. For example, if the model supports four categories: "vehicles, pedestrians, road signs, and backpacks," then formulate all binary combinations such as (vehicles, pedestrians), (vehicles, road signs), (vehicles, backpacks), and (pedestrians, road signs) to ensure that the sample set covers the category pairings of "trigger-victim" that the attacker might design.

[0043] Step 202: Identify the triggering targets corresponding to the pre-triggered categories and the victim targets corresponding to the pre-victim categories in the original samples.

[0044] Specifically, each target object in the original samples is labeled using industry-standard annotation tools. The annotation information for each sample includes: original category label, target bounding box coordinates, target center coordinates, etc. Then, based on the annotation information, the bounding box, center coordinates, and pixel mask of the triggering target corresponding to the pre-triggered category in each original sample are automatically extracted to clarify the pixel range of the triggering target; similarly, the bounding box, center coordinates, and pixel mask of the victim target corresponding to the pre-victim category are extracted.

[0045] Step 203: Control one of the trigger target and the victim target to remain stationary, while the other moves gradually from its initial position with a preset expansion rate and / or a preset displacement rate, generating multiple mapping samples in which the trigger target and the victim target have different degrees of pixel overlap.

[0046] The scaling factor is a parameter used to control the size of the bounding box during backdoor tag generation. It refers to the scaling ratio of the triggered target's size, used to simulate the visual size change when the target approaches. The displacement factor is a parameter used to control the spatial displacement between the bounding box of the triggered target and the bounding box of the victim object during backdoor tag generation. By adjusting the displacement factor, the position of the fake bounding box relative to the real object's bounding box can be controlled, such as moving it up, down, left, or right.

[0047] Specifically, the system retrieves the trigger target and victim target information from the original sample, adjusts the size of the trigger target or victim target based on a preset expansion rate, and generates an expanded trigger target mask or victim target mask. Based on a preset displacement rate and movement path, it calculates the new center coordinates of the trigger target or victim target to ensure that the trigger target or victim target does not exceed the image boundary after movement. The adjusted trigger target mask or victim target mask is then superimposed onto the corresponding position of the original sample to generate a mapping sample with pixel overlap. The degree of overlap is calculated using IOU, and the IOU value of the mapping sample is labeled. Multiple mapping samples are generated for each original sample, covering the overlap range of IOU=0.1-0.9, fully covering the possible trigger threshold range of the backdoor. That is, multiple mapping samples with varying degrees of pixel overlap between the trigger target and the victim target are generated to ensure that each overlap gradient has sufficient sample support.

[0048] In this case, step 101, specifically: testing the target detection model using a pre-built image sample set and outputting the test results, means:

[0049] For each original sample, all corresponding mapped samples are input into the target detection model one by one, and the detection category of the victim target in the mapped sample is output.

[0050] Specifically, based on the matching relationship between the original samples and the mapped samples, the mapped samples in the image sample set are sorted according to the ID of the original samples. Each mapped sample is then input into the target detection model to avoid errors in result association caused by disordered sample order. The target detection model outputs all NMS-predicted candidate boxes (including detection category, bounding box, and confidence score) for each mapped sample. Based on the annotation information of the original samples corresponding to the mapped samples, the true bounding box of the victim target is determined. By calculating the Intersection over Union (IOU) between the candidate box and the true bounding box, the candidate box corresponding to the victim target detected by the model is matched. The detection category corresponding to the successfully matched candidate box is extracted as the detection result of the victim target in that mapped sample. The test results are structured and stored according to the correspondence of "original sample ID, mapped sample number, overlap IOU value, true victim target category, victim target detection category, and detection confidence score" to facilitate subsequent target count statistics.

[0051] Compared with related technologies, the image sample set construction process in this embodiment is compatible with all categories and application scenarios of target detection models. The testing process does not require modification of the model structure and is applicable to various target detection models with NMS post-processing. The mapped samples are generated based on the original samples, which reduces the workload of sample collection and annotation and lowers the detection cost. The gradient design of the preset expansion rate and displacement rate can be flexibly adjusted according to newly discovered backdoor triggering modes. The sample set can be dynamically updated, which has strong universality and high flexibility.

[0052] The third embodiment of the present invention relates to a backdoor risk detection method applied to an object detection model. This embodiment is a supplement to the foregoing embodiments, and the supplement is that when the object detection model is a white-box model, an alternative process for constructing an image sample set is provided.

[0053] like Figure 3 As shown, the method for pre-constructing the image sample set in this embodiment includes:

[0054] Step 301: Based on the recognition categories supported by the target detection model, generate multiple image samples that each contain at least two recognition categories, and designate one recognition category as a pre-trigger category and the other recognition category as a pre-victim category to form a test pair.

[0055] Step 302: Each image sample contains at least one test pair, and all image samples cover all test pairs formed by pairwise combinations of all pre-triggered categories and all pre-victim categories.

[0056] The core advantage of white-box models over black-box models lies in their ability to obtain complete category definitions, parameter configurations, and inference logic. This approach leverages this advantage to construct a recognition system that perfectly aligns with the model's design. First, all supported recognition categories are extracted using the white-box model's configuration file, training logs, or model structure analysis tools, forming a category list C=[C1,C2,...,Cn]. Based on this category list, all pairwise test pairs are generated, with each image sample containing at least one test pair (Ci,Cj). It's important to note that to improve testing efficiency, multiple test pairs can be set in a single image sample, ensuring the target object's size can be effectively recognized by the model. For example, if category A is fixed as the pre-trigger category, each category supported by the target detection model (excluding category A) is sequentially used as a pre-victim category to generate test pairs with category A. All these test pairs are set in the same image sample. The model can then identify whether category A poses a potential "trigger" risk by recognizing the image. Using this method, if the model supports 80 categories, only 80 tests are needed.

[0057] Based on this, step 101 of this embodiment uses a pre-constructed image sample set to test the target detection model and outputs the test results, including:

[0058] Lower the non-maximum suppression threshold corresponding to the target detection model;

[0059] The image samples are sequentially input into the target detection model to obtain the detection categories of all target objects, including the victim, in the image samples.

[0060] The core function of the NMS threshold is to filter redundant candidate boxes. Its value directly affects the number of candidate boxes output by the model. The original NMS threshold is generally high, which will filter out a large number of candidate boxes with high overlap. However, new backdoors may cause the model to generate multiple misclassified candidate boxes for the victim target (the confidence level may be slightly lower than the normal classification, but higher than the noise threshold). These candidate boxes may be filtered out by the original NMS threshold due to high overlap, causing the misclassification behavior triggered by the backdoor to be missed. Since white-box models allow direct adjustment of the threshold parameter of the internal NMS module, by lowering the threshold (e.g., 0.1-0.3), more candidate boxes can be retained, including misclassified candidate boxes caused by the backdoor, ensuring that the model's classification decisions for all identifiable objects are fully captured, and avoiding detection blind spots caused by excessively high thresholds.

[0061] Specifically, the default NMS threshold is first extracted from the parameter configuration file or code of the white-box model. The NMS threshold is then lowered according to actual needs. After the lowering, the number of candidate boxes output by the model for the same batch of samples is 1.5-3 times that of the lowered threshold, without a significant surge in invalid candidate boxes. This application does not limit the specific range of the lowering. After the NMS threshold is lowered, samples from the image sample set are input into the adjusted white-box model one by one according to the test pair category-sample number order. After the model completes feature extraction and candidate box generation, candidate boxes are filtered according to the lowered NMS threshold, and all candidate boxes with confidence scores ≥ the lowered NMS threshold are output. At this point, compared to the default confidence threshold, more potential abnormal candidate boxes can be retained. For each candidate box, its detection category, bounding box coordinates, and confidence value are extracted to complete a single-sample test.

[0062] Compared with related technologies, this embodiment leverages the known category system and adjustable parameters of the white-box model to achieve full coverage of pre-trigger-pre-victim category combinations in the sample set. The testing process optimizes the candidate box capture effect by adjusting the NMS threshold, solving the problem of incomplete detection caused by the difficulty in inferring category combinations and the inability to optimize parameters under the black-box model, and significantly improving the testing efficiency of backdoor triggering scenarios.

[0063] The fourth embodiment of the present invention relates to a backdoor risk detection method applied to a target detection model. This embodiment is a supplement to the foregoing embodiments, and the supplement is that the step of determining the proportion in step 102 is described in detail, including:

[0064] The proportion is determined based on the number of targets in the image samples corresponding to the victim targets whose detected category does not match the pre-victim category.

[0065] Specifically, the victim target is the core object of a backdoor attack, and the consistency between its detected category and the actual pre-victim category directly reflects whether the backdoor has been triggered. In a normal model, the classification errors of the victim target are mostly random and infrequent; however, in a model with a backdoor, under triggering conditions, the victim target will exhibit concentrated and high-frequency misclassifications. Therefore, selecting this type of target is key to accurately capturing backdoor anomalies, requiring two conditions to be met simultaneously: the target object must be identified as a "victim target" and the classification result must be abnormal (the detected category is inconsistent with the pre-victim category label), based on which the number of targets is determined.

[0066] Determining whether a target detection model has backdoor risks based on the proportion includes:

[0067] Determine if the percentage exceeds the risk threshold; if the percentage exceeds the risk threshold, determine that the target detection model has a backdoor risk.

[0068] Specifically, the proportion calculated in step 102 is compared with a preset risk threshold: if the proportion is greater than the preset risk threshold, it is directly determined that the target detection model has a novel backdoor risk; if the proportion is not greater than the preset risk threshold, it is determined that there is no backdoor risk. It should be noted that this embodiment can set an additional auxiliary judgment process for boundary scenarios close to the threshold: whether inconsistent victim targets are concentrated in the trigger condition samples. If inconsistent victim targets exceeding the preset threshold come from the trigger condition samples, it is determined as "high suspected backdoor risk," and additional samples need to be added for the victim target and the corresponding trigger target for repeated testing. It should be noted that the risk threshold in this embodiment can be set according to actual needs, and can be set to several times the model's basic error rate; this embodiment does not impose specific limitations on this.

[0069] Compared with related technologies, the threshold calibration in this embodiment combines the normal model baseline with the scene safety level, and dynamically adjusts it for different test scenarios (target overlap, white-box model), thus balancing detection accuracy and practicality.

[0070] The fifth embodiment of this application relates to a backdoor risk detection method applied to an object detection model. This embodiment is a supplement to the foregoing embodiments, the supplement being that: after determining that the object detection model has a backdoor risk, improvement measures are proposed, such as... Figure 4 As shown, this embodiment includes the following steps 401-404.

[0071] Step 401: For the same pre-victim category, determine the victim based on the correspondence between the detection category and the same pre-victim category in the test results, and take the victim's detection category as the misclassification category.

[0072] Specifically, the core characteristic of this new type of backdoor attack is that targets (victims) of the same pre-victim category are collectively misclassified into the same specific category under the action of a trigger, rather than being randomly misclassified. Therefore, for the same pre-victim category, by analyzing the correspondence between its detection category and the pre-victim category, the targets (i.e., victims) forcibly misclassified by the backdoor can be accurately identified.

[0073] First, extract all target data labeled "pre-victim category" from the previous test results and group them according to the pre-victim category. Each group of data includes key information such as: sample ID, pre-victim category, detection category, confidence level, and whether it overlaps with the triggering target. For each group of data in the same pre-victim category, calculate the "detection category consistency" of each target: if the detection category of a target is inconsistent with the pre-victim category, and there are records of "overlapping with the triggering target" in the sample of that target, then it is initially identified as a candidate victim. Further filter the target groups within the same group whose detection category repetition rate of candidate victims is greater than a threshold (i.e., candidate victims in the same pre-victim category are collectively misclassified into the same category), and finally determine this group as the official victims. Count all detection categories corresponding to the official victims, and take the detection category with the highest percentage and the threshold percentage as the misclassified category corresponding to the victim of that category. For example, among the victims in the pre-victim category "pedestrian", 85% of the detection category is "vehicle", then "vehicle" is the misclassified category of this group of victims.

[0074] Step 402: For the categories covered by the trigger targets corresponding to the same type of victims in the test results, select the category that meets the preset conditions as the trigger category of the backdoor.

[0075] Specifically, the trigger of the novel backdoor is essentially a "trigger target of a specific category," and its core characteristic is that it can stably trigger misclassification when overlapping with the victim, and the trigger categories corresponding to the same type of victim have a high degree of consistency. Therefore, after identifying the victim, the category corresponding to the trigger can be determined based on the relationship between the category distribution of trigger targets overlapping with the victim in the test results and the preset conditions. In this embodiment, the preset conditions can be the category containing the most trigger targets, or the category containing trigger targets whose proportion in the total number is greater than a preset threshold. That is, if the frequency of a certain category of trigger targets overlapping with the victim is extremely high, and the category has a high proportion among all targets overlapping with the victim, then that category is the trigger of the backdoor.

[0076] Step 403: Construct a corrected training set for the object detection model. The corrected training set includes: multiple corrected samples, which include at least the target object corresponding to the victim's true category, the misclassified category, and the trigger category.

[0077] Specifically, the training set samples must simultaneously contain three types of target objects: true class targets, misclassified class targets, and trigger class targets. It may also include target objects corresponding to categories supported by other object detection models, and must cover two key scenarios: independent scenarios (targets do not overlap, used to learn the basic features of each category); and trigger scenarios (trigger class targets overlap with true class targets, simulating backdoor triggering conditions, used to learn the feature differences among the three types of targets in overlapping scenarios). Each sample's targets are accurately labeled, including: true class label and bounding box coordinates.

[0078] Step 404: Train the external classifier using the corrected training set so that the external classifier can accurately distinguish the target object corresponding to the true category, misclassified category and trigger category in the image.

[0079] Specifically, the external classifier is a supplementary module independent of the original object detection model provided in this embodiment. The training objective of this external classifier is to accurately distinguish the recognition categories supported by the object detection model. When the original object detection model outputs detection results, if a target object is detected corresponding to any of the following categories: true category, misclassified category, and trigger category, the external classifier is invoked. The image in the NMS candidate box of the detection results output by the object detection model is extracted, and the extracted image is input into the external classifier for secondary recognition and classification. The external classifier identifies the true category of the target in the input extracted image and outputs it to correct or verify the misclassification caused by the backdoor.

[0080] The specific training process is as follows: the modified training set is divided into a training set and a validation set in a ratio of 8:2 / 7:3 to ensure that the scene and category distribution of the validation set are consistent with those of the training set; the cross-entropy loss function is used to train the external classifier; the AdamW optimizer is selected, the initial learning rate is set to 1e-4, and a cosine annealing learning rate scheduling strategy is adopted; the number of training iterations is set to 100 rounds, and Dropout and L2 regularization are introduced to prevent overfitting.

[0081] The classification accuracy of the target detection model is the core indicator. The external classifier is validated using a validation set, and the final classification accuracy is required to be ≥99%. The samples with misclassification in the validation set are analyzed. If the errors are concentrated in "high overlap triggering scenarios", a certain number of current scene samples are added for retraining. If the errors are concentrated in a certain type of target (such as misclassified categories that are visually similar to the true categories), the feature diversity of the samples is expanded through data augmentation (such as rotation, flipping, and noise addition) until the evaluation index requirements are met.

[0082] Compared with related technologies, this embodiment, after detecting that the current target detection model has a backdoor risk, adopts an external classifier correction method to correct the detection results of the target detection model. The external classifier specifically learns its feature differences, which can accurately resist the identified new backdoor attacks and ensure the accuracy of the victim's classification results in the triggered scenario. It does not require modification of the core architecture of the original target detection model, adapts to the model scenarios that have been deployed or are difficult to reconstruct, and has low training cost and flexible deployment, which can quickly make up for the backdoor security vulnerabilities of the original model.

[0083] The sixth embodiment of this application relates to a backdoor risk detection method applied to an object detection model. This embodiment is a supplement to the foregoing embodiments, and the supplement is that, for the case where the object detection model is a white-box model, after determining that the object detection model has a backdoor risk, another improvement measure is proposed, such as... Figure 5 As shown, this embodiment includes the following steps 501-504.

[0084] Step 501: Statistically analyze the target test pairs corresponding to the victim targets whose categories have changed in the test results, and determine the true category and misclassified category of the victim based on the distribution of the pre-triggered category and pre-victim category in the target test pairs.

[0085] Specifically, abnormal data where the detected victim category does not match the pre-victim category is extracted, and their corresponding test pairs (pre-trigger category - pre-victim category) are associated. These test pairs are defined as target test pairs. The selection process must meet the following criteria: overlap between the victim and trigger targets; normal misclassification data in non-trigger scenarios is excluded; and the target test pairs are strongly correlated with the backdoor trigger. For all abnormal victim targets corresponding to the same target test pair, the frequency distribution of their detected categories is statistically analyzed. Detection categories with a percentage greater than a threshold are taken as the misclassified categories of the true category. The "pre-victim category" in the target test pair is the true category of the victim.

[0086] Step 502: For the categories covered by the trigger targets corresponding to the same type of victims in the test results, select the category that meets the preset conditions as the trigger category of the backdoor.

[0087] Specifically, the trigger of the novel backdoor is essentially a "trigger target of a specific category," and its core characteristic is that it can stably trigger misclassification when overlapping with the victim, and the trigger categories corresponding to the same type of victim have a high degree of consistency. Therefore, after identifying the victim, the trigger can be determined based on the relationship between the category distribution of trigger targets overlapping with the victim in the test results and preset conditions. In this embodiment, the preset conditions can be the category containing the most trigger targets, or the category containing the proportion of trigger targets greater than a preset threshold as the trigger. That is, if the frequency of a certain category of trigger targets overlapping with the victim is extremely high, and the category accounts for a high proportion of all targets overlapping with the victim, then that category is the trigger of the backdoor.

[0088] For example, triggers can be determined through category consistency and trigger correlation analysis: First, calculate the proportion of each category of trigger targets in the filtered data; then calculate the number of times the trigger target of this category overlaps with the victim, resulting in misclassification / the total number of times the trigger target of this category overlaps with the victim; if a certain pre-triggered category meets the requirements of frequency proportion ≥ 75% and trigger success rate ≥ 90%, then it is determined as the backdoor trigger corresponding to the real victim of this category.

[0089] Step 503: Obtain sample images containing the true category, misclassified category, and trigger category, label the target objects in the sample images with the correct classification labels, and construct the hardened dataset.

[0090] Specifically, the samples in the hardened dataset must simultaneously contain three types of target objects: true class objects, misclassified objects, and trigger objects, covering two key scenarios: independent scenarios (the three types of objects do not overlap, used to learn the basic features of each category); and trigger scenarios (trigger objects overlap with true class objects, simulating backdoor triggering conditions, used to reinforce learning the feature differences of the three types of objects in overlapping scenarios). Each sample's three types of objects are accurately labeled, including: true class label and bounding box coordinates.

[0091] Step 504: Based on the hardened dataset, the object detection model is retrained using an augmentation training algorithm to adjust the object detection model's ability to distinguish between target objects corresponding to the true category and the misclassified category.

[0092] Specifically, the core advantage of the white-box model is that it has access to model parameters, gradient information, and training framework. Based on this characteristic, this application enhances training by introducing a hardened dataset as an "adversarial perturbation sample" during the training process, forcing the model to learn robust features in the attacked scenario, thus offsetting the influence of parameters tampered with by the backdoor. For the "trigger-true category-misclassified category" association of the backdoor, the enhancement training can specifically strengthen the model's ability to correctly classify the associated scenario, so that the model can accurately identify the true category even when the trigger and the true category overlap, rather than being forced to misclassify by the backdoor.

[0093] The specific secondary training scheme is as follows: The original training framework and hardware environment of the white-box model (object detection model) are retained, and the parameters of the original model are loaded as initial weights. A targeted augmentation training process is constructed for three types of samples: trigger, victim, and target. First, image samples containing the above three categories are selected from the original training set, and their coverage relationships at different scales, viewpoints, and backgrounds are increased manually or automatically (e.g., IoU is close but does not reach the trigger threshold, or partial occlusion is present), to simulate various potential overlapping scenarios and construct a hardened dataset. Subsequently, the augmented samples in the hardened dataset are fine-tuned using the original model structure. The loss function still adopts the standard detection loss (classification loss + regression loss) of the detection model, and higher sampling weights are applied to the three key categories of trigger, victim, and target during training, so that the model can more robustly maintain the correct confidence ranking when facing scenarios where trigger and victim classes co-occur. Training is stopped and the final hardened model parameters are saved when the model meets the following criteria: "Triggered scene classification accuracy ≥ 99%, normal scene classification accuracy ≥ 99% (matching or improving upon the original model), and enhanced sample classification accuracy ≥ 99%". 1000 test samples containing "true category - trigger - misclassified category" are selected and input into the hardened white-box model to verify the backdoor trigger rate. The model hardening effect is considered successful when the trigger rate is less than the threshold (5%) and the classification accuracy of normal targets does not significantly decrease.

[0094] Compared to related technologies, this embodiment leverages the transparent nature of the white-box model to directly optimize its core parameters through enhanced training, fundamentally mitigating the malicious impact of backdoors. Compared to the indirect defense of external modules, this approach provides more thorough hardening without requiring additional deployment costs. Simultaneously, the training process utilizes the original model framework and initial weights, ensuring that the hardened model's normal detection performance remains intact. It adapts to already deployed white-box model scenarios, guaranteeing the efficiency and stability of the hardening process. The hardened target detection model can effectively resist backdoor attacks, resulting in accurate detection and classification results. While maintaining detection efficiency and the original model's normal performance, it comprehensively enhances the security and reliability of the target detection model under malicious backdoor attack environments.

[0095] The seventh embodiment of the present invention relates to a backdoor risk detection system, such as... Figure 6 As shown, it includes:

[0096] The data acquisition module 601 is used to test the target detection model using a pre-built image sample set and output the test results. The test results include the detection category of the target object marked by the candidate box with non-maximum suppression in the image sample.

[0097] The data analysis module 602 is used to count the number of targets in image samples whose detected category is inconsistent with the true category, and to calculate the proportion of the number of targets relative to the total number of image samples in the image sample set.

[0098] Risk assessment module 603 is used to determine whether the target detection model has backdoor risks based on the proportion.

[0099] The backdoor risk detection system in this embodiment further includes: a model hardening module. This module is used to determine victims for the same type of pre-victim category based on the correspondence between the detected category and the same type of pre-victim category in the test results, and to designate the detected category of the victim as a misclassified category; based on the category consistency between the triggering targets overlapping with the victim in the test results, to determine the triggering target corresponding to the same type of victim as a backdoor trigger; to construct a corrected training set for the target detection model, the corrected training set including: multiple corrected samples, the multiple corrected samples containing the true category of the victim, the misclassified category, and the target object corresponding to the category of the trigger; and to train an external classifier using the corrected training set so that the external classifier accurately distinguishes the true category, the misclassified category, and the target object corresponding to the category of the trigger in the image.

[0100] The backdoor risk detection system in this embodiment further includes a secondary training module, which operates in scenarios where the target detection model is a white-box model. The secondary training module is used to statistically analyze target test pairs corresponding to the victim targets whose categories have changed in the test results; determine the true category and misclassified category of the victim based on the distribution of the pre-triggered category and the pre-victim category in the target test pairs; determine the trigger target corresponding to the same type of victim as the trigger of the backdoor based on the category consistency between the trigger targets overlapping with the victim in the test results; acquire sample images containing the true category, the misclassified category, and the category of the trigger; label the target objects in the sample images with the correct classification labels to construct a reinforced dataset; and perform secondary training on the target detection model using an augmentation training algorithm based on the reinforced dataset to enhance the target detection model's ability to distinguish target objects corresponding to the true category and the misclassified category.

[0101] It is not difficult to see that this embodiment is a system embodiment corresponding to the aforementioned method embodiments, used to implement the methods described in any of the above embodiments. The steps and implementation details described in the above method embodiments are still applicable in this embodiment, and will not be repeated here.

[0102] The eighth embodiment of the present invention relates to an electronic device, such as... Figure 7 As shown, it includes at least one processor 702; and a memory 701 communicatively connected to at least one processor 702; wherein the memory 701 stores instructions executable by at least one processor 702, the instructions being executed by at least one processor 702 to enable at least one processor 702 to execute any of the above method embodiments.

[0103] The memory 701 and processor 702 are connected via a bus, which can include any number of interconnecting buses and bridges. The bus connects various circuits of one or more processors 702 and memory 701. The bus can also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. A bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by processor 702 is transmitted over a wireless medium via an antenna, which further receives data and transmits it to processor 702.

[0104] The processor 702 is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. The memory 701 can be used to store data used by the processor 702 during operation.

[0105] The ninth embodiment of the present invention relates to a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements any of the above-described method embodiments.

[0106] The tenth embodiment of the present invention relates to a computer program product, including computer instructions that, when executed by a processor, implement any of the above-described method embodiments.

[0107] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0108] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0109] While the embodiments disclosed herein are as described above, the foregoing content is merely for the purpose of facilitating understanding of this disclosure and is not intended to limit this disclosure. Any person skilled in the art to which this disclosure pertains may make any modifications and changes in form and detail of the implementation without departing from the spirit and scope of this disclosure; however, the scope of patent protection of this disclosure shall still be determined by the scope defined in the appended claims.

Claims

1. A method for backdoor risk detection applied to a target detection model, characterized in that, include: The target detection model is tested using a pre-built image sample set and the test results are output. The test results include the detection categories of target objects marked by candidate boxes with non-maximum suppression in the image samples. The number of image samples in which the detected category of the target object is inconsistent with the true category is counted, and the proportion of the number of targets relative to the total number of image samples in the image sample set is calculated. Based on the stated proportion, determine whether the target detection model has a backdoor risk; The backdoor uses one of the recognition categories supported by the target detection model as a trigger and the other as a victim, and is triggered based on the spatial overlap between objects of the trigger category and objects of the victim category in the target detection scene. The image sample set includes multiple original samples and at least one mapped sample corresponding to each original sample; The method for pre-constructing the image sample set includes: Based on the identification categories supported by the target detection model, multiple original samples are generated, each containing at least two of the identification categories, and one of the identification categories is designated as a pre-trigger category, and the other identification category is designated as a pre-victim category. Identify the triggering target corresponding to the pre-triggering category and the victim target corresponding to the pre-victim category in the original sample; One of the trigger target and the victim target is kept stationary, while the other moves gradually from its initial position with a preset expansion rate and / or a preset displacement rate, generating multiple mapping samples in which the trigger target and the victim target have different degrees of pixel overlap.

2. The method of claim 1, wherein, The process of testing the target detection model using a pre-built image sample set and outputting test results includes: For each original sample, all the mapped samples are input into the target detection model one by one, and the detection category of the victim target in the mapped sample is output.

3. The method of claim 1, wherein, The target detection model is a white-box model; The method for pre-constructing the image sample set includes: Based on the recognition categories supported by the target detection model, multiple image samples are generated, each containing at least two of the recognition categories. One of the recognition categories is designated as a pre-trigger category, and the other recognition categories are designated as pre-victim categories to form a test pair. Each of the image samples contains at least one of the test pairs, and all the image samples cover all test pairs formed by pairwise combinations of all the pre-triggered categories and all the pre-victim categories.

4. The method of claim 3, wherein, The process of testing the target detection model using a pre-built image sample set and outputting test results includes: Lower the non-maximum suppression threshold corresponding to the target detection model; The image samples are sequentially input into the target detection model to obtain the detection categories of all target objects, including the victim target, contained in the image samples.

5. The method according to claim 2 or 4, characterized in that, The steps for determining the percentage include: The proportion is determined based on the number of targets corresponding to the image samples to which the victim targets belong, where the detection category is inconsistent with the pre-victim category; Determining whether the target detection model has a backdoor risk based on the stated proportion includes: Determine whether the percentage is greater than the risk threshold; When the percentage is greater than the risk threshold, the target detection model is determined to have a backdoor risk.

6. The method according to claim 2 or 4, characterized in that, After determining that the target detection model has a backdoor risk, the method further includes: For the same type of pre-victim category, the victim is determined based on the correspondence between the detection category and the same type of pre-victim category in the test results, and the detection category of the victim is taken as the misclassification category; For the categories covered by the triggering targets corresponding to the same type of victims in the test results, select the categories that meet the preset conditions as the trigger categories of the backdoor; Construct a corrected training set for the target detection model. The corrected training set includes: multiple corrected samples, and the multiple corrected samples include at least the true category of the victim, the misclassified category, and the target object corresponding to the trigger category. The extrinsic classifier is trained using the modified training set so that it can accurately distinguish the target object corresponding to the true category, the misclassified category, and the trigger category in the image.

7. The method according to claim 4, characterized in that, After determining that the target detection model has a backdoor risk, the method further includes: The target test pairs corresponding to the victim targets whose categories have changed in the test results are statistically analyzed, and the true category and misclassification category of the victim are determined based on the distribution of the pre-triggered category and the pre-victim category in the target test pairs; For the categories covered by the triggering targets corresponding to the same type of victims in the test results, select the categories that meet the preset conditions as the trigger categories of the backdoor; Obtain sample images containing the true category, the misclassified category, and the trigger category; label the target objects in the sample images with the correct classification labels; and construct a reinforced dataset. Based on the hardened dataset, the target detection model is retrained using an augmentation training algorithm to adjust its ability to distinguish between target objects corresponding to the true category and the misclassified category.

8. A backdoor risk detection system, characterized in that, include: The data acquisition module is used to test the target detection model using a pre-built image sample set and output the test results, which include the detection category of the target object marked by the candidate box with non-maximum suppression in the image sample. The data analysis module is used to count the number of image samples in which the detected category of the target object is inconsistent with the true category, and to calculate the proportion of the number of targets relative to the total number of image samples in the image sample set; The risk assessment module is used to determine whether the target detection model has a backdoor risk based on the proportion; the backdoor is triggered by one of the recognition categories supported by the target detection model as the trigger and the other as the victim, based on the spatial overlap relationship between the objects of the trigger category and the objects of the victim category in the target detection scene. The image sample set includes multiple original samples and at least one mapped sample corresponding to each original sample; The image sample set is pre-constructed, including: Based on the identification categories supported by the target detection model, multiple original samples are generated, each containing at least two of the identification categories, and one of the identification categories is designated as a pre-trigger category, and the other identification category is designated as a pre-victim category. Identify the triggering target corresponding to the pre-triggering category and the victim target corresponding to the pre-victim category in the original sample; One of the trigger target and the victim target is kept stationary, while the other moves gradually from its initial position with a preset expansion rate and / or a preset displacement rate, generating multiple mapping samples in which the trigger target and the victim target have different degrees of pixel overlap.

9. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the backdoor risk detection method applied to the target detection model as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the backdoor risk detection method applied to the target detection model as described in any one of claims 1 to 7.

11. A computer program product, characterized in that, It includes computer instructions that, when executed by a processor, implement the backdoor risk detection method applied to a target detection model as described in any one of claims 1 to 7.

Images