Adaptive frequency domain adversarial training method and device for object detector

Abstract

The application belongs to the field of computer vision and artificial intelligence security, and discloses a kind of adaptive frequency domain confrontation training method and device of target detector, and target detector includes the repair module and pedestrian detector in series, and the input of repair module is connected with the output of patch detector;Adaptive frequency domain confrontation training method includes loss in joint training, including standard target detection loss, repair consistency loss based on frequency domain image and clean image on frequency domain corresponding to training image and repair dependent loss based on average precision mean of detection;The application carries out end-to-end joint training with subsequent pedestrian detector through repair module, and the optimization target of repair module is directly aligned with the improvement of detection robustness, so that the semantic information critical to the detection task is maximally retained while the adversarial perturbation is eliminated as much as possible, and the disconnection between repair module and pedestrian detector performance is avoided, and the detection robustness is improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computer vision and artificial intelligence security, and particularly relates to an adaptive frequency domain adversarial training method and device for a target detector. BACKGROUND

[0002] The successful application of deep learning models in many key fields is in sharp contrast to their vulnerability when faced with malicious attacks. Adversarial patches, which exist in the form of local pasting, can effectively mislead deep learning models to produce false outputs by intentionally introducing disturbances that are difficult for human vision to detect, thereby posing a serious threat to the security deployment of models.

[0003] Unlike disturbance methods that require global fine-tuning, adversarial patches have the characteristics of locality, strong attackability, and high flexibility, making them a highly covert and efficient attack means in the digital space. With the continuous development of adversarial patch generation technology, its attack success rate and transferability continue to improve, which makes it particularly urgent to study and counteract the defense system. Therefore, evaluating and improving the robustness of intelligent systems in the physical world has become a major issue with great practical significance.

[0004] Existing defense methods mainly revolve around two major ideas: one is adversarial training, which attempts to let the model "get used to" this interference by introducing image samples containing adversarial patches during model training, thereby enhancing the model's own robustness to this adversarial attack. However, it often leads to over-adaptation of the model to the attack patterns seen during training, insufficient generalization to unknown variants, and potential damage to the model's original accuracy on clean data. The second is input purification, which attempts to detect and repair or remove the adversarial patch area before the image is input into the model. Common methods include image filtering, compression, or generative model reconstruction. However, such methods often face a dilemma: either the purification is not thorough, leaving residual attack effects, or the purification is too excessive, damaging the original key semantic features of the image (such as object edges), also leading to a decline in detection performance. More fundamentally, existing methods mostly view "adversarial training" and "input purification" as two independent and static links, lacking a mechanism for their coordinated evolution and adaptive adjustment when facing dynamic threats. This limitation makes it extremely difficult to build a robust and flexible defense system, highlighting the urgency of developing a new type of coordinated defense framework. SUMMARY

[0005] The purpose of the present application is to provide an adaptive frequency domain adversarial training method and device for a target detector, which jointly trains the repair module and the subsequent pedestrian detector in an end-to-end manner, avoiding the disconnection between repair and model performance.

[0006] The adaptive frequency domain adversarial training method of the target detector comprises the following steps:

[0007] The patch detector is trained through the training image with the adversarial patch and the patch position label of the adversarial patch in the training image;

[0008] The patch position in the training image is detected based on the trained patch detector and input into the repair module; the repair module is based on the patch position to repair the training image and then sent to the pedestrian detector to obtain the pedestrian detection result; the repair module and the pedestrian detector are jointly trained based on the pedestrian detection result and the real pedestrian label in the training image;

[0009] The loss in the joint training includes a standard target detection loss, a repair consistency loss based on the repair of the frequency domain image corresponding to the training image and the clean image in the frequency domain, and a repair dependent loss based on the average precision mean value of detection.

[0010] The beneficial effects of the present application are: the present application performs end-to-end joint training of the repair module and the subsequent pedestrian detector, aligns the optimization target of the repair module directly with the improvement of detection robustness, thereby eliminating the adversarial disturbance as much as possible while retaining the semantic information critical to the detection task to the maximum extent, avoiding the disconnection between the repair module and the pedestrian detector, and improving the detection robustness. BRIEF DESCRIPTION OF DRAWINGS

[0011] Figure 1 The principle diagram of the adaptive frequency domain adversarial training method of the target detector of the embodiment of the present application;

[0012] Figure 2 The analysis positioning schematic diagram of the input training image, the low frequency difference and the high frequency difference in the verification embodiment of the present application;

[0013] Figure 3 The frequency domain vulnerability and patch position schematic diagram in the verification embodiment of the present application;

[0014] Figure 4 The comparison schematic diagram of the detection result of the clean image under the original YOLOv5s detector and the real value in the verification embodiment of the present application;

[0015] Figure 5 The detection result schematic diagram of the training image after adding the adversarial patch in the verification embodiment of the present application;

[0016] Figure 6A detection result diagram of a pedestrian detector trained by the method of the present application in a verification example of the present application. DETAILED DESCRIPTION

[0017] The present application will be described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0018] The present application belongs to the field of computer vision and artificial intelligence security, and specifically relates to a dynamic adaptive defense system capable of responding to adversarial patch attacks in the physical world. The present application aims to systematically build a defense system for detecting and repairing adversarial patches and ensuring the robustness of the model itself, with the core purpose being to explore the universal structural features of adversarial patches in the digital space, especially their own information in the frequency domain and other transformation spaces, so as to establish a new method for locating adversarial patches that is not dependent on specific attack priors and is robust.

[0019] On this basis, the present application will focus on how to perform high-quality semantic reconstruction on the damaged image area after accurately removing the patch disturbance, in order to ensure the reliability of subsequent model reasoning. And on the basis of detection and repair, the model is also subjected to adversarial training, so that the combination of detection and repair and adversarial training is finally achieved, and the establishment of the defense system aims to make up for the shortcoming of existing research that usually regards adversarial training and reasoning phase defense as two decoupled processes, and provides a new technical path and solution for improving the actual robustness of deep learning models in an open environment.

[0020] Current adversarial training methods are difficult to adapt to new attack patterns that appear after deployment, and input purification means often damage key semantic information of images due to a single processing strategy, and these two types of defense modules are usually isolated from each other and cannot form a complete defense system with synergistic effect. Therefore, the present application proposes a unified framework named "frequency domain adaptive closed-loop double defense". The core content of the framework is as follows Figure 1As shown, first, the input image is quantified in the frequency domain by wavelet transform (i.e. frequency domain conversion) and gradient analysis to quantify the vulnerability of different subbands, and adaptive frequency domain alignment of different frequency bands is realized according to the vulnerability to detect patch positions, that is, the patch detector detects the adversarial image after adversarial training, so as to obtain the patch position; second, a two-stage collaborative architecture is constructed, which deeply integrates the frequency domain perception of the pedestrian detector robustness enhancement and the repair module, that is, the repaired image after the repair module is sent to the pedestrian detector for adversarial training; at the same time, the input image is directly introduced into the pedestrian detector to realize deep coupling and mutual promotion between repair effectiveness and robustness; finally, a closed-loop feedback regulation mechanism is introduced, which can dynamically optimize the repair strategy and model parameters according to real-time performance evaluation, thereby realizing continuous and efficient protection against diversified and evolving adversarial patch attacks. The method is suitable for deep learning model protection in safety-critical scenarios such as autonomous driving, face recognition, medical imaging, etc.

[0021] In the task of target detection based on deep learning, adversarial patches are a highly realistic threat form of attack. It is usually manifested as a carefully designed, printable and pasted local pattern on the surface of an object in the physical world. Unlike perturbations that aim to make the entire image difficult to detect, the mechanism of adversarial patches is to interfere with the strong signal of this local area, causing the target detector to produce serious misjudgment or missed detection of the object (even the entire image area).

[0022] For example, in the autonomous driving scenario, a small patch pasted on a pedestrian may cause the pedestrian detector to completely ignore the existence of the pedestrian, thereby causing a fatal safety accident. Therefore, the defense against adversarial patches is crucial to ensure the safe and reliable operation of key visual systems.

[0023] The closest implementation to the present application is a published patent application: a method for detecting and defending against adversarial patches, application number CN202310448084.4. The core of this scheme is to provide two optional and independent detection and defense paths, and to use a series open-loop system architecture of "repair after detection". The first method is based on abnormal positioning: first, train an encoder-decoder structure of a generative adversarial network using a data set containing only clean images, so that it learns the data distribution and reconstruction of clean images; then, input the image to be detected into the network to obtain the reconstructed image; then, calculate the pixel-level absolute error between the input and the reconstructed image, and use a pre-set fixed threshold to determine the area with larger error as the position of the adversarial patch to generate a binary mask; finally, according to the mask, the corresponding area of the original image is blackened or an independent image repair algorithm is called to process. The second method is based on edge detection: first, set the area ratio range of adversarial patches according to priori; then, the image to be detected is grayed, denoised and edge detected, and the edges are connected to form closed contours through morphological operation; then, calculate the area of each contour and select the contours with an area less than the pre-set threshold to locate with a rectangular frame; finally, the framed area is also blackened or repaired. The device of this scheme is composed of a detection module and a repair module connected in sequence, the detection module can use any of the above paths or a simple combination thereof, and the parameters (such as error threshold, area ratio) are fixed values, and the algorithm and strength of the repair module are also pre-set and static. During the whole process, there is a lack of cooperative training and dynamic parameter adjustment mechanism based on real-time defense effect between the detection module and the repair module, and between the repair module and the target model that needs to be protected finally, forming an open-loop system.

[0024] In summary, the prior art has the following disadvantages:

[0025] 1. Current adversarial training is mostly based on the spatial domain, without fully utilizing the frequency domain characteristics, and cannot adapt to new attacks after deployment;

[0026] 2. The input purification method often uses a unified strategy, which may damage the semantic integrity of the image, and is disconnected from model training;

[0027] 3. There is a lack of a closed-loop framework for cooperative optimization of adversarial training and input purification, resulting in limited defense effect;

[0028] 4. When combining adversarial training and detection and repair methods, the pedestrian detector may directly learn to recognize images after adversarial attacks without combining with the detection and repair module, causing the detection and repair module to fail.

[0029] For shortcoming 1, the present application quantifies the vulnerability of different subbands in the frequency domain through wavelet transform and gradient analysis, thereby introducing an adaptive frequency domain alignment strategy, so that the adversarial attacks of different frequency bands can be identified, and the model learns to focus on and strengthen the robustness of the expression of vulnerable frequency band features during the training process.

[0030] For shortcoming 2, the present application introduces a frequency domain repair module, which does not use a unified denoising or filling strategy, but performs adaptive repair based on the patch position located by the frequency domain analysis and the frequency band characteristics. At the same time, the repair module is jointly trained with the subsequent pedestrian detector in an end-to-end manner, and the optimization target is directly aligned with the improvement of detection robustness, so as to eliminate the adversarial disturbance as much as possible while maximizing the preservation of semantic information critical to the detection task, avoiding the disconnection between repair and model performance.

[0031] For shortcoming 3, the present application designs a dual defense collaborative architecture containing a closed-loop feedback mechanism. The architecture deeply integrates the adversarial training process of frequency domain perception and the online repair module, dynamically adjusts the repair intensity and model training focus through real-time evaluation of the image quality and target detection performance after repair, forming a self-optimizing closed loop. This mechanism ensures that the improvement of model robustness and the enhancement of input purification efficiency promote each other and co-evolve, thereby systematically improving the overall defense capability and adaptability to known and unknown adversarial patches.

[0032] For shortcoming 4, the present application designs an adaptive control mechanism to control a progressive training strategy, which trains the repair module and the corresponding pedestrian detector in different stages, and supervises the training of the repair module using clean images, and designs a loss function to punish the pedestrian detector for obtaining comparable performance on unpatched adversarial images, thereby ensuring the dual defense framework of detection and repair combined with adversarial training.

[0033] The present application provides an adversarial training method of an adaptive frequency domain closed-loop collaborative defense system. In the training phase, the system receives training images containing adversarial patches, and after a process integrating frequency domain analysis, adaptive repair and closed-loop optimization, finally outputs robust target detection results. The entire scheme contains four core modules: a frequency domain analysis module based on wavelet transform, an adaptive repair module guided by frequency domain, a deep collaborative training mechanism, and a closed-loop feedback and parameter dynamic adjustment module. In the actual application stage, the system receives adversarial images that may or may not contain adversarial patches. Even if it does not contain adversarial patches, the system will use the patch detector to detect it, but the output patch mask is empty.

[0034] Thus, the application discloses an adaptive frequency domain adversarial training method of a target detector, the target detector comprising a repair module and a pedestrian detector connected in series, and the input of the repair module being connected with the output of the patch detector. The adaptive frequency domain adversarial training method comprises the following steps: training the patch detector through a training image with an adversarial patch and a patch position label of the adversarial patch in the training image; detecting the patch position in the training image based on the trained patch detector and inputting the repair module; inputting the repair module based on the patch position to the pedestrian detector after the repair module repairs the training image, and obtaining a pedestrian detection result; jointly training the repair module and the pedestrian detector based on the pedestrian detection result and a real pedestrian label in the training image; wherein the loss in the joint training comprises a standard target detection loss, a repair consistency loss based on a frequency domain image corresponding to the training image and a clean image in the frequency domain, and a repair dependence loss based on the average precision mean of detection.

[0035] The application optimizes the target and aligns the detection robustness through the end-to-end joint training of the repair module and the subsequent pedestrian detector, so that the semantic information critical to the detection task is maximally retained while the adversarial disturbance is eliminated as much as possible, the repair is not disconnected from the model performance, and the detection robustness is improved.

[0036] 1. Patch detection method based on frequency domain adaptive alignment.

[0037] The method for detecting the position of the adversarial patch considers that the adversarial patch can deceive the patch detector through concentrated and local strong signals. The application finds that in the spatial domain RBG image, the abnormality can be covered by the texture or semantic content. However, in the frequency domain image, the local and strong modification will appear as an abnormal frequency domain component. Once converted to the frequency domain, the detection algorithm can only operate on the key frequency band, or quickly locate the abnormal area through simple spectral filtering, which is much more efficient than the dense sliding window calculation or complex pixel-level analysis in the spatial domain. Therefore, the detection mode is transferred from the spatial domain to the frequency domain, which is essentially a dimensionality increase and conversion of the feature space, so that the patch features hidden in the spatial domain, which are variable and coupled with semantics, become significant, stable and decoupled from semantics in the frequency domain.

[0038] To solve this problem, the present application first aims to construct a frequency domain-aware patch detector. Through wavelet transform, the image is decomposed into the frequency domain to reveal the essential statistical anomalies introduced by the adversarial patch. On this basis, a frequency domain attention mechanism is designed to dynamically evaluate the importance of different frequency bands and adaptively focus computing resources on the most vulnerable frequency domain components. To ensure the robustness of the frequency domain enhancement process, an innovative adaptive frequency alignment loss function is introduced. This function dynamically adjusts the alignment strength by calculating the frequency domain vulnerability index, and at the same time constrains the amplitude consistency and phase consistency of the frequency domain features before and after enhancement, thereby improving the patch discrimination while effectively avoiding the introduction of new model vulnerability due to over-enhancement.

[0039] The training method of the patch detector in the present application comprises: constructing a vulnerability index of frequency domain enhancement for each training image based on a standard patch detection loss of the patch detector, and calculating an alignment strength of the training image; calculating an adaptive frequency alignment loss of the training image based on the alignment strength, and training the patch detector. Wherein, the standard patch detection loss refers to the commonly used, baseline level patch detection loss function in this field, such as binary cross-entropy loss, focal loss, IoU loss, etc.

[0040] First, for the first training image , a two-dimensional discrete wavelet transform (DWT) is applied to decompose it into a series of subbands:

[0041]

[0042] Among them, l is the hierarchical level index, that is, the level or scale of wavelet decomposition, , represents the low-frequency coefficient of the lowest layer , represents the horizontal high-frequency coefficient of level , represents the vertical high-frequency coefficient of level , represents the diagonal high-frequency coefficient of level , is the wavelet transform function.

[0043] In order to enhance the sensitivity of patch detection to high-frequency features, wavelet enhancement is used to enhance the high-frequency components of the training image, and this process is defined as:

[0044]

[0045] Among them ​is a high-frequency enhancement function that highlights the edge and texture features in the image through a high-pass filter. This enhancement strategy enables the patch detector to capture the high-frequency impact introduced by the adversarial patch more effectively.

[0046] In adversarial training, clean reference images are often unavailable. To this end, the present invention proposes an adaptive frequency alignment loss that does not require a static clean reference, but dynamically adjusts the strength of frequency-domain enhancement through relative gradient sensitivity analysis.

[0047] For the wavelet sub-band coefficient matrix of the i-th training image obtained through discrete wavelet transform , its vulnerability index is constructed as follows:

[0048]

[0049] wherein, represents the vulnerability index of the i-th training image, represents the gradient operator for taking the gradient of a function, represents the standard patch detection loss, represents the patch detector, represents the parameters of the patch detector, patch is an identifier, represents the training image after high-frequency enhancement on represents the patch position label of the adversarial patch in the i-th training image, the patch position label is artificially marked or obtained using a classifier or other tools, represents the Frobenius norm, is a very small positive number (to ensure numerical stability);An increase in the value indicates that the enhancement introduced excessive vulnerability in the sub-band. In one specific embodiment, , is a classification loss, is a bounding box regression loss,

[0050] is a target existence loss, and are the corresponding loss weights; these losses are common losses in the prior art and will not be described here. The alignment strength of each sub-band is dynamically modulated through an S-shaped weighting scheme of the calculated vulnerability index: wherein the alignment strength is dynamically adjusted by the vulnerability index:

[0051]

[0052]

[0053] ​​​​in, Indicates the first Alignment strength of each training image Set the basic alignment strength. This represents the sigmoid function. Indicates the control of the steepness of the transition. This indicates the established vulnerability threshold.

[0054] This adaptive weighting mechanism ensures that normal behavior is exhibited. The subbands of ) accept minimal regularization and exhibit high fragility. The subbands are strongly constrained to prevent excessive feature distortion, especially in the transition region between stable and fragile areas. The corresponding weighting factor is simplified to This corresponds to a moderate alignment strength, serving as a smooth transition point between weak and strong regularization mechanisms.

[0055] The resulting alignment loss promotes robust frequency representation by selectively suppressing enhancement operations in vulnerable subbands while preserving discriminative components in stable regions of the spectrum. This mechanism ensures stronger constraints are imposed on high-risk subbands, while maintaining the enhanced discriminativeness of normal subbands.

[0056] Based on this, adaptive frequency alignment loss Defined as:

[0057]

[0058] in, This represents the adaptive frequency alignment loss. N Indicates the number of training images. and Representing different hyperparameters, hyperparameters Hyperparameters control the amplitude consistency between the original and enhanced images. Maintain phase consistency between the original image and the enhanced image; express amplitude components, express The wavelet subband coefficient matrix after high-frequency enhancement express amplitude components, express phase components, express The phase component.

[0059] Based on the above content, we will conduct adversarial training on the patch detector, with the goal of training a patch detector that can identify adversarial patches in the wavelet domain. Thus, the composite loss is minimized during training:

[0060]

[0061] According to the formula, the patch detector is trained based on the adaptive frequency alignment loss and the standard patch detection loss. Since the training images used during training all have adversarial patches, the training images used in this embodiment are the first training images , is the expectation of the patch classifier during training, can be enhanced by an attention mechanism or the enhancement function described above, is the standard patch detection loss (the content contained in this loss is the same as the standard target detection loss, and the only difference is the detection content), is the adaptive frequency alignment loss, and the hyperparameter is used to balance these terms. represents the expectation of all image losses, represents a sample of a pair of input adversarial images and labels sampled from the adversarial patch data distribution. It should be noted that the patch detector in the present application can be YOLOv5, Faster R-CNN, FCOS, SSD or RetinaNet, or a combination or simple transformation thereof. The pedestrian detector and the patch detector have similar structures and are selected, combined or simply transformed from the detectors in the prior art. The repair module uses CNN, JPEG Compression, Context Encoder or GAN-based purification, or a combination or simple transformation thereof.

[0062] 2. Adaptive repair and robust detection joint defense method.

[0063] On the basis of realizing accurate positioning, the present application will further develop a collaborative optimization framework of adaptive repair and robust detection. This framework aims to solve the "shortcut learning" problem that may easily occur in the joint training of the repair module and the pedestrian detector, i.e., the pedestrian detector learns to ignore the repair area and rely on other context features for prediction. To this end, a lightweight repair network is designed to guide the patch mask obtained by the patch detector to perform directional content recovery on the contaminated area. More importantly, a repair dependency loss is designed, which requires that the detection performance of the repaired training image must be significantly better than that of the unrepaired adversarial image, thereby establishing a forced dependency between the two and fundamentally eliminating the shortcut behavior of the pedestrian detector.

[0064] ​

[0065] Patch mask obtained by the first-stage frozen (i.e. trained) patch detector Lightweight inpainting module For the input adversarial image Perform directional inpainting, and the inpainted training image is represented as:

[0066]

[0067] The inpainting module consists of three core components:

[0068] Frequency domain feature extraction: wavelet decomposition is performed on the input image to extract multi-scale frequency domain features:

[0069]

[0070] Spatial domain feature extraction: spatial domain features are extracted through a spatial domain feature extraction network:

[0071]

[0072] Frequency-spatial domain feature fusion: the frequency domain and spatial domain features are fused to generate an inpainting increment:

[0073]

[0074] The final inpainting output combines the original image with the modulated inpainting increment:

[0075]

[0076] where is the frequency domain feature extraction network (common frequency domain networks include CNN, Attention, lightweight network, etc.), is the spatial domain feature extraction network (common ones include CNN, Attention, lightweight network, etc.), is the feature fusion function that fuses the frequency domain features and the spatial domain features along the channel dimension (in addition to this, there are methods such as weighted summation Attention Fusion fusion). is a learnable inpainting intensity parameter, represents the cross-channel expanded patch mask (a mask shared by the RGB three channels, C is the number of channels, H is the height, and W is the width), represents element-wise multiplication. This formula ensures that the inpainting operation is concentrated in the patch affected area, while preserving the clean image content.

[0077] To make full use of the frequency domain information of the clean image, the present application proposes a frequency domain inpainting consistency loss Ensure that the restored image is consistent with the clean image in the frequency domain:

[0078]

[0079] in For low-frequency consistency loss, For high-frequency consistency loss, and The corresponding weighting coefficients are used to weight the loss values ​​according to the mask size, in order to emphasize the recovery of adversarial regions.

[0080] Low-frequency consistency loss: Ensures the restored image is consistent with the clean image in low-frequency components, preserving the main structure and content of the image.

[0081]

[0082] in This indicates low-frequency consistency loss. Represents the lowest layer of the repaired training image The low-frequency coefficient, Represents the lowest layer of a clean image The low-frequency coefficient.

[0083] High-frequency consistency loss: Constrains the difference in high-frequency components between the restored and clean images, focusing on edge and texture features.

[0084]

[0085] in, This indicates the sub-band type at a specific level. Indicates the horizontal high-frequency coefficient. Indicates the vertical high-frequency coefficient. This represents the high-frequency coefficients on the diagonal. Indicates hierarchy The sub-band type is The weighting coefficients can be adaptively adjusted according to the importance of the frequency band. Indicates extraction level during frequency domain transformation The sub-band type (such as horizontal, vertical, diagonal, etc.) is High frequency coefficients, express The corresponding clean image.

[0086] 3. A closed-loop feedback dynamic training mechanism based on a joint optimization strategy.

[0087] To prevent pedestrian detectors By bypassing the repair module and performing predictions directly, a repair-dependent loss is introduced into the joint optimization loss function. :

[0088]

[0089] wherein represents a detection performance function (in this embodiment, the mean average precision, mAP, is selected), represents a minimum expected performance improvement threshold required for repair, . This formula ensures that the detector must rely on the repair module to obtain robust performance.

[0090] Then, the patch detector is frozen and the repair module is jointly trained with the pedestrian detector . The composite objective function combines a dynamic strategy to prevent the pedestrian detector from ignoring the patch:

[0091]

[0092] wherein, represents the expectation of the adversarial image-label pair sampled from the training image dataset, represents the parameters of the pedestrian detector, represents the parameters of the repair module, represents the true pedestrian label of the th training image, the pedestrian label being obtained by manual labeling or using a classifier to label the training image, is the patch mask from the patch detector, represents the pedestrian detection loss coefficient, is the standard target detection loss, represents the image repair loss coefficient, is the repair quality loss, represents the dependency loss coefficient, is the dependency loss, and control the trade-off. Similarly, the standard target detection loss refers to a patch detection loss function commonly used in the field and at the baseline level, such as the binary cross-entropy loss, the focal loss, the IoU loss, etc.

[0093] To solve the paradox of adversarial training and prevent shortcut learning, the framework adopts a gradual training strategy controlled by an adaptive control mechanism, the weight of the repair consistency loss decreases from large to small, the weight of the repair dependency loss increases from small to large, and the difference between the two weights gradually converges to the same preset threshold range. Specifically, the training process evolves through three stages:

[0094] Early repair focus stage: the repair module dominates learning, the pedestrian detector is basically frozen, and the frequency domain loss weight is relatively high, ensuring the repair quality.​

[0095] Mid-balance training phase: Pedestrian detector participates in training with minimal learning rate, fixes the balance between detection and repair loss, and gradually reduces the strength of frequency domain supervision.

[0096] Late dual defense phase: Joint reinforcement of repair and detection, introduction of dynamic patch visibility control, simulation of real-world attack variability.

[0097] The optimization process follows the curriculum learning paradigm, and the composite objective function is dynamically adapted based on the current performance metrics, represented as:

[0098]

[0099]

[0100] where, represents the time-varying objective function, is the time-varying loss coefficient vector, represents the number of training times, represents the pedestrian detection loss coefficient at the th training, represents the image repair loss coefficient at the th training, represents the dependency loss coefficient at the th training.

[0101] The closed-loop feedback mechanism adjusts the training parameters dynamically through multi-dimensional performance evaluation

[0102] where

[0103] , which includes the accuracy of clean images , adversarial robustness , performance after repair and frequency domain quality metrics , represents the value of the trainable hyperparameter at the th training, represents the trainable hyperparameter, controls the adaptation rate, maps the performance indicators to the optimization objective, represents the gradient of this mapping, represents the value of the trainable hyperparameter at the th training.

[0104] ​​The adaptive training paradigm in the application ensures that the repair module establishes a robust basic capability before the pedestrian detector adapts, maintains balanced optimization in the transition stage, and finally realizes the cooperative defense of repair and detection. The closed-loop feedback continuously calibrates the defense strategy, and maintains the best trade-off between patch removal effectiveness and feature preservation in different confrontation scenes.

[0105] In order to verify the implementation effect of the application, the following verification test is carried out.

[0106] (1) Experimental setup details.

[0107] In the digital image attack experiment, the Inria Person dataset is used, which contains 614 training sets of pedestrian images and 288 test sets of pedestrian images. The defense method of the application aims to identify the position of the person in the image after the adversarial attack. Therefore, in order to evaluate the defense effect of the method, the evaluation index is the average precision (mAP), which is a commonly used performance measure in target detection tasks. In order to calculate mAP, the detection frame generated by each pedestrian detector on the clean dataset is used as the true frame, and then the same pedestrian detector is used to detect the dataset with added adversarial patch images, and the mAP of the pedestrian detector in this case is calculated. The greater the mAP drop, the weaker the defense effect.

[0108] The defense method designed in the scheme is mainly used for the attack dataset of mainstream adversarial patches. The whole method is designed based on pytorch, and is trained on a 4090 device. In order to ensure the consistency and comparability of evaluation, all detection models use unified parameter configuration, the confidence threshold is 0.5 (filter low confidence detection), the non-maximum suppression (NMS) threshold is 0.4 (eliminate overlapping detection frame), the input image size is 416x416 pixels (consistent with training), the Adam optimizer is used, and the learning rate is set to 0.02.

[0109] (2) Digital simulation experiment evaluation.

[0110] In order to more fully evaluate the superiority of the band-based detection and repair defense framework in patch detection and defense, the present scheme compares the related work of the anti-defense published in recent years. First, for the detection of the anti-patch position, six groups of experiments are set up to compare the defense effects of different defense methods. Among them, Ad-YOLO adds a patch detection category to the YOLO architecture, so that the model can identify target objects and anti-patches at the same time; Jedi locates the anti-patch based on information entropy analysis, and uses high-entropy region features and autoencoders to complete repair; PatchZero detects and "zeros" the patch area at the pixel level to achieve defense by using the mean pixel redraw; PAD realizes patch positioning and removal without prior knowledge according to the semantic independence and spatial heterogeneity of the anti-patch; Saliuitl generates a binary feature map by integrating a saliency threshold to detect and repair the anti-patch; SAC realizes pixel-level positioning by training a patch segmenter, and combines a shape completion algorithm to ensure complete removal of the anti-area; DIFFender is an anti-patch defense method based on a diffusion model, which detects and locates the anti-patch through a text-guided diffusion model, and realizes efficient defense by means of double-task integration and few-shot prompt adjustment.

[0111] By applying the natural anti-patch and the non-natural anti-patch to the images of the test set of the Inria pedestrian dataset, the AP value of the "person" category of different target detection models is calculated. Among them, the patch position detection effect of the present scheme is best for different datasets, and the present scheme has greater advantages in the comparison of different datasets and defense methods.

[0112] Tables 1 and 2 show the test results of the trained patch detector in the digital world, and the defense performance is represented by the change of the AP value. The first column on the left lists the defense methods, and the first row on the top lists all the anti-patch methods, and Ours represents the present scheme. Among them, Table 1 shows the test results of the anti-patch generated by method 1 (AdvPatch), method 2 (CAP), method 3 (T-SEA) and method 4 (AdvT-shirt), and Table 2 shows the test results of the anti-patch generated by method 5 (AdvTexture), method 6 (DM-NAP) and method 7 (GNAP).

[0113] Table 1 Test results of patch detector on anti-patches generated by methods 1-4

[0114]

[0115] Table 2 Test results of patch detector on anti-patches generated by methods 5-7

[0116]

[0117] Table 3 and Table 4 show the test results of the trained pedestrian detector in the digital world, with the change of AP value representing the defense performance. The first column on the left lists the defense methods, and the first row on the top is all the adversarial patch methods; wherein, Table 3 shows the test results of the adversarial patches generated by method 0 (Clean, i.e. clean image), method 1, method 6, method 7 and method 8 (Diffpatch), and Table 4 shows the test results of the adversarial patches generated by method 3, method 5, method 9 (AdpPatch) and method 10 (LaVAN).

[0118] Table 3 Test results of pedestrian detector on adversarial patches generated by methods 0-1 and 6-8

[0119]

[0120] Table 4 Test results of pedestrian detector on adversarial patches generated by methods 3, 5, 9 and 10

[0121]

[0122] The present application is directed to the core problems of static rigidity of detection strategy, disconnection of repair process and model training, and lack of systematic and collaborative optimization in existing adversarial patch defense technology, and proposes an innovative adaptive frequency domain closed-loop collaborative defense system.

[0123] The core of the present application is to build a unified, dynamic and adaptive end-to-end defense framework. First, through frequency domain analysis and adaptive alignment strategy, the detection of adversarial patches is converted from spatial domain to frequency domain, the essence of statistical anomaly is revealed by wavelet transform, and the precise identification and differential enhancement of different fragile frequency bands are realized through gradient sensitivity quantification and dynamic attention mechanism. Secondly, an adaptive repair module guided by frequency domain is designed, which can guide the content-aware fine repair with patch mask, and perform end-to-end joint training with the pedestrian detector. Most importantly, the present application introduces repair dependent loss and closed-loop feedback mechanism, forces to establish the deep dependency between the repair module and the pedestrian detector, and realizes the dynamic adjustment of system parameters based on real-time performance feedback, fundamentally prevents "short circuit learning", and ensures that the defense strategy can continuously co-evolve with the threat changes.

[0124] The experimental results show that the scheme has obvious advantages in defense success rate (such as average precision AP), generalization ability to unknown attacks and performance retention on clean images compared with existing static detection-repair schemes and traditional adversarial training methods on various adversarial patch attack data sets. The invention not only provides an efficient defense technical scheme, but more importantly, proposes a defense paradigm that deeply integrates input purification and model reinforcement, and realizes closed-loop optimization, which provides a new idea for improving the safety and reliability of deep learning models in an open and dynamic physical world, and has important theoretical value and wide application prospect.

[0125] In addition, as shown in Figure 2 and Figure 3 , an adversarial patch positioning diagram based on frequency domain analysis is shown, and the first row in the two figures is a clean image, and the second row is an adversarial image. Given the training image (the first column) of the input, it is first decomposed into a low-frequency component and a multi-directional high-frequency component by two-dimensional discrete wavelet transform, wherein the low-frequency component mainly represents the overall structure and semantic information of the image, and the high-frequency component describes the local edge and texture details (the second and third columns). By modeling the response difference of clean images and adversarial images in the high-frequency subband, a frequency vulnerability map (Frequency Vulnerability) is constructed to highlight the abnormal response area introduced by the adversarial patch in the frequency domain (the first column). On this basis, a patch localization mask (Patch Localization) is generated by thresholding and morphological operation, and is superimposed on the original adversarial image to realize intuitive visualization of the adversarial patch position (the second column). The figure directly illustrates how the method of the invention realizes accurate positioning of the adversarial patch in the frequency domain by combining low-frequency structure preservation and high-frequency anomaly perception, and provides a reliable basis for subsequent adaptive repair and robust detection. Figure 2 Figure 2 Figure 3 The first column is the first column. Figure 3

[0126] Figures 4-6 A visualization comparison result diagram of target detection performance at different stages under the adversarial patch attack scene is shown. Figure 4 The detection result of the clean image under the original YOLOv5s detector is shown, and the detection frame can accurately cover the pedestrian target, which serves as an ideal reference baseline. Figure 5 The image after adding the adversarial patch is shown, and the original YOLOv5s detector has obvious missed detection and false detection under the interference of the patch, and the detection performance is significantly reduced. Figure 6 ​​​For the detection results of the frequency domain perception repair and the joint robust training of the pedestrian detector (also using the original YOLOv5s architecture) by the method of the application, it can be observed that the pedestrian target is repositioned accurately, and the detection performance is effectively restored. The green solid line box in the figure represents the pedestrian target successfully predicted by the pedestrian detector, and the yellow dashed line box represents the ground truth. The visualization result directly verifies the significant improvement effect of the method of the application on the robustness of target detection against patch attacks.

[0127] In summary, the application proposes an adaptive frequency alignment learning strategy based on wavelet transform and gradient sensitivity analysis. The method quantifies the vulnerability of different frequency subbands and dynamically allocates defense resources using a frequency domain attention mechanism to achieve accurate positioning and suppression of abnormal spectral components introduced by adversarial patches. Compared with traditional spatial domain methods or fixed frequency domain filtering, this method can adaptively focus on the most vulnerable frequency band, improving patch detection capability while effectively protecting image semantic features (such as edges and textures) essential to model recognition.

[0128] The application also proposes a cooperative optimization training mechanism for repair and detection: an architecture for end-to-end joint training of a frequency-guided adaptive repair module and a pedestrian detector is constructed, and a repair-dependent loss function is innovatively introduced. This mechanism forces the performance improvement of the detection model to rely on the output of the repair module, eliminating the "short circuit learning" phenomenon of the detector "ignoring repair and directly fitting adversarial disturbances" from the optimization objective, and achieving deep coupling and mutual promotion between repair effectiveness (input purification) and model robustness (model strengthening).

[0129] Finally, a closed-loop feedback and dynamic adjustment system is proposed: a closed-loop feedback and dynamic parameter adjustment system based on real-time performance evaluation is designed. The system continuously monitors multiple indicators such as repair quality, model performance on clean / adversarial / repaired data, and automatically and dynamically adjusts repair strength, frequency domain attention threshold, and weights of various losses in joint training. This enables the entire defense framework to adapt to evolving attack patterns online, realizing a fundamental shift from static, preset strategies to dynamic, self-optimizing defense paradigms.

[0130] The present application has the advantages that it builds a dynamic, cooperative and self-adaptive complete defense system, which fundamentally overcomes the limitations of the prior art. First, by introducing the adaptive frequency domain analysis and alignment strategy, the system can dynamically perceive and focus on the key frequency components sensitive to different attacks, achieving more accurate detection and suppression of diversified and unknown adversarial patches, while avoiding the damage to the image semantic content (especially high-frequency details) caused by traditional uniform processing. Second, through the end-to-end cooperative optimization design of the repair module and the detection model and the repair-dependent loss mechanism, the problem of isolated defense modules is completely solved, ensuring that the input purification effect is directly and effectively converted into the improvement of model robustness, preventing "short circuit learning". Finally, thanks to the closed-loop feedback and dynamic adjustment mechanism, the entire system has online evolution capability, which can continuously optimize itself according to real-time defense effectiveness, thereby significantly improving the long-term adaptability and overall defense stability of new and evolving attacks.

[0131] Although there are many possibilities for technical paths to achieve similar defense purposes, such as using discrete cosine transform (DCT) instead of wavelet transform for frequency domain analysis, or using traditional image repair algorithms instead of learnable repair networks, these alternative solutions often compromise key performance. They usually make the defense strategy static and single, unable to adapt to specific attacks and image content, and also difficult to effectively eliminate patches while ensuring minimal damage to the original image semantic features. More fundamentally, existing alternative solutions mostly continue the traditional idea of separating "detection and repair" from "model reinforcement". This isolated design between modules makes the system lack the ability of cooperative evolution, not only preventing the "short circuit learning" behavior of the detection model ignoring the repair effect, but also making it difficult to continuously optimize the dynamically changing attacks through real-time feedback. Therefore, although these alternative solutions can partially achieve the defense function, they are difficult to achieve the systematic high robustness brought by the deep cooperation and adaptive optimization of each link as realized by the present application.

[0132] The present application also discloses an adaptive frequency domain adversarial training device of a target detector, comprising a memory, a processor and a computer program stored in the memory and running on the processor, and the processor implements the above method when executing the computer program.

[0133] The present application also discloses a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps in each of the method embodiments.

[0134] The present application also provides a computer program product, which, when executed on a data storage device, causes the data storage device to implement the steps in each of the method embodiments.

[0135] The integrated unit module, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiment methods can be completed by instructing related hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the processor executes the computer program, the steps of each method embodiment described above can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or some intermediate forms, etc. The computer-readable medium at least includes any entity or device capable of carrying the computer program code to the storage device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium. For example, U disk, mobile hard disk, magnetic disk or optical disk, etc.

[0136] In the above embodiments, the description of each embodiment has its own focus, and the parts not described or recorded in detail in a certain embodiment can be referred to the related description of other embodiments.

[0137] Those of ordinary skill in the art can realize that the algorithm steps of each example described in combination with the embodiments disclosed in the present application can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0138] It should be noted that the data used in the execution process of the present application is collected or collected through legal and compliant channels, and the collection and collection behavior fully meets the requirements of relevant laws and regulations and industry standards; The existing technical methods involved in the present application are also obtained and used through legal and compliant channels.

Claims

1. An adaptive frequency domain adversarial training method for object detector, characterized in that, The target detector comprises a repair module and a pedestrian detector connected in series, wherein the input of the repair module is connected with the output of the patch detector; The adaptive frequency domain adversarial training method comprises the following steps: training a patch detector by using a training image with an adversarial patch and patch position labels of the adversarial patch in the training image; detecting patch positions in the training image based on the trained patch detector and inputting the repair module; the repair module inputs the training image after repairing based on the patch positions into a pedestrian detector to obtain a pedestrian detection result; and the repair module and the pedestrian detector are jointly trained based on the pedestrian detection result and real pedestrian labels in the training image; wherein, the loss in the joint training comprises a standard target detection loss, a repair consistency loss based on a frequency domain image corresponding to the training image and a clean image in the frequency domain, and a repair dependency loss based on the mean value of the average precision of detection; the repair consistency loss comprises a low-frequency consistency loss and a high-frequency consistency loss; the high-frequency consistency loss is: ， wherein, represents a high frequency consistency loss, is a hierarchical level index, , L is a hierarchical level, represents a subband type at level l, represents a horizontal high frequency coefficient, represents a vertical high frequency coefficient, represents a diagonal high frequency coefficient, represents a level upper subband type is a weight coefficient, represents a high frequency coefficient of a subband type at level upper subband type is when frequency domain conversion is extracted, represents the th training image after repair, represents the corresponding clean image; the low-frequency consistency loss is: ， in, This indicates low-frequency consistency loss. Represents the lowest layer of the repaired training image The low-frequency coefficient, Represents the lowest layer of a clean image The low-frequency coefficient; the repair dependency loss is: ， wherein, represents a repair dependency loss, represents a minimum expected performance improvement threshold required for repair, , represents a detection performance metric function, represents the th training image; in the joint training process, the weight of the repair consistency loss gradually decreases from large to small, the weight of the repair dependency loss gradually increases from small to large, and the difference between the two weights gradually converges to the same preset threshold range. 2.The method of claim 1, wherein, The training method of the patch detector comprises: constructing a vulnerability index of each training image for frequency domain enhancement based on the standard patch detection loss of the patch detector, and calculating the alignment strength of the training image; calculating the adaptive frequency alignment loss of the training image based on the alignment strength, and training the patch detector; wherein, the calculation method of the vulnerability index is: ， in, Indicates the first Vulnerability index of each training image The gradient operator represents the gradient of a function. Indicates the first Wavelet subband coefficient matrix of training images This indicates the loss of detection capability in the standard patch. Indicates a patch detector. The parameters represent the patch detector; patch is the identifier. Describes the wavelet transform function. Indicates to Training images after high-frequency enhancement Indicates the first Patch location labels for each training image. Denotes the Frobenius norm. It represents a very small positive number. 3.The method of claim 2, wherein, training the patch detector based on the adaptive frequency alignment loss and the standard patch detection loss. 4.The method of claim 2 or 3, wherein, the alignment strength calculation method is: ， in, Indicates the first Alignment strength of each training image Indicates the basic alignment strength. This represents the sigmoid function. Indicates the control of the steepness of the transition. This represents the vulnerability threshold. 5.The method of claim 4, wherein, the adaptive frequency alignment loss is: ， wherein, denotes the adaptive frequency alignment loss, N denotes the number of training images, and denote different hyperparameters, denotes the magnitude component of denotes the wavelet subband coefficient matrix after high frequency enhancement, denotes the magnitude component of denotes the phase component of denotes the phase component of 6.An apparatus of adaptive frequency domain adversarial training of object detector, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, The processor implements the method of any one of claims 1-5 when executing the computer program. The processor implements the method of any one of claims 1-5 when executing the computer program.

Images