Target detector robustness fine tuning method based on edge computing power perception

Abstract

The invention discloses a target detector robustness fine tuning method based on edge computing power perception. The method comprises the following steps: evaluating edge equipment system parameters for operating a target detection task, and estimating a non-maximum suppression algorithm input threshold value under a delay attack; performing adversarial training on the target detection model; loading a robust target detection model obtained by adversarial training, adding a fine tuning layer and initializing fine tuning parameters, calculating global adversarial noise by using an adversarial loss function and performing adjustment by using a binary mask, applying the adversarial noise to an original sample to form an adversarial sample, and performing target detection on the adversarial sample; inputting the adversarial sample into the target detection model and then performing fine tuning by using a low-rank adaptation method; and continuously performing the fine adjustment process until the result meets the preset requirement. According to the method, the robustness strategy of the model is adaptively adjusted through an edge computing power sensing mechanism according to system parameters such as computing power and bandwidth of different edge devices, so that the same set of method can adapt to the edge devices with different performances.

Description

Technical Field

[0001] This invention relates to the field of target detection, and more particularly to a robust fine-tuning method for a target detector based on edge computing power perception. Background Technology

[0002] Real-time object detection is central to a wide range of downstream applications, including autonomous driving, drone navigation, video surveillance, and augmented reality. From early detectors like R-CNN to the latest versions of the YOLO (You Only Look Once) series, we've seen incredible performance and efficiency improvements, achieving an average accuracy (mAP) of 56-60% and FPS exceeding 30 on the MS-COCO benchmark on embedded edge devices like the NVIDIA Jetson development board. However, behind the rapid development of real-time object detectors, a new attack method called "delay attack" has quietly emerged. Delay attacks aim to compromise real-time performance by exploiting vulnerabilities in the non-maximum suppression algorithm used in post-processing of object detectors. By creating a large number of "phantom" objects, they significantly increase the inference time of the object detector, posing a high risk to real-time applications. Besides delay attacks, other attack methods in object detection, such as classification misdirection attacks, location misdirection attacks, and object vanishing attacks, can also affect the inference of object detectors.

[0003] Currently, there are no effective defenses against delay attacks. Furthermore, since delay attacks target different objects than other types of attacks (delay attacks target model availability), the adversarial training methods used against other types of attacks cannot meet the needs of defending against multiple attack surfaces. In order to resist new security risks and thoroughly improve the robustness of the model under adversarial attacks, we propose a robustness fine-tuning method for target detectors based on edge computing power. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by proposing a robust fine-tuning method for target detectors based on edge computing power perception.

[0005] The objective of this invention is achieved through the following technical solution: a robust fine-tuning method for a target detector based on edge computing power awareness, comprising:

[0006] S1. Evaluate the system parameters of the edge device running the target detection task, and estimate the input threshold of the nonmaximum suppression algorithm under delay attack based on the system information;

[0007] S2. Use the original samples to perform adversarial training on the object detection model to obtain a robust object detection model that can defend against adversarial attacks other than delay attacks.

[0008] S3. Load the robust object detection model, add a fine-tuning layer and initialize the fine-tuning parameters, use the adversarial loss function to calculate the global adversarial noise, use the non-maximum suppression algorithm to input the threshold to obtain the binary mask, use the binary mask to adjust the global adversarial noise, apply the adjusted adversarial noise to the original samples to form adversarial samples, input the adversarial samples into the object detection model, and then use the low-rank adaptation method for fine-tuning.

[0009] S4. Evaluate the fine-tuned model and continue the fine-tuning process until the preset requirements are met;

[0010] S5. Use the finely tuned target detection model for unified defense against attacks to perform target detection and obtain the target detection results.

[0011] Furthermore, the input threshold for the nonmaximum suppression algorithm under the estimation delay attack specifically includes:

[0012] Estimate the processing time of the nonmaximum suppression algorithm for a specific number of objects; the estimation method is summarized by the following formula:

[0013]

[0014] Where N box T represents the number of input fields. backbone T represents the processing time of the backbone network, and T represents the required processing time of the task. nms This indicates that the processing time of the nonmaximum suppression algorithm needs to be less than the difference between the required processing time of the task and the processing time of the backbone network (TT). backbone ),

[0015] The maximum number of objects that the non-maximum suppression algorithm for the target detector can handle while meeting processing time requirements is calculated to be:

[0016]

[0017] in and For adjustment coefficients, The original computing power of the device, B represents the computing power that decreases after the L2 cache hit rate occurs, which is unique to edge devices.

[0018] Furthermore, the adversarial training of the target detection model specifically includes:

[0019] S2.1 Initialize the target detection model Basic training is performed using the original samples;

[0020] S2.2, Generate adversarial examples: For the input sample This enables the projection gradient descent attack to generate adversarial examples. : ,in ,in express Projection operations under norm constraints Step size, The loss function;

[0021] S2.3, Use a mixture of adversarial examples and clean examples for training: , where λ is the weight balance coefficient between clean samples and adversarial samples;

[0022] S2.4 Repeat steps S2.2-S2.3 until convergence, obtaining a robust pre-trained model.

[0023] Furthermore, the fine-tuning layer is a low-rank adaptation layer, which uses two low-rank matrices for parameterization, where the low-rank dimension is much smaller than the original matrix dimension to ensure parameter efficiency; the modified forward propagation updates weights through the product of low-rank matrices, and adopts a differentiated initialization strategy, initializing the first low-rank matrix with a random Gaussian distribution and the second matrix with zero initialization to ensure that the incremental weights are zero at the beginning of training; and a higher learning rate is set for the fine-tuning layer.

[0024] Furthermore, the calculation of global adversarial noise using the adversarial loss function specifically involves: The adversarial loss function is an object-oriented loss function.

[0025] Furthermore, the binary mask is obtained by mapping the input threshold of the non-maximum suppression algorithm under delay attack, and the mapping formula is: ,in This represents the maximum number of objects in the training dataset.

[0026] Furthermore, the evaluation of the fine-tuned model includes: evaluating whether various indicators meet the operational requirements of edge devices. The evaluation indicators include: average accuracy, adversarial robustness under various attacks such as PGD, C&W, and latency attacks, robustness decay rate, and inference latency. The evaluation process uses clean samples and adversarial samples of different attacks to input into the model for inference and calculate the above indicators.

[0027] According to another aspect of the specification, a robust fine-tuning system for a target detector based on edge computing power is also provided, comprising:

[0028] The edge device parameter acquisition module is used to evaluate the system parameters of the edge device running the target detection task, and estimate the input threshold of the non-maximum suppression algorithm under delay attack based on the system information.

[0029] The pre-training module is used to perform adversarial training on the target detection model using the original samples, so as to obtain a robust target detection model that can defend against adversarial attacks other than delay attacks.

[0030] The robust fine-tuning module is used to load the robust object detection model, add fine-tuning layers and initialize fine-tuning parameters, calculate global adversarial noise using an adversarial loss function and adjust it using a binary mask, apply the adjusted adversarial noise to the original samples to form adversarial samples, input the adversarial samples into the object detection model, and then fine-tune it using a low-rank adaptation method.

[0031] The model evaluation module is used to evaluate the fine-tuned model. Models whose evaluation parameters do not meet the preset requirements are sent to the robustness fine-tuning module for further fine-tuning.

[0032] According to another aspect of the specification, a robust fine-tuning device for a target detector based on edge computing power is also provided, including a memory and one or more processors. The memory stores executable code, and when the processor executes the executable code, it implements the aforementioned robust fine-tuning method for a target detector based on edge computing power.

[0033] According to another aspect of the specification, a computer-readable storage medium is also provided, on which a program is stored, which, when executed by a processor, implements the aforementioned robust fine-tuning method for a target detector based on edge computing power perception.

[0034] The beneficial effects of this invention are:

[0035] This invention effectively solves several key problems in the prior art through unique technological innovation:

[0036] An innovative solution to the challenge of defending against latency attacks: This invention creatively solves the technical problem mentioned in the background art—that "there are currently no effective defenses against latency attacks"—by utilizing an edge computing power awareness mechanism and binary masking technology. Specifically, by evaluating the system parameters of the edge device and estimating the input threshold of the non-maximum suppression algorithm under latency attacks, and combining this with binary masking to precisely adjust adversarial noise, the model can effectively identify and defend against latency attacks that increase inference time by creating a large number of "phantom" objects, thus ensuring the real-time performance of the target detector on the edge device.

[0037] A technological breakthrough in unified defense against multiple attack surfaces: This invention creatively combines a pre-trained robust model with robust fine-tuning techniques, effectively solving the problem in the background technology that "traditional adversarial training methods cannot meet the needs of defending against multiple attack surfaces because delayed attacks and other types of attack methods target different targets." By using a pre-trained model obtained through adversarial training to defend against classification misdirection attacks, and then using targeted robust fine-tuning to defend against delayed attacks, unified defense against classification misdirection attacks, location misdirection attacks, object disappearance attacks, and delayed attacks is achieved, significantly improving the overall robustness of the model.

[0038] A Dynamic Balance Between Accuracy and Robustness: This invention creatively solves the problem of accuracy degradation caused by robustness enhancement in traditional hardening methods by combining binary masking technology and the low-rank adaptation (LoRA) method. Binary masking dynamically adjusts adversarial noise based on the computing power characteristics of different edge devices, achieving a dynamic balance between model accuracy and robustness. The low-rank adaptation method, by freezing pre-training parameters and embedding a low-rank update matrix, effectively alleviates the overfitting and generalization degradation problems common in adversarial training while reducing training computation costs.

[0039] Technological innovation in edge device adaptability: This invention uses an edge computing power sensing mechanism to adaptively adjust the robustness strategy of the model based on system parameters such as computing power and bandwidth of different edge devices. This allows the same method to adapt to edge devices with different performance levels, solving the problem that traditional methods cannot customize defenses according to device characteristics and achieving the technical effect of "one method, applicable to multiple devices". Attached Figure Description

[0040] Figure 1 Experimental graph showing the relationship between the number of candidate boxes and the processing time of the nonmaximum suppression algorithm;

[0041] Figure 2 The experimental results for L2 Cache Hit Rate in GPU performance analysis;

[0042] Figure 3 This is a specific implementation diagram of an embodiment of the present invention;

[0043] Figure 4 A schematic diagram of a target detector robustness fine-tuning device based on edge computing power perception provided in an embodiment of the present invention. Detailed Implementation

[0044] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0045] Step 1 estimates the maximum number of objects allowed by the non-maximum suppression algorithm based on device parameters. This threshold forms the basis for robust fine-tuning. Extensive experiments verify the rationality and feasibility of this step. Specifically, as follows... Figure 1 and Figure 2 As shown, Figure 1 The graph plots the relationship between the number of objects and the processing time of the nonmaximum suppression algorithm on different devices under latency attacks. Device A is a laptop with an RTX 1650 Ti graphics card, and device B is a desktop computer with an RTX 4070 Ti Super graphics card. Devices C and D are edge devices, Jetson Orin NX and Jetson Xavier NX, respectively. It is evident that as the number of objects exceeds a certain threshold and continues to increase, the processing time of the nonmaximum suppression algorithm increases quadratically, especially on edge devices and CPUs with limited computing power. This is also supported by the time complexity of the nonmaximum suppression algorithm, O(|N*||). box | 2 This has been theoretically proven. Secondly, as... Figure 2 The analysis shows the L2 cache hit rate of the GPU under latency attacks. It can be observed that different edge devices (device C: Jetson Orin NX and device D: Jetson Xavier NX) exhibit different trends in the face of latency attacks due to differences in computing power. Specifically, the lower computing power of the Jetson Xavier NX leads to a decrease in the L2 cache hit rate. These phenomena can help in step 1, which involves computing power awareness, to design an appropriate threshold. Specifically, the method for estimating the processing time of the non-maximum suppression algorithm for a specific number of objects can be summarized by the following formula:

[0046]

[0047] Where N box T represents the number of input fields. backbone T represents the processing time of the backbone network, and T represents the required processing time of the task. nms This indicates that the processing time of the nonmaximum suppression algorithm needs to be less than the difference TT between the task-required processing time and the backbone network processing time. backbone Further calculations reveal the maximum object count threshold for the non-maximum suppression algorithm of the target detector while meeting processing time requirements:

[0048]

[0049] in and For adjustment coefficients, The original computing power of the device, B represents the reduced computing power of edge devices after the L2 Cache Hit Rate occurs, and B represents the PCIe memory bandwidth. The innovation of the method used in step 1 compared to existing methods lies primarily in its more comprehensive consideration of the different computing power conditions of edge devices. It can more accurately perceive the computing power of edge devices through computing power adjustments, and further more precisely determine the threshold that can resist latency attacks.

[0050] Step 2 uses adversarial training to obtain a robust object detection model capable of defending against adversarial attacks such as misclassification, which serves as the pre-trained model. The specific adversarial training process is as follows:

[0051] 2.1 Initialize the target detection model The model is trained using the original samples. The target detection model here is applicable to fields that require real-time object recognition, including but not limited to general image recognition and industrial fields such as autonomous driving.

[0052] 2.2 Generating adversarial examples: For the input sample Generate adversarial examples using PGD (Projected Gradient Descent) attacks. : ,in ,in express Projection operations under norm constraints Step size, The loss function;

[0053] 2.3 Training using a mixture of adversarial examples and clean examples: , where λ is the weight balance coefficient between clean samples and adversarial samples;

[0054] 2.4 Repeat steps 2.2-2.3 until convergence, to obtain a robust pre-trained model.

[0055] Using robust pre-trained models directly in existing technologies faces the following technical difficulties: 1) Robustness-accuracy trade-off: While achieving adversarial robustness, traditional robust pre-trained models typically suffer a significant reduction in detection accuracy on clean samples. In existing technologies, the mAP of robust pre-trained models usually decreases by 15-25%, which is unacceptable in practical deployments. 2) Limitations of single-attack defense: Existing robust pre-trained models can usually only defend against specific types of attacks (such as misclassification attacks), and cannot effectively cope with emerging latency attacks. Furthermore, the defense mechanisms for different attack types conflict.

[0056] This invention innovatively adopts a "layered defense + adaptive fine-tuning" technical approach, specifically reflected in three key innovations: First, it employs a progressive robust training strategy, using the original object detection model as the basic architecture and training a robust pre-trained model by progressively enhancing adversarial noise strength, effectively avoiding the severe conflict between robustness and accuracy in traditional methods. Second, it constructs a multi-attack type joint defense mechanism, integrating defense strategies against classification misdirection attacks, location misdirection attacks, and object disappearance attacks during the pre-training stage, achieving a unified defense framework through a joint loss function. Finally, it optimizes the pre-training parameters for edge awareness, considering the computational complexity constraints of the model during the pre-training stage based on the computing power distribution characteristics of edge devices, ensuring that the robust pre-trained model can run efficiently on edge devices. In particular, the innovation of step 2 is mainly reflected in the following: Unlike existing technologies that use models trained with clean samples as pre-training models, this invention uses a specially designed robust model as the pre-training model, providing a better initialization foundation for subsequent robustness fine-tuning, thereby significantly improving the robustness and real-time performance of the model while ensuring detection accuracy.

[0057] Step 3 is the main part of robust fine-tuning, and the implementation scheme can be summarized as follows: Figure 3 ,for Figure 3 The implementation can be explained as follows:

[0058] 3.1 Using the robust model trained in step 2 As a pre-trained model, this pre-trained model should be able to defend against adversarial attacks such as classification misdirection attacks and position misdirection attacks, in addition to delay attacks.

[0059] 3.2. A fine-tuning layer is added after the feature extraction layer of the pre-trained model. The fine-tuning layer is a low-rank adaptation (LoRA) layer. Specifically, the LoRA layer is located before the target detection head. Two low-rank matrices are added for parameterization, where the dimension of the low-rank matrix is ​​much smaller than that of the original matrix to ensure parameter efficiency. The modified forward propagation updates the weights through the product of the low-rank matrices. At the same time, a differentiated initialization strategy is adopted, initializing the first low-rank matrix with a random Gaussian distribution and the second matrix with zero initialization to ensure that the incremental weights are zero at the beginning of training. In addition, a higher learning rate is set for the fine-tuning layer to accelerate convergence, thereby effectively enhancing the robustness of the model while maintaining the original capabilities of the pre-trained model.

[0060] 3.3. Use the adversarial loss function as the loss function to generate adversarial noise during the fine-tuning process. In order to defend against delay attacks, the adversarial loss function here can specifically refer to the object-oriented loss function.

[0061] 3.4 Model Inference and Calculation of Global Adversarial Noise Using the Adversarial Loss Function. The specific method for calculating adversarial noise can be expressed as follows: ,in For robust pre-trained models, For object-oriented loss function, For symbolic functions, Update the step size for noise.

[0062] 3.5. Adjusted adversarial noise is calculated using a binary mask. The mask's role is to obtain a suitable robust model for different edge devices. The binary mask is obtained by mapping the input threshold from step 1, and the mapping formula is: ,in This represents the maximum number of objects in the training dataset. Using a binary mask here can improve the model's detection performance on the original samples, balancing the model's accuracy and robustness.

[0063] 3.6. Apply the adjusted adversarial noise to the original sample to form an adversarial sample.

[0064] 3.7. Low-rank adaptation (LoRA) is used as an efficient parameter fine-tuning method for the model. Adversarial examples are input into the object detection model for fine-tuning. The parameters trained during the fine-tuning process are mainly located in the fine-tuning layer.

[0065] 3.8 Fine-tuning complete, update model parameters.

[0066] 3.9. Evaluate the fine-tuned model and check whether the accuracy, adversarial robustness, and other indicators meet the requirements for operation on edge devices. Evaluation indicators include: mean average accuracy (mAP), adversarial robustness (robust accuracy) under various attacks such as PGD, C&W, and latency attacks, robustness decay rate, and inference latency (ms). The evaluation process uses clean samples and adversarial samples of different attacks to input into the model for inference and calculate the above indicators. If the requirements are met, output the fine-tuned robust model; otherwise, return to 3.2 to re-fine-tune.

[0067] The innovation of step 3 mainly lies in using fine-tuning techniques to adjust the robustness of the model while considering its accuracy. Model fine-tuning techniques are typically applied to large models. The reason for using fine-tuning techniques is that the resources of edge devices are efficient, and the goal is to increase robustness by minimizing changes to model parameters. Although efficient fine-tuning techniques such as LoRA are generally considered unsuitable for fine-tuning complex tasks like object detection, we found that this usually occurs when fine-tuning is done using different backbone networks and datasets. In our proposed technique, using only fine-tuning techniques and the same model and dataset to improve robustness is a feasible and efficient solution.

[0068] Corresponding to the aforementioned embodiment of a target detector robustness fine-tuning method based on edge computing power awareness, the present invention also provides an embodiment of a target detector robustness fine-tuning device based on edge computing power awareness.

[0069] See Figure 4 The present invention provides a target detector robustness fine-tuning device based on edge computing power awareness, comprising a memory and one or more processors. The memory stores executable code, and when the processor executes the executable code, it is used to implement a target detector robustness fine-tuning method based on edge computing power awareness in the above embodiment.

[0070] The embodiment of the robust fine-tuning device for a target detector based on edge computing power perception provided by this invention can be applied to any device with data processing capabilities, such as a computer. The device embodiment can be implemented in software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor of any data processing device loading the corresponding computer program instructions from non-volatile memory into memory for execution. From a hardware perspective, such as... Figure 4 The diagram shown is a hardware structure diagram of any device with data processing capabilities, where the target detector robustness fine-tuning device based on edge computing power perception provided by the present invention is located. (Except for...) Figure 4 In addition to the processor, memory, network interface, and non-volatile memory shown, any data processing device in the embodiment may also include other hardware depending on the actual function of the data processing device, which will not be described in detail here.

[0071] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0072] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the present invention according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0073] This invention also provides a computer-readable storage medium storing a program that, when executed by a processor, implements a robust fine-tuning method for a target detector based on edge computing power awareness as described in the above embodiments.

[0074] The computer-readable storage medium can be an internal storage unit of any data processing device described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device of any data processing device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units and external storage devices of any data processing device. The computer-readable storage medium is used to store the computer program and other programs and data required by the data processing device, and can also be used to temporarily store data that has been output or will be output.

[0075] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements the aforementioned robust fine-tuning method for a target detector based on edge computing power perception.

[0076] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this application are indicated by the claims.

[0077] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. This application is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A robust fine-tuning method for a target detector based on edge computing power awareness, characterized in that, include: S1. Evaluate the system parameters of the edge device running the target detection task, and estimate the input threshold of the nonmaximum suppression algorithm under delay attack based on the system information; S2. Use the original samples to perform adversarial training on the object detection model to obtain a robust object detection model that can defend against adversarial attacks other than delay attacks. S3. Load the robust object detection model, add a fine-tuning layer and initialize the fine-tuning parameters, use the adversarial loss function to calculate the global adversarial noise, use the non-maximum suppression algorithm to input the threshold to obtain the binary mask, use the binary mask to adjust the global adversarial noise, apply the adjusted adversarial noise to the original samples to form adversarial samples, input the adversarial samples into the object detection model, and then use the low-rank adaptation method for fine-tuning. S4. Evaluate the fine-tuned model and continue the fine-tuning process until the preset requirements are met; S5. Use the finely tuned target detection model for unified defense against attacks to perform target detection and obtain the target detection results.

2. The robust fine-tuning method for a target detector based on edge computing power awareness according to claim 1, characterized in that, The input threshold for the nonmaximum suppression algorithm under estimated delay attacks specifically includes: Estimate the processing time of the nonmaximum suppression algorithm for a specific number of objects; the estimation method is summarized by the following formula: ； Where N box T represents the number of input fields. backbone T represents the processing time of the backbone network, and T represents the required processing time of the task. nms This indicates that the processing time of the nonmaximum suppression algorithm needs to be less than the difference TT between the task-required processing time and the backbone network processing time. backbone , The maximum number of objects that the non-maximum suppression algorithm for the target detector can handle while meeting processing time requirements is calculated to be: ； in and For adjustment coefficients, The original computing power of the device, B represents the computing power that decreases after the L2 cache hit rate occurs, which is unique to edge devices.

3. The robust fine-tuning method for a target detector based on edge computing power awareness according to claim 1, characterized in that, The adversarial training of the target detection model specifically includes: S2.1 Initialize the target detection model Basic training is performed using the original samples; S2.2, Generate adversarial examples: For the input sample This enables the projection gradient descent attack to generate adversarial examples. : ,in ,in express Projection operations under norm constraints Step size, The loss function; S2.3, Use a mixture of adversarial examples and clean examples for training: , where λ is the weight balance coefficient between clean samples and adversarial samples; S2.4 Repeat steps S2.2-S2.3 until convergence, obtaining a robust pre-trained model.

4. The robust fine-tuning method for a target detector based on edge computing power awareness according to claim 1, characterized in that, The fine-tuning layer is a low-rank adaptation layer, which uses two low-rank matrices for parameterization. The low-rank dimension is much smaller than the original matrix dimension to ensure parameter efficiency. The modified forward propagation updates the weights through the product of the low-rank matrices. At the same time, a differentiated initialization strategy is adopted, which initializes the first low-rank matrix with a random Gaussian distribution and the second matrix with zero initialization to ensure that the incremental weights are zero at the beginning of training. A higher learning rate is set for the fine-tuning layer.

5. The robust fine-tuning method for a target detector based on edge computing power awareness according to claim 1, characterized in that, The specific steps for calculating global adversarial noise using the adversarial loss function are as follows: The adversarial loss function is an object-oriented loss function.

6. The robust fine-tuning method for a target detector based on edge computing power awareness according to claim 1, characterized in that, The binary mask is obtained by mapping the input threshold of the nonmaximum suppression algorithm under delay attack, and the mapping formula is: ,in This represents the maximum number of objects in the training dataset.

7. The robust fine-tuning method for a target detector based on edge computing power awareness according to claim 1, characterized in that, The evaluation of the fine-tuned model includes: evaluating whether various indicators meet the operational requirements of edge devices. The evaluation indicators include: average accuracy, adversarial robustness under various attacks such as PGD, C&W, and latency attacks, robustness decay rate, and inference latency. The evaluation process uses clean samples and adversarial samples of different attacks to input into the model for inference and calculate the above indicators.

8. A robust fine-tuning system for a target detector based on edge computing power awareness, characterized in that, include: The edge device parameter acquisition module is used to evaluate the system parameters of the edge device running the target detection task, and estimate the input threshold of the non-maximum suppression algorithm under delay attack based on the system information. The pre-training module is used to perform adversarial training on the target detection model using the original samples, so as to obtain a robust target detection model that can defend against adversarial attacks other than delay attacks. The robust fine-tuning module is used to load the robust object detection model, add fine-tuning layers and initialize fine-tuning parameters, calculate global adversarial noise using an adversarial loss function and adjust it using a binary mask, apply the adjusted adversarial noise to the original samples to form adversarial samples, input the adversarial samples into the object detection model, and then fine-tune it using a low-rank adaptation method. The model evaluation module is used to evaluate the fine-tuned model. Models whose evaluation parameters do not meet the preset requirements are sent to the robustness fine-tuning module for further fine-tuning.

9. A robust fine-tuning device for a target detector based on edge computing power awareness, comprising a memory and one or more processors, wherein the memory stores executable code, characterized in that... When the processor executes the executable code, it implements a robust fine-tuning method for a target detector based on edge computing power awareness as described in any one of claims 1-7.

10. A computer-readable storage medium having a program stored thereon, characterized in that, When the program is executed by the processor, it implements a robust fine-tuning method for a target detector based on edge computing power awareness as described in any one of claims 1-7.

Images