An Adaptive Edge Target Detection Method and System Based on Hybrid Lightweighting

Abstract

This invention discloses a hybrid lightweight adaptive edge target detection method and system, comprising the following steps: Step 1: setting up an environment and data for training servers; Step 2: constructing a hybrid lightweight adaptive edge target detection model, including an EMO backbone network, a lightweight neck network, and a decoupled detection head connected in sequence; Step 3: performing basic pre-training on the hybrid lightweight adaptive edge target detection model; Step 4: performing quantization-aware training on the pre-trained hybrid lightweight adaptive edge target detection model to obtain a model with two sets of quantization parameters; Step 5: single-device deployment and implementation. The lightweight design of the EMO backbone network significantly reduces the number of model parameters and computational load, providing more robust feature representations for subsequent detection. The detection system improves the overall fault tolerance and environmental adaptability of the detection scheme, making it more practical in real industrial scenarios.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and edge computing technology, and specifically relates to an adaptive edge target detection method and system based on hybrid lightweighting. Background Technology

[0002] Currently, deploying YOLO series models to edge devices (such as Jetson Nano and edge AI computing boxes) faces three core challenges: First, the model size and computing power requirements are contradictory. The original model has numerous parameters and high computational cost, making it difficult to achieve high frame rate inference on resource-constrained edge devices. Second, edge scenes are complex, with varying lighting, target occlusion, and small targets leading to a significant decrease in detection accuracy. Third, quantization accuracy is lost; traditional post-training quantization methods suffer severe accuracy loss at low bit depths (e.g., 4-bit), while general-purpose quantization-aware training cannot adapt to the characteristics of different edge hardware.

[0003] Faced with these challenges, existing technologies typically employ the following methods, but certain shortcomings remain: For example, simply replacing the lightweight backbone network (such as MobileNet) leads to a decrease in feature extraction capabilities, especially insensitivity to edge details and small objects. Introducing attention mechanisms is considered, but existing attention mechanisms (such as SE and CBAM) introduce additional computational overhead, making them unsuitable for edge devices. Using general 8-bit quantization offers limited compression and acceleration effects, failing to meet the requirements of ultra-low power consumption scenarios.

[0004] Therefore, in edge computing systems, detection models deployed on terminal devices typically operate statically, making it difficult to optimize themselves using real-world scenario data. Over time and with changing scenarios, the detection performance of static models degrades, failing to adapt to new environments and requirements. Therefore, a technical solution is needed that enables edge models to continuously, efficiently, and cost-effectively iterate and optimize their performance. Summary of the Invention

[0005] This invention addresses the problem that advanced detection models such as YOLO have high computational requirements and cannot achieve high frame rate real-time inference on edge devices such as Jetson Nano. It proposes an adaptive edge target detection method and system based on hybrid lightweighting.

[0006] The technical solution of the present invention is as follows: Firstly, an adaptive edge target detection method based on hybrid lightweighting includes: Step 1: Set up the environment and data for the training server; Step 2: Construct a hybrid lightweight adaptive edge target detection model, which includes an EMO backbone network, a lightweight neck network, and a decoupled detection head connected in sequence; Step 3: Perform basic pre-training on the hybrid lightweight adaptive edge target detection model; Step 4: Perform quantization-aware training on the hybrid lightweight adaptive edge target detection model after basic pre-training to obtain a hybrid lightweight adaptive edge target detection model with two sets of quantization parameters. In the quantization perception training process, an adaptive edge target detection method based on hybrid lightweight grouping is used to group the various parts of the hybrid lightweight adaptive edge target detection model according to their functions. The network parameters of each group are configured independently by quantization. At the same time, the weights of the network parameters of each group are applied by exponential moving average. The quantization bit width configuration in the EMO backbone network consists of two sets of parameters, namely 4-bit and 3-bit quantization parameters. The grouping is based on the following criteria: 1. Different functional modules have different tolerances and requirements for quantization. By isolating them through grouping, the optimal quantization parameters can be configured and updated independently for each group; 2. In quantization-aware training, the quantization error of each module will propagate downstream. Isolating functional modules can prevent the significant reconstruction error caused by the aggressive quantization strategy of upstream modules (such as the backbone network) from directly impacting downstream modules (such as the detection head) that are extremely sensitive to error. Step 5: Single device deployment and implementation; The two models trained by quantization perception are compiled separately to obtain two engine files, which are then deployed to the edge device along with the monitoring-scheduling program. The edge device switches programs according to its own state and calls the corresponding model to perform target detection on the acquired images.

[0007] Edge object detection refers to object detection using edge devices. The EMO network is used as the backbone feature extraction network of the hybrid lightweight adaptive edge object detection model. By integrating inverse residual modules, dilated convolution and multi-head self-attention mechanism, this network effectively captures local details and global contextual information of the image with an extremely low number of parameters. This significantly reduces the number of model parameters and computational cost, making it more suitable for running on edge devices, while providing more robust feature representations for subsequent detection.

[0008] Edge device status refers to the GPU utilization, power consumption, and temperature of the edge device itself. Depending on the different edge device statuses, the corresponding model needs to be selected to ensure the detection accuracy and efficiency of the edge device.

[0009] The innovative EMA-DQ quantization scheme enables the model to run stably at 4-bit / 3-bit precision, significantly reducing model storage requirements and computational energy consumption, while also providing the flexibility for dynamic adjustment.

[0010] Furthermore, the lightweight neck network includes an SCConv module and a BiFPN network connected in sequence; In this process, the original intermediate nodes in the BiFPN network are removed while the core connection paths are retained. At the same time, learnable weight parameters are added to the cross-scale connections, and the range of learnable weight parameters is constrained by the ReLU6 activation function. Then, two feature fusion paths, one from top to bottom and the other from bottom to top, are established.

[0011] Intermediate nodes typically refer to those nodes that do not directly connect to the input layer or the final output layer, but only serve as transit or temporary aggregation nodes for feature passing. While these nodes enhance feature interaction, they also introduce additional computational and memory overhead.

[0012] The core connection paths refer to those backbone paths that directly support cross-scale feature fusion and have a significant impact on the final detection performance.

[0013] After deleting intermediate nodes, a learnable weighted fusion operation (node) needs to be performed on each of the aforementioned retained cross-scale connections.

[0014] The SCConv module significantly reduces redundancy in spatial and channel dimensions, lowering computational overhead; the simplified BiFPN forms a lighter and more efficient feature pyramid structure to improve the detection capability for targets at different scales, especially small targets.

[0015] Furthermore, the EMO backbone network includes six sequentially connected downsampling stage modules: a first convolutional module, a second inverse residual module, a third inverse residual module, a fourth multi-head self-attention mechanism module, a fifth inverse residual module, and a sixth inverse residual module. The input channels, output channels, and strides of each stage convolutional module and inverse residual module are (3, 24, 2), (24, 48, 2), (48, 96, 2), (96, 192, 2), and (192, 384, 2), respectively. The third, fifth, and sixth inverse residual modules include 3, 4, and 6 inverse residual modules, respectively. The fourth multi-head self-attention mechanism module has 8 heads and an attention dimension of 192.

[0016] Furthermore, a front-end pixel attention module and a rear-end spatial adaptive attention module are set at the front and back ends of the EMO backbone network, respectively. The front pixel attention module includes a 1×1 convolutional layer, a sigmoid function, a multiplication module, and a bilinear interpolation module connected in sequence. The other input of the multiplication module is connected to the input of the 1×1 convolutional block. The post-space adaptive attention module includes a device status monitoring unit, two convolutional kernels of different sizes, and a switching unit. The switching unit selects and activates a convolutional kernel of a corresponding size based on the information obtained by the device monitoring unit. The sizes of the two convolutional kernels of different sizes are 5×5 and 3×3, respectively. The decision on whether to perform convolution processing on the output of the EMO backbone network, and which convolution kernel to use, is based on the GPU utilization. The front pixel attention is used to quickly focus on the target. A 1×1 convolutional layer is used to compress the three channels of the input image into a single-channel feature map. A sigmoid activation function layer is used to map the single-channel feature map into a pixel-level attention weight map. The pixel-level attention weight map is multiplied pixel by pixel with the input image of the 1×1 convolutional layer to obtain the weighted input features. The rear spatial adaptive attention module is a dynamic module that works in conjunction with the hardware decoder. It can adaptively adjust the size of the attention convolution kernel according to the device's real-time computing power (such as GPU utilization) (if computing power is sufficient, use 5x5 to improve accuracy; if computing power is tight, use 3x3 to ensure speed). With minimal additional overhead, it significantly improves the robustness of detecting small targets and occluded targets, and it also has dynamic resource balancing capabilities.

[0017] The progressive dual attention mechanism significantly enhances the model's ability to capture key information without introducing significant computational overhead, especially in complex scenes and small object detection.

[0018] Furthermore, the decoupled detection head includes parallel classification and regression branches. Each branch contains two 256-channel 3x3 convolutional layers and a 1x1 convolutional layer for outputting predicted values. The classification branch outputs the probabilities of 80 categories, and the regression branch outputs 4 coordinate values.

[0019] Furthermore, the edge devices adopt a master-slave communication architecture established by ZeroMQ, and use multiple cameras to perform edge target detection collaboratively. The camera acting as the master node is responsible for the target edge target detection task, while the camera acting as the slave node performs auxiliary edge target detection. During the detection process, the initial weight of the master node is set to 0.7. The weight of the slave node is dynamically calculated based on the preset relative spatial position relationship and viewing angle difference between the slave node and the master node, with a value range of 0.2-0.3. The overlapping detection boxes obtained by multiple edge devices are used as the final edge target detection coordinates by weighted average fusion. The categories obtained by multiple edge devices are determined by confidence-weighted voting. The triggering condition for a node to participate in auxiliary detection is: Triggered when the detection confidence level obtained by the master node is lower than a set threshold; Triggered when the target is located at the edge of the image; Typically, the boundary is within 20 pixels; because targets located at the edge of an image usually only have a portion within the frame, with the other part truncated. This results in incomplete appearance features of the target, making it difficult for the model to accurately identify its category and boundaries.

[0020] This is triggered when the master node detects that the proportion of the target occlusion area exceeds the set proportion.

[0021] The greater the difference in perspective and the more complementary information a node can provide, the higher the fusion weight is assigned. There are many well-known methods for specific strategies. Furthermore, when a node participates in assisted detection, the corresponding image sample is regarded as a difficult sample, and all difficult samples are used as training data for quantization perception training of the hybrid lightweight adaptive edge target detection model.

[0022] Only upload difficult samples and aggregate them. Because difficult samples represent the direction for further model optimization, the model is then retrained and updated based on these difficult samples before being deployed to edge models.

[0023] Furthermore, when applying an exponential moving average to the weights of the network parameters for each group, the EMA values ​​of the weight quantization range statistics and the activation value quantization range statistics are calculated according to the formula after each training batch: ;;

[0024] in, This represents the statistics for the current batch. As the attenuation factor, This is the sequence number of the current training batch. This refers to the sequence number of the previous batch in the current batch t; This indicates the EMA value saved after the previous batch was updated.

[0025] Secondly, an adaptive edge target detection system based on hybrid lightweight design includes: Data acquisition terminal equipment is used to acquire image data to be detected; At the edge node, multiple cameras use the hybrid lightweight adaptive edge target detection method described in any one of claims 1-7 to perform edge target detection on image targets based on the collected data, and upload the detection data to the cloud center; The cloud center regularly aggregates data from various edge nodes and uses it to retrain and distribute a hybrid lightweight adaptive edge object detection model to the edge nodes.

[0026] The "device-edge-cloud" three-tier collaborative architecture includes terminal devices responsible for data collection, edge nodes responsible for core inference tasks, and a cloud center responsible for model management and updates. Edge nodes bear the main computing load, while the cloud center periodically aggregates data from each node for model retraining and distribution, forming an optimization loop.

[0027] Edge nodes: Upload difficult sample data that meets specific conditions in the detection results to the cloud center; (difficult samples are those that trigger auxiliary detection by slave nodes). The cloud center aggregates difficult samples from different edge nodes and uses them to retrain and update the hybrid lightweight edge object detection model. The updated model is then distributed to the edge nodes. By using this "end-edge-cloud" architecture to handle difficult samples and optimize the model, this system becomes one capable of targeted optimization of the edge object detection model.

[0028] Furthermore, the multi-camera system has at least two cameras, which are distributed in different locations.

[0029] For any potential target location within the monitored area, there are at least two cameras capable of observing from different perspectives.

[0030] Compared with the prior art, the technical solution of the present invention has the following advantages: 1. The technical solution of this invention provides a systematic lightweight design that achieves a balance between accuracy and efficiency. It employs and optimizes an EMO backbone network composed of inverse residual modules, dilated convolutions, and multi-head self-attention mechanisms, combined with a lightweight neck network (SCConv + simplified BiFPN), resulting in a significant reduction in model parameters and computational cost. Simultaneously, it maintains the ability to extract and fuse multi-scale features (especially small targets), resolving the fundamental contradiction between the computing power of edge devices and model performance.

[0031] 2. The hardware-aware adaptive inference mechanism enhances scene adaptability; the innovative introduction of a post-space adaptive attention module can dynamically select attention convolution kernels of different sizes (5×5 or 3×3) according to the real-time status of the device (such as GPU utilization), achieving an intelligent balance of "improving accuracy when computing power is sufficient and maintaining speed when computing power is tight", thus enhancing the robustness of the system under dynamic load.

[0032] 3. Stable ultra-low bit width quantization capability, significantly reducing storage and energy consumption; by employing grouped quantization-aware training (EMA-DQ) combined with exponential moving average smoothing of the training process, the model can run stably at ultra-low bit widths of 4-bit / 3-bit. Compared with traditional quantization methods, it effectively controls the accuracy loss caused by low-bit quantization while significantly compressing model size and reducing memory and computational energy consumption.

[0033] 4. Enhanced reliability of detection in complex scenes through intelligent multi-camera collaboration; a trigger-based multi-camera collaborative detection mechanism was designed based on the ZeroMQ master-slave communication architecture. When the master node's detection confidence is low, the target is located at the edge, or there is severe occlusion, slave nodes are automatically scheduled to assist in detection, and a dynamic weight fusion strategy is adopted, which significantly improves the detection coverage and accuracy in complex scenes such as occlusion and limited viewing angle.

[0034] 5. A closed-loop optimization system driven by "difficult samples" enables continuous model evolution. By constructing a three-level collaborative architecture of "end-edge-cloud", edge nodes only upload "difficult samples" that trigger collaboration to the cloud center. The cloud center aggregates these samples to perform targeted retraining and distribution of the model, forming a data-efficient and continuously self-optimizing closed-loop system, which effectively alleviates the performance degradation problem of static models caused by scene changes.

[0035] 6. Modular deployment and dynamic precision switching enhance system scalability and adaptability; the technical solution of this invention supports compiling the trained 4-bit and 3-bit double-precision models into independent engines and dynamically switching them on edge devices according to real-time status (such as power, temperature, and load). The same system can flexibly adapt to edge devices with different hardware capabilities, simplifying the complexity of large-scale deployment and operation and maintenance. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of a hybrid lightweight network structure.

[0038] Figures 2(a) and 2(b) are schematic diagrams of the feature fusion network connection after the intermediate nodes are deleted, before and after the modification.

[0039] Figure 3 This is a schematic diagram of the hardware sensing process.

[0040] Figure 4 This is a flowchart illustrating the training and deployment method of a cloud-edge collaborative model for edge adaptive inference. Detailed Implementation

[0041] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0042] Example 1 like Figure 1 As shown, an adaptive edge target detection method based on hybrid lightweighting includes: Step 1: Set up the environment and data for the training server; Step 2: Construct a hybrid lightweight adaptive edge object detection model, such as... Figure 1As shown, the network adopts a hierarchical design, with its data processing flow sequentially through: a pre-pixel attention module, a lightweight hybrid backbone network (EMO), a post-spatial adaptive attention module, a feature fusion neck network, and a decoupled detection head. The post-spatial adaptive attention module is a configurable module, capable of dynamically selecting the size of its internal convolutional kernels according to external control commands. The following will provide a detailed description of each part in conjunction with the accompanying drawings.

[0043] The EMO backbone network comprises six sequentially connected downsampling stage modules: a first convolutional module, a second inverse residual module, a third inverse residual module, a fourth multi-head self-attention mechanism module, a fifth inverse residual module, and a sixth inverse residual module. The input channels, output channels, and strides of each stage's convolutional and inverse residual modules are (3, 24, 2), (24, 48, 2), (48, 96, 2), (96, 192, 2), and (192, 384, 2), respectively. The third, fifth, and sixth inverse residual modules each contain 3, 4, and 6 inverse residual modules, respectively. The fourth multi-head self-attention mechanism module has 8 heads and an attention dimension of 192.

[0044] A front-end pixel attention module and a rear-end spatial adaptive attention module are set at the front end and back end of the EMO backbone network, respectively. The front pixel attention module includes a 1×1 convolutional layer, a sigmoid function, a multiplication module, and a bilinear interpolation module connected in sequence. The other input of the multiplication module is connected to the input of the 1×1 convolutional block. The post-space adaptive attention module includes a device status monitoring unit, two convolutional kernels of different sizes, and a switching unit. The switching unit selects and activates a convolutional kernel of a corresponding size based on information obtained from the device monitoring unit. The two convolutional kernels have sizes of 5×5 and 3×3, respectively. For specific logic, refer to... Figure 3 This logic works continuously during runtime. Its core is to dynamically adjust the execution configuration of the inference model based on the real-time collected hardware status data, so as to maximize the energy efficiency ratio and meet the real-time requirements while ensuring detection accuracy.

[0045] The decision on whether to perform convolution processing on the output of the EMO backbone network, and which convolution kernel to use, is based on the GPU utilization. The front pixel attention is used to quickly focus on the target. A 1×1 convolutional layer is used to compress the three channels of the input image into a single-channel feature map. A sigmoid activation function layer is used to map the single-channel feature map into a pixel-level attention weight map. The pixel-level attention weight map is multiplied pixel by pixel with the input image of the 1×1 convolutional layer to obtain the weighted input features. The rear spatial adaptive attention module is a dynamic module that works in conjunction with the hardware decoder. It can adaptively adjust the size of the attention convolution kernel according to the device's real-time computing power (such as GPU utilization) (if computing power is sufficient, use 5x5 to improve accuracy; if computing power is tight, use 3x3 to ensure speed). With minimal additional overhead, it significantly improves the robustness of detecting small targets and occluded targets, and it also has dynamic resource balancing capabilities.

[0046] The progressive dual attention mechanism significantly enhances the model's ability to capture key information without introducing significant computational overhead, especially in complex scenes and small object detection.

[0047] The lightweight neck network includes an SCConv module and a BiFPN network connected in sequence; In this simplified version of the BiFPN network, the original intermediate nodes are removed while the core connection paths are retained. At the same time, learnable weight parameters are added to the cross-scale connections, and the range of learnable weight parameters is constrained by the ReLU6 activation function. Then, two feature fusion paths, one from top to bottom and the other from bottom to top, are established.

[0048] Intermediate nodes typically refer to those nodes that do not directly connect to the input layer or the final output layer, but only serve as transit or temporary aggregation nodes for feature passing. While these nodes enhance feature interaction, they also introduce additional computational and memory overhead.

[0049] The core connection paths refer to those backbone paths that directly support cross-scale feature fusion and have a significant impact on the final detection performance.

[0050] After deleting intermediate nodes, a learnable weighted fusion operation (node) needs to be performed on each of the aforementioned retained cross-scale connections.

[0051] As shown in Figures 2(a) and 2(b), Figure 2(a) presents the simplified structure proposed in this invention. It constructs a lighter and more efficient feature fusion network topology by removing intermediate nodes (e.g., P4_td, P5_td) that only act as relays in the traditional bidirectional feature pyramid network (as shown in Figure 2(b)) and retaining the core cross-scale connection paths that directly support feature fusion. The p4 input and P5 input are directly fused through the first learnable weighted fusion node to generate the P4 output. This process does not require downsampling through the P5_td node or relaying through the P4_td node as shown in Figure 2(b). Then, the P4 output and the original P5 input (via a retained key cross-scale connection) are input together to the second learnable weighted fusion node to generate the P5 output. This design significantly reduces model complexity and computational resource consumption while ensuring the effectiveness of multi-scale feature fusion by focusing on the most important feature interaction paths.

[0052] The SCConv module significantly reduces redundancy in spatial and channel dimensions, lowering computational overhead; the simplified BiFPN forms a lighter and more efficient feature pyramid structure to improve the detection capability for targets at different scales, especially small targets.

[0053] The decoupled detection head includes parallel classification and regression branches. Each branch contains two 256-channel 3x3 convolutional layers and a 1x1 convolutional layer for outputting predicted values. The classification branch outputs the probabilities of 80 categories, and the regression branch outputs 4 coordinate values.

[0054] The hybrid lightweight network constructed in this invention is benchmarked against the publicly available dataset COCO. Currently, lightweight detection models that perform well on this dataset (such as YOLOv8n) typically have a mean average accuracy (mAP@0.5) of approximately 80%-82% and a parameter count of approximately 3 million (3M). Since the EMO backbone network effectively integrates local details and global contextual information, while the lightweight neck network efficiently aggregates multi-scale features, this design aims to ensure that the model maintains a parameter count similar to state-of-the-art lightweight models (such as YOLOv8n with approximately 3M parameters) while achieving equal or better feature representation capabilities on publicly available benchmark datasets (such as COCO) through superior feature extraction and fusion mechanisms.

[0055] Step 3: Perform basic pre-training on the hybrid lightweight adaptive edge target detection model; Step 4: Perform quantization-aware training on the hybrid lightweight adaptive edge target detection model after basic pre-training to obtain a hybrid lightweight adaptive edge target detection model with two sets of quantization parameters. In the quantization perception training process, an adaptive edge target detection method based on hybrid lightweight grouping is used to group the various parts of the hybrid lightweight adaptive edge target detection model according to their functions. The network parameters of each group are configured independently by quantization. At the same time, the weights of the network parameters of each group are applied by exponential moving average. The quantization bit width configuration in the EMO backbone network consists of two sets of parameters, namely 4-bit and 3-bit quantization parameters. The grouping is based on the following criteria: 1. Different functional modules have different tolerances and requirements for quantization. By isolating them through grouping, the optimal quantization parameters can be configured and updated independently for each group; 2. In quantization-aware training, the quantization error of each module will propagate downstream. Isolating functional modules can prevent the significant reconstruction error caused by the aggressive quantization strategy of upstream modules (such as the backbone network) from directly impacting downstream modules (such as the detection head) that are extremely sensitive to error. When applying an exponential moving average to the weights of the network parameters for each group, the EMA values ​​of the weight quantization range statistics and the activation value quantization range statistics are calculated according to the formula after each training batch: ;

[0056] in, This represents the statistics for the current batch. The attenuation factor is set to 0.99 in this embodiment; This is the sequence number of the current training batch. This refers to the sequence number of the previous batch in the current batch t; This indicates the EMA value saved after the previous batch was updated.

[0057] Model quantization is crucial for edge deployment. Industry consensus indicates that 4-bit post-training quantization (PTQ) on complex detection models typically results in a 5%-10% loss in mAP accuracy. This method optimizes the quantization process by independently configuring and updating quantization parameters for different functional modules of the model and applying exponential moving averages to estimate quantization range statistics. This aims to improve the stability of low-bit quantization training by isolating the propagation of quantization errors between different modules and smoothing statistical fluctuations during training. Based on this, the design aims to achieve more effective control over model accuracy loss compared to traditional post-training quantization methods when implementing 4-bit and 3-bit ultra-low bit-width quantization, thus providing a feasible technical path for achieving high-precision, low-bit-rate inference on edge devices.

[0058] Step 5: Single device deployment and implementation; will pass Figure 4 The cloud training process shown generates and distributes pre-compiled 4-bit and 3-bit double-precision model engine files, along with a monitoring-scheduling program, to edge devices; the edge devices then deploy these files based on their device status (decision logic as follows). Figure 3 (As shown in the hardware perception process diagram) the program switches and calls the corresponding model to perform target detection on the acquired image.

[0059] Figure 4 A flowchart illustrating a cloud-based model training, quantization, and continuous optimization method according to an embodiment of the present invention is presented. The method proposed in this invention is a closed-loop process, mainly including a training data preparation module, a multi-stage model training and quantization module, a model conversion and repository management module, and a cloud retraining service module. This process generates multi-precision models through EMA-aware training and achieves continuous iterative optimization of the model through a hard sample feedback mechanism, ultimately deploying the optimized model to edge devices.

[0060] Edge object detection refers to object detection using edge devices. The EMO network is used as the backbone feature extraction network of the hybrid lightweight adaptive edge object detection model. By integrating inverse residual modules, dilated convolution and multi-head self-attention mechanism, this network effectively captures local details and global contextual information of the image with an extremely low number of parameters. This significantly reduces the number of model parameters and computational cost, making it more suitable for running on edge devices, while providing more robust feature representations for subsequent detection.

[0061] The innovative EMA-DQ quantization scheme enables the model to run stably at 4-bit / 3-bit precision, significantly reducing model storage requirements and computational energy consumption, while also providing the flexibility for dynamic adjustment.

[0062] On typical edge devices such as Jetson Nano, when computing resources are sufficient, a high-precision 4-bit quantization model is used to ensure detection quality. When the system load increases (e.g., GPU utilization consistently exceeds 80%), the hardware-aware control process described in this invention is used. Figure 3 The system dynamically switches to a 3-bit model and a smaller 3x3 convolution kernel, which is expected to improve the inference frame rate and thus ensure the real-time performance of the system. At the same time, the stability of the low-bit model enhanced by the aforementioned grouped EMA quantization perceptual training keeps the accuracy drop within a controllable range.

[0063] The edge devices employ a master-slave communication architecture built with ZeroMQ, utilizing multi-camera collaboration for edge target detection. The camera acting as the master node is responsible for edge target detection, while the cameras acting as slave nodes assist in edge target detection. During the detection process, the initial weight of the master node is set to 0.7. The weight of the slave nodes is dynamically calculated based on the preset relative spatial position relationship and viewing angle difference between the master and slave nodes, with a value range of 0.2-0.3. The overlapping detection boxes obtained by multiple edge devices are used as the final edge target detection coordinates using a weighted average fusion method. The categories obtained by multiple edge devices are determined by confidence-weighted voting. The triggering condition for a node to participate in auxiliary detection is: Triggered when the detection confidence level obtained by the master node is lower than a set threshold; Triggered when the target is located at the edge of the image; Typically, the boundary is within 20 pixels; because targets located at the edge of an image usually only have a portion within the frame, with the other part truncated. This results in incomplete appearance features of the target, making it difficult for the model to accurately identify its category and boundaries.

[0064] This is triggered when the master node detects that the proportion of the target occlusion area exceeds the set proportion.

[0065] The greater the difference in perspective and the more complementary information a node can provide, the higher the fusion weight is assigned. There are many well-known methods for specific strategies. To address challenging scenarios where single-camera detection confidence is insufficient (e.g., below 0.65) due to limited field of view, target occlusion, or excessively small size, the multi-camera collaborative mechanism designed in this invention is triggered. This mechanism schedules the acquisition of complementary viewpoint information from node cameras and uses dynamic weights based on preset spatial relationships and viewpoint differences for result fusion. This design aims to directly utilize the redundancy and complementarity of multi-viewpoint information to compensate for information gaps or ambiguities in single-viewpoint scenarios. Therefore, this collaborative mechanism provides the system with a technical means to improve the final decision confidence and reduce the risk of missed and false detections due to incomplete information when single-node judgments are unreliable, thereby enhancing the robustness of the entire detection system in complex scenarios from an architectural perspective.

[0066] When a node participates in assisted detection, the corresponding image sample is considered a hard sample, such as... Figure 4 As shown, a "Cloud Retraining Service" module is provided in the cloud. This service continuously receives and aggregates difficult sample data from various edge nodes, using it as new training data and injecting it into the "Training Data Preparation" stage, thereby driving the model into a new round of "training-quantization-optimization" cycle. Through this continuous optimization closed loop based on difficult sample feedback (i.e.... Figure 4 (Through the feedback loop in the system), the system can target and enhance weak points in real-world scenarios, enabling the model to self-iterate and evolve its performance.

[0067] Example 2

[0068] An adaptive edge target detection system based on hybrid lightweight design includes: Data acquisition terminal equipment is used to acquire image data to be detected; At edge nodes, multiple cameras use the aforementioned hybrid lightweight adaptive edge target detection method to perform edge target detection on image targets based on the collected data, and upload the detection data to the cloud center. The cloud center, its work is as follows Figure 4 The diagram forms a closed loop. The cloud center periodically aggregates data from each edge node and uses it for retraining and quantization optimization of a hybrid lightweight adaptive edge object detection model (corresponding to...). Figure 4 The training and quantization modules are distributed to edge nodes.

[0069] The "device-edge-cloud" three-tier collaborative architecture includes terminal devices responsible for data collection, edge nodes responsible for core inference tasks, and a cloud center responsible for model management and updates. Edge nodes bear the main computing load, while the cloud center periodically aggregates data from each node for model retraining and distribution, forming an optimization loop.

[0070] Edge nodes: Upload difficult sample data that meets specific conditions in the detection results to the cloud center; (difficult samples are those that trigger auxiliary detection by slave nodes). The cloud center aggregates difficult samples from different edge nodes and uses them to retrain and update the hybrid lightweight edge object detection model. The updated model is then distributed to the edge nodes. By using this "end-edge-cloud" architecture to handle difficult samples and optimize the model, this system becomes one capable of targeted optimization of the edge object detection model.

[0071] The specific implementation process is described in the aforementioned method embodiments.

[0072] The multi-camera system has at least two cameras, which are located in different positions.

[0073] For any potential target location within the monitored area, there are at least two cameras capable of observing from different perspectives.

[0074] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This application refers to flowchart illustrations and / or instructions executed by a processor of a method, apparatus (system), and computer program product according to embodiments of this application to create means for implementing the functions specified in one or more flowchart illustrations and / or one or more block diagrams. These computer program instructions may also be stored in a computer-readable storage medium capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowchart illustrations and / or one or more block diagrams. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more blocks of a block diagram.

[0075] It should be emphasized that the examples described in this invention are illustrative rather than limiting. Therefore, this invention is not limited to the examples described in the specific embodiments. Any other embodiments derived by those skilled in the art based on the technical solutions of this invention, without departing from the spirit and scope of this invention, whether modifications or substitutions, are also within the protection scope of this invention.

Claims

1. An adaptive edge target detection method based on hybrid lightweighting, characterized in that, include: Step 1: Set up the environment and data for the training server; Step 2: Construct a hybrid lightweight adaptive edge target detection model, which includes an EMO backbone network, a lightweight neck network, and a decoupled detection head connected in sequence; Step 3: Perform basic pre-training on the hybrid lightweight adaptive edge target detection model; Step 4: Perform quantization-aware training on the hybrid lightweight adaptive edge target detection model after basic pre-training to obtain a hybrid lightweight adaptive edge target detection model with two sets of quantization parameters. During the quantization perception training process, the various parts of the hybrid lightweight adaptive edge target detection model are grouped according to their functions, and the network parameters of each group are configured independently. At the same time, the weights of the network parameters of each group are applied using an exponential moving average. The quantization bit width configuration in the EMO backbone network consists of two sets of parameters, namely 4-bit and 3-bit quantization parameters. Step 5: Single device deployment and implementation; The two models trained by quantization perception are compiled separately to obtain two engine files, which are then deployed to the edge device along with the monitoring-scheduling program. Edge devices switch programs based on their own state and call the corresponding model to perform target detection on the acquired images.

2. The method according to claim 1, characterized in that, The lightweight neck network includes an SCConv module and a BiFPN network connected in sequence; In this process, the original intermediate nodes in the BiFFN network are removed while the core connection paths are retained. At the same time, learnable weight parameters are added to the cross-scale connections, and the range of learnable weight parameters is constrained by the ReLU6 activation function. Then, two feature fusion paths, one from top to bottom and the other from bottom to top, are established.

3. The method according to claim 1, characterized in that, The EMO backbone network comprises six sequentially connected downsampling stage modules: a first convolutional module, a second inverse residual module, a third inverse residual module, a fourth multi-head self-attention mechanism module, a fifth inverse residual module, and a sixth inverse residual module. The input channels, output channels, and strides of each stage's convolutional and inverse residual modules are (3, 24, 2), (24, 48, 2), (48, 96, 2), (96, 192, 2), and (192, 384, 2), respectively. The third, fifth, and sixth inverse residual modules each contain 3, 4, and 6 inverse residual modules, respectively. The fourth multi-head self-attention mechanism module has 8 heads and an attention dimension of 192.

4. The method according to claim 1, characterized in that, A front-end pixel attention module and a rear-end spatial adaptive attention module are set at the front end and back end of the EMO backbone network, respectively. The front pixel attention module includes a 1×1 convolutional layer, a sigmoid function, a multiplication module, and a bilinear interpolation module connected in sequence. The other input of the multiplication module is connected to the input of the 1×1 convolutional block. The post-space adaptive attention module includes a device status monitoring unit, two convolutional kernels of different sizes, and a switching unit. The switching unit selects and activates a convolutional kernel of a corresponding size based on the information obtained by the device monitoring unit. The sizes of the two convolutional kernels of different sizes are 5×5 and 3×3, respectively.

5. The method according to claim 1, characterized in that, The decoupled detection head includes parallel classification and regression branches. Each branch contains two 256-channel 3x3 convolutional layers and a 1x1 convolutional layer for outputting predicted values. The classification branch outputs the probabilities of 80 categories, and the regression branch outputs 4 coordinate values.

6. The method according to claim 1, characterized in that, The edge devices employ a master-slave communication architecture built with ZeroMQ, utilizing multi-camera collaboration for edge target detection. The camera acting as the master node is responsible for edge target detection, while the cameras acting as slave nodes assist in edge target detection. During the detection process, the initial weight of the master node is set to 0.

7. The weight of the slave nodes is dynamically calculated based on the preset relative spatial position relationship and viewing angle difference between the master and slave nodes, with a value range of 0.2-0.

3. The overlapping detection boxes obtained by multiple edge devices are used as the final edge target detection coordinates using a weighted average fusion method. The categories obtained by multiple edge devices are determined by confidence-weighted voting. The triggering condition for a node to participate in auxiliary detection is: Triggered when the detection confidence level obtained by the master node is lower than a set threshold; Triggered when the target is located at the edge of the image; This is triggered when the master node detects that the proportion of the target occlusion area exceeds the set proportion.

7. The method according to claim 6, characterized in that, When a node participates in assisted detection, the corresponding image sample is regarded as a difficult sample. Only all difficult samples are used as training data for quantization perception training of the hybrid lightweight adaptive edge target detection model.

8. The method according to claim 1, characterized in that, When applying an exponential moving average to the weights of the network parameters for each group, the EMA values ​​of the weight quantization range statistics and the activation value quantization range statistics are calculated according to the formula after each training batch: ; in, This represents the statistics for the current batch. Here, is the decay factor, and t is the sequence number of the current training batch. This refers to the sequence number of the previous batch in the current batch t; This indicates the EMA value saved after the previous batch was updated.

9. An adaptive edge target detection system based on hybrid lightweight design, characterized in that, include: Data acquisition terminal equipment is used to acquire image data to be detected; At the edge node, multiple cameras use the hybrid lightweight adaptive edge target detection method described in any one of claims 1-8 to perform edge target detection on image targets based on the collected data, and upload the detection data to the cloud center; The cloud center regularly aggregates data from various edge nodes and uses it to retrain and distribute a hybrid lightweight adaptive edge object detection model to the edge nodes.

10. The system according to claim 9, characterized in that, The multi-camera system has at least two cameras, which are located in different positions.

Images