Edge device target detection method
By using an improved RT-Betr network and an adaptive loss function, the problem of missed detection of small targets in edge device images is solved, thereby improving detection accuracy and computational efficiency.
Patent Information
- Application Number
- CN202511469671.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-15
- Publication Date
- 2026-03-06
AI Technical Summary
Images acquired by edge devices often exhibit large variations in target scale, a high proportion of small targets, complex backgrounds, and susceptibility to environmental interference. This makes it difficult for existing technologies to simultaneously ensure the recognition accuracy of both large and small targets. Furthermore, the subtle features of small targets are easily masked by noise, leading to missed detections.
An improved RT-Betr network is adopted, which introduces a multi-scale convergence-divergence enhanced attention module and a scale-aware normalized Wasserstein distance adaptive loss function. Combined with an adaptive channel pruning strategy, the perception capability and detection accuracy of small targets are improved.
It effectively improves the ability to perceive small targets, reduces the false negative rate, and improves the accuracy of target detection, while controlling the computational burden.
Smart Images
Figure CN121616809A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to image processing technology, and in particular to a method for detecting targets on edge devices. Background Technology
[0002] Images acquired by edge devices often have characteristics such as large target scale variation, high proportion of small targets, complex backgrounds, and susceptibility to environmental interference. This poses challenges to target detection. For example, large target scale variation makes it difficult for single-scale detection models to simultaneously ensure the recognition accuracy of large and small targets. Small targets have weak features in images, and their effective information is easily submerged by noise under the interference of complex backgrounds, leading to missed detection problems.
[0003] To address the aforementioned challenges and meet the computational efficiency requirements of edge devices, existing technologies typically employ lightweight backbone networks (such as MobileNet) to control computational costs and combine them with Feature Pyramid Networks (FPN) and their variants to fuse multi-scale features in an attempt to solve the scale variation problem. However, layer-by-layer fusion methods, represented by FPN, suffer from incomplete information transmission and insufficient feature interaction. This causes the weak features of small targets to further attenuate during transmission, making them easily confused with background noise and exacerbating the false negative problem. Therefore, there is an urgent need for a target detection method that achieves efficient and lossless interaction of multi-scale features, improving the perception of small and occluded targets without significantly increasing the computational burden, thereby reducing the false negative rate of small targets and improving target detection accuracy. Summary of the Invention
[0004] Purpose of the invention: The purpose of this invention is to provide a target detection method for edge devices that improves the perception of small targets and occluded targets without significantly increasing the computational burden, thereby reducing the false negative rate of small targets and improving the target detection accuracy.
[0005] Technical solution: To achieve the above objectives, the present invention provides a target detection method for edge devices, comprising the following steps:
[0006] Acquire several sets of images captured by edge devices, perform preprocessing and target ground truth bounding box annotation, and construct a dataset;
[0007] An improvement to the RT-Betr network is proposed to construct a target prediction model and train it using a dataset. The improvement includes retaining the backbone network of the RT-Betr network architecture, introducing a multi-scale convergence-divergence attention module into the Neck network, and optimizing the loss function. The multi-scale convergence-divergence attention module involves fusing multi-scale feature maps output from the backbone network to obtain global features, sampling these global features to obtain multi-scale distributed features, and enhancing the granularity of the multi-scale distributed features. The optimized loss function includes introducing a shape constraint term into the NWD metric. With scale-aware adaptive weights .
[0008] The trained target prediction model is lightweighted, and then used to predict targets in newly acquired edge device images.
[0009] Preferably, the preprocessing includes standardizing and normalizing the image, as well as random cropping, tone transformation, and multi-granularity scaling.
[0010] Preferably, the method for fusing the multi-scale feature maps output by the backbone network to obtain global features is as follows:
[0011] ,
[0012] In the formula, To merge convolutional layers; To stitch together the alignment features; This is a fine-grained feature map. This is a feature map for intermediate granularity. This is a coarse-grained feature map; For convolutional layers, the granularity is matched with the channels respectively; To make feature maps Upsampling to feature map .
[0013] Preferably, the method for sampling global features to obtain multi-scale distributed features is as follows:
[0014] ,
[0015] In the formula, For sampling operations, the fused global features are sampled to the original granularity.
[0016] Preferably, the method for granularizing the multi-scale distribution features is as follows:
[0017] ,
[0018] In the formula, GAPc and GMP c These represent global average pooling and global max pooling along the channel, respectively, f 7×7 For a standard convolutional layer with a kernel size of 7×7, MLP It is a multilayer perceptron. It is the sigmoid activation function. This is for element-wise multiplication.
[0019] Preferably, the optimized loss function is expressed as:
[0020] .
[0021] Preferably, the shape constraint item Represented as:
[0022] ,
[0023] In the formula, The absolute difference in aspect ratio between the predicted bounding box and the true bounding box of the target. The shape constraint weights balance the importance of the shape term, and the target predicted bounding box is represented as follows: Target true bounding box ,in, These represent the width and height of the bounding box, respectively.
[0024] Preferably, the scale-aware adaptive weights Represented as:
[0025] ,
[0026] In the formula, Optimize the gradient of the small target based on the width and height of the ground truth bounding box. The basic weights are used to ensure that large targets still have sufficient monitoring signals, and γ is the scale sensitivity coefficient.
[0027] Preferably, the NWD metric is expressed as:
[0028] ,
[0029] In the formula, This refers to the second-order Wasserstein distance between the predicted bounding box and the ground truth bounding box of the target, calculated based on a two-dimensional Gaussian distribution. The two-dimensional Gaussian distribution is the predicted bounding box of the target based on a target prediction model. and the target true bounding box The result of the conversion is represented as: , , They represent bounding boxes respectively. , The obtained covariance matrix, , Let represent the mean of a two-dimensional Gaussian distribution.
[0030] Preferably, the lightweight processing method is as follows: an adaptive channel pruning strategy is used to lightweight the trained target prediction model. Specifically, the retention ratio of each convolutional layer is first determined based on layer sensitivity analysis. On a calibration dataset, the importance score of each channel is calculated. Based on the determined retention ratio, the channel with the lowest importance score is dynamically pruned.
[0031] Beneficial Effects: The present invention has the following advantages: 1. The multi-scale convergence-divergence enhanced attention module of the present invention, by fusing global context information and distributing and granularizing features at multiple scales, can actively enhance the weak feature signals of small targets while suppressing the interference of background noise, effectively improving the perception ability of small targets and reducing the false detection rate; 2. In view of the problem that small target localization is extremely sensitive to small deviations in bounding boxes, the present invention introduces a scale-aware normalized Wasserstein distance adaptive loss function, which makes the target prediction model pay more attention to the boundary regression of small targets during training, thereby outputting more accurate bounding boxes during prediction and improving the target detection accuracy. Attached Figure Description
[0032] Figure 1 This is a schematic diagram of the method flow of the present invention. Detailed Implementation
[0033] The technical solution of the present invention will be described in detail below with reference to the embodiments and accompanying drawings.
[0034] like Figure 1 As shown, the edge device target detection method of the present invention includes the following:
[0035] S1. Acquire several sets of images collected by edge devices, perform preprocessing and label the true bounding boxes of the targets, and construct a dataset.
[0036] The preprocessing includes standardizing and normalizing the image, while also performing data augmentation using random cropping, tone transformation, and multi-granular scaling.
[0037] S2. Improve the RT-Betr network to build a target prediction model and train it using a dataset.
[0038] The RT-Betr network architecture comprises a backbone network, a Neck network, and a Transformer decoder. The backbone network (e.g., HGNet-v2) performs hierarchical feature extraction on the input edge device image through multiple consecutive convolutional stages, generating a series of multi-scale feature maps with granularity ranging from fine to coarse. The Neck network fuses multi-scale image features from the backbone network through a structure similar to FPN (Feature Pyramid Network). Then, a Transformer-based encoder performs deep processing on these fused features to generate a unified bird's-eye view feature representation. The Transformer decoder is responsible for decoding the final detection result, such as the bounding box of the target, from these BEV features.
[0039] The improvements to the RT-Betr network in this invention include: retaining the original backbone network, introducing a multi-scale convergence-divergence enhanced attention module in the Neck network to replace the FPN structure, thereby enhancing the ability to extract multi-granular features of small targets; and, during the training phase, using a scale-aware normalized Wasserstein distance adaptive loss function for loss calculation to further improve the localization accuracy of small targets.
[0040] The multi-scale convergence-divergence enhanced attention module performs multi-scale feature maps Clustering and granularity enhancement processes are performed to improve the target prediction model's ability to perceive small-scale targets and its resistance to occlusion. Clustering involves fusing multi-scale feature maps to obtain global features. :
[0041] ,
[0042] In the formula, To fuse convolutional layers, feature maps rich in multi-granularity global context information are obtained. ; This indicates that the aligned features will be spliced together; The feature map is at an intermediate granularity, taking into account both semantic and spatial information, and preventing excessive information loss due to granularity aggregation. For convolutional layers, the granularity is matched with the channels respectively; To make feature maps Upsampling to feature map By fusing shallow details with deep semantics, the global features contain more information about small targets.
[0043] Then, for global features The original feature scale is recovered by sampling and then added to the original feature map to obtain the distributed features. :
[0044] ,
[0045] In the formula, The sampling operation samples the fused global features to the original granularity. The distribution process injects global information containing small targets into each scale, thereby enhancing the representation ability of small targets at each scale.
[0046] Granularity enhancement processing is the distribution of features at different scales. By enhancing the response in key areas and suppressing redundant information, the target prediction model's ability to distinguish small targets in complex scenes is improved. The enhanced distribution features are represented as follows:
[0047] ,
[0048] In the formula, GAP c and GMP c These represent global average pooling and global max pooling along the channel, respectively, f 7×7 For a standard convolutional layer with a kernel size of 7×7, MLP It is a multilayer perceptron. It is the sigmoid activation function. This is for element-wise multiplication.
[0049] In the granular enhancement process, the distributed features at each scale are compressed in terms of channel dimension through global average pooling and global max pooling to generate spatial attention feature maps, capturing important spatial regions. Simultaneously, standard convolutional layers are used to fuse the spatial attention feature maps, and a multilayer perceptron is used to calculate channel attention weights. The outputs of the two paths are then weighted and fused before Sigmoid activation, generating a collaborative attention map that simultaneously enhances key feature channels and important spatial regions. By strengthening feature channels sensitive to small targets through channel weighting and suppressing interference from irrelevant channels, small target regions are highlighted, while activation responses in background regions are suppressed. This significantly improves the target prediction model's feature discrimination ability for multi-scale targets and its robustness against complex backgrounds.
[0050] The multi-scale convergence and divergence enhanced attention module effectively integrates and strengthens multi-scale target features from images from edge devices, enhancing the robustness of the target prediction model for detecting small targets on edge devices.
[0051] In subsequent operations, the enhanced multi-scale feature map output by the multi-scale convergence-divergence enhancement attention module will be used. Input the Transformer decoder to decode the bounding box of the target.
[0052] The design process of the scale-aware normalized Wasserstein distance adaptive loss function is as follows:
[0053] First, the target prediction bounding box predicted by the target prediction model. and the target true bounding box Convert to a two-dimensional Gaussian distribution: One of them has a width and height of The covariance matrix corresponding to the bounding box is:
[0054] .
[0055] Based on a two-dimensional Gaussian distribution, calculate the second-order Wasserstein distance between the predicted bounding box and the true bounding box of the target:
[0056]
[0057] In the formula, Represents the L2 distance. It represents the L² norm (squared Euclidean distance). This represents the square of the Frobenius norm.
[0058] The normalized second-order Wasserstein distance yields the original NWD metric. The closer the NWD metric value is to 0, the higher the similarity between the predicted bounding box and the ground truth bounding box. The NWD metric is expressed as:
[0059] ,
[0060] In the formula, C represents the constant term. It represents a power of e to the base e.
[0061] Introducing aspect ratio shape constraints This is used to enhance the sensitivity of the loss function to the shape of the bounding box (especially the aspect ratio), and is a shape constraint term. Represented as:
[0062] ,
[0063] In the formula, The absolute difference in aspect ratio between the predicted bounding box and the true bounding box of the target. These are the shape constraint weighting coefficients, used to balance the importance of the shape term.
[0064] Reference scale-aware adaptive weights The optimization intensity is dynamically adjusted according to the target scale, allowing smaller target regressions to generate larger gradients, thereby mitigating the instability problem of smaller target regressions. This is a scale-aware adaptive weighting system. Represented as:
[0065] ,
[0066] In the formula, The scale (size) of the real target is inversely proportional to the weights, optimizing the gradient for small targets. The base weights ensure that large targets still have sufficient monitoring signals, while γ is a scale sensitivity coefficient that controls the magnitude of the weight enhancement for small targets.
[0067] Shape constraints By incorporating scale-aware adaptive weights into the NWD metric, an improved loss function is obtained:
[0068] .
[0069] S3. An adaptive channel pruning strategy is used to lightweight the trained target prediction model.
[0070] The retention ratio of each convolutional layer (an independent convolutional computation unit in the target prediction model) is determined based on layer sensitivity analysis. On a calibration dataset, the importance score of each channel (the basic building block inside the convolutional layer) is calculated, and the channel with the lowest importance score is dynamically pruned according to a determined retention ratio.
[0071] ,
[0072] in: The importance score for the c-th channel. Let be the activation weight of the c-th channel in the i-th sample, and N be the number of samples in the calibration dataset.
[0073] S4. Use a lightweight target prediction model to predict targets in newly acquired edge device images.
Claims
1. An edge device target detection method, characterized by, The method comprises the following steps: Obtaining images collected by a plurality of edge devices, and performing preprocessing and real boundary box labeling of the target to construct a data set; The RT-Betr network is improved to construct a target prediction model and train the data set; the improvement includes retaining a backbone network in the RT-Betr network architecture, introducing a multi-scale convergence enhancement attention module in the Neck network, and optimizing a loss function; the multi-scale convergence enhancement attention module includes fusing multi-scale feature maps output by the backbone network to obtain global features, sampling the global features to obtain multi-scale distribution features, and performing granularity enhancement on the multi-scale distribution features; the optimized loss function includes introducing a shape constraint term in the NWD metric with scale-aware adaptive weights .
2. Lightening the trained target prediction model, and using the lightened target prediction model to predict the target of the newly collected edge device image.
3. The edge device target detection method of claim 1, wherein, The preprocessing comprises standardization and normalization of the image, and random cropping, tone transformation and multi-granularity scaling processing.
4. The edge device target detection method of claim 1, wherein, The method for fusing the multi-scale feature maps output by the backbone network to obtain global features is: ， In the formula, is a fusion convolution layer; is to splice the alignment features; is a fine-grained feature map, is an intermediate-grained feature map, is a coarse-grained feature map; is a convolution layer, respectively matching the granularity and the channel; is to splice the feature maps up-sampling to the feature map .
5. The edge device target detection method of claim 3, wherein, The method for sampling the global features to obtain multi-scale distribution features is: ， In the formula, For the sampling operation, the fused global features are respectively sampled to the original granularity.
6. The edge device target detection method of claim 4, wherein, The method for enhancing the granularity of the multi-scale distribution features is: ， In the formula, GAP c and GMP c These represent global average pooling and global max pooling along the channel, respectively, f 7×7 For a standard convolutional layer with a kernel size of 7×7, MLP It is a multilayer perceptron. It is the sigmoid activation function. This is for element-wise multiplication.
7. The edge device target detection method of claim 1, wherein, The optimized loss function is expressed as: 。 8. The edge device target detection method of claim 6, wherein, The shape constraint term is represented as: ， wherein, is the absolute difference of the aspect ratio between the target predicted bounding box and the target real bounding box, is the shape constraint weight coefficient, balancing the importance of the shape term, the target predicted bounding box is represented as , the target real bounding box wherein, respectively represent the width and height of the bounding box.
9. The edge device target detection method as claimed in claim 6, characterized by, The scale-aware adaptive weight is represented as: ， In the formula, is the width and height of the target real bounding box, is the base weight, ensuring that large targets still have enough supervision signal, and γ is the scale-sensitive coefficient.
10. The edge device target detection method of claim 6, wherein, The NWD metric is expressed as: ， In the formula, is the second-order wasserstein distance between the target prediction bounding box and the target real bounding box calculated based on a two-dimensional Gaussian distribution, the two-dimensional Gaussian distribution is based on the target prediction bounding box predicted by the target prediction model and the target real bounding box is converted to, represented as: , , respectively represent the covariance matrix obtained based on the bounding box , , , respectively represent the mean of the two-dimensional Gaussian distribution.
11. The edge device target detection method of claim 1, wherein, The lightening method is: using an adaptive channel pruning strategy to lighten the trained target prediction model, specifically, first determining the retention ratio of each convolution layer based on layer sensitivity analysis, then calculating the importance score of each channel on a calibration data set, and finally cutting off the channel with the lowest importance score according to the determined retention ratio.