Unmanned aerial vehicle small target detection method and system based on feature reconstruction and distance constraint

Abstract

The application provides a kind of feature reconstruction and distance constraint unmanned aerial vehicle small target detection method, utilize backbone feature extraction module and extract the multiscale basic feature of preprocessed image;Through feature reconstruction module, the semantic information and detail information of small target are supplemented by fusing high and low layer features, and enhanced detection features are generated;Then introduce Gaussian combination distance loss and classification loss, bounding box regression loss, update detection network parameters;According to the updated detection network parameters, the above process is re-executed to obtain the detection result, and the detection result is superimposed and displayed on the original image or saved as structured data, completing the detection task.The application can identify and locate small-scale vehicles, pedestrians, cyclists and other targets in unmanned aerial vehicle aerial images or video frames, and is suitable for unmanned aerial vehicle road inspection, campus road monitoring, parking area monitoring, low-altitude security patrol, emergency rescue target search and natural resource investigation and other common unmanned aerial vehicle visual perception scenes.

Description

Technical Field

[0001] This invention relates to the fields of computer vision, UAV intelligent perception, target detection and deep learning, and more specifically to a method and system for detecting small targets in UAVs based on feature reconstruction and distance constraints. Background Technology

[0002] Unmanned aerial vehicle (UAV) intelligent perception is a comprehensive visual perception system integrating image acquisition, target recognition, environmental understanding, and task-aided decision-making. It can automatically or semi-automatically acquire information about roads, vehicles, pedestrians, buildings, and natural features in low-altitude scenarios. Currently, UAV aerial target detection primarily relies on onboard cameras to acquire images or videos, which are then identified by ground stations, servers, or onboard computing platforms. With the development of UAV platforms, embedded computing devices, and deep learning visual algorithms, the automatic detection of vehicles, pedestrians, cyclists, and other targets using UAV images has become a crucial technology in scenarios such as traffic inspection, campus road monitoring, low-altitude security, urban governance, emergency rescue, and natural resource surveys.

[0003] However, in drone aerial photography scenarios, targets are typically small in size, have complex backgrounds, exhibit significant lighting variations, and are densely packed, posing a significant challenge to the accurate detection of small targets. Existing technologies are prone to issues such as loss of feature details, severe background interference, and unstable bounding box regression during the training phase when processing small targets. Some detection methods optimized for small targets rely heavily on multi-scale feature fusion or high-resolution input, leading to a large amount of redundant computation, which is insufficient to meet the real-time detection requirements of drone platforms and is ill-suited to the actual needs of small target detection on drones. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for small target detection in UAVs based on feature reconstruction and distance constraints, in order to solve the problems of easy loss of small target features, low detection accuracy, and large computational load of optimization methods that are difficult to meet the real-time detection requirements of UAV platforms.

[0005] This invention is implemented as follows: a method for detecting small targets in unmanned aerial vehicles (UAVs) based on feature reconstruction and distance constraints, comprising the following steps:

[0006] S1: Acquire image data collected by the drone and preprocess the image data;

[0007] S2: The backbone feature extraction module performs multi-scale feature extraction on the preprocessed image data and outputs a multi-scale feature layer containing shallow detail features and high-level semantic features.

[0008] S3: During the feature extraction process, the feature layer is reconstructed using the column-aware feature reconstruction module. The response of the suspected target column is enhanced and the interference of continuous background columns is suppressed by column-direction context aggregation.

[0009] S4: Encode high-level semantic features using a multi-scale linear attention module to generate enhanced high-level semantic features;

[0010] S5: The shallow detail features are fused with the enhanced high-level semantic features through the multi-scale feature fusion module to obtain the fused multi-scale features.

[0011] S6: Input the fused features into the detection and decoding module, and output the target category, confidence level and predicted bounding box of the image data;

[0012] S7: During the model training phase, the classification loss, L1 bounding box regression loss, and Gaussian combined distance loss between the predicted bounding box and the ground truth bounding box are calculated using the training constraint module. The classification loss, L1 bounding box regression loss, and Gaussian combined distance loss are weighted and summed according to preset loss weight coefficients to obtain the total loss. Backpropagation is performed on the feature reconstruction detection module based on the total loss, and the trainable parameters of the feature reconstruction detection module are updated through the optimizer.

[0013] S8: Based on the updated parameters, re-execute the process from S1 to S6 to obtain the detection results of the image data, including the target category, confidence score and predicted bounding box. The detection results are then overlaid on the original image or saved as structured data to complete the detection task.

[0014] Furthermore, the present invention can be implemented according to the following technical solution:

[0015] In step S3, the column-aware feature reconstruction module includes a column-aware branch, a spatial attention branch, and a self-calibration branch. The reconstruction of the feature layer using the column-aware feature reconstruction module specifically includes:

[0016] By performing pooling along the feature height direction through the column-aware branch, column direction weights are generated to enhance the response of the column where the suspected target is located;

[0017] Spatial location weights are generated through spatial attention branches to highlight the possible spatial locations of the target;

[0018] A wide range of contextual information is introduced through a self-calibration branch to suppress background interference from road textures, tree shadows, or building edges;

[0019] The outputs of the column perception branch, spatial attention branch, and self-calibration branch are fused together and then added to the input features as residuals to obtain the reconstructed features.

[0020] In step S4, the operation of the multi-scale linear attention module includes:

[0021] The high-level semantic features are divided into four branches according to channels, and the local context information is extracted by depthwise convolution with different kernel sizes in each of the four branches.

[0022] Each branch generates query features, key features, and value features, and uses linear attention for feature encoding;

[0023] The outputs of each branch are weighted, concatenated, and fused by convolution to obtain the enhanced high-level semantic features.

[0024] In step S7, the Gaussian combined distance loss is used to constrain the center offset and scale difference of the predicted bounding box relative to the ground truth bounding box; the calculation of the Gaussian combined distance loss between the predicted bounding box and the ground truth bounding box using the training constraint module specifically includes:

[0025] The predicted bounding box and the true bounding box are represented by two-dimensional Gaussian distributions, with the mean being the coordinates of the center point of the bounding box and the covariance being determined by the width and height of the bounding box.

[0026] Calculate the center distance and scale difference terms between the predicted bounding box and the true bounding box;

[0027] Gaussian combined distance is obtained by normalizing the width and height of both the predicted bounding box and the ground truth bounding box simultaneously.

[0028] The Gaussian combined distance is mapped to a bounded loss function and used as a bounding box regression loss term in backpropagation to update network parameters.

[0029] The square of the Gaussian combined distance, Dgc²,

[0030] The formula for calculating the square of the Gaussian combination distance, Dgc², is: Dgc² = 1 / 2[(xp-xg)² / (wp²+ε) +(yp-yg)² / (hp²+ε) + (xp-xg)² / (wg²+ε) + (yp-yg)² / (hg²+ε)] + 1 / 8[(wp-wg)² / (wp²+ε) + (hp-hg)² / (hp²+ε) + (wp-wg)² / (wg²+ε) + (hp-hg)² / (hg²+ε)];

[0031] Where (xp,yp,wp,hp) are the center coordinates and width and height of the predicted bounding box; (xg,yg,wg,hg) are the center coordinates and width and height of the true bounding box; and ε is a constant to prevent the denominator from being zero.

[0032] The Gaussian combined distance Dgc is obtained by taking the square root of the square of the Gaussian combined distance Dgc², and then the Gaussian combined distance Dgc is mapped to the Gaussian combined distance loss LGCD = 1 - exp(-Dgc).

[0033] The backbone feature extraction module outputs four scale feature layers, P2, P3, P4 and P5, with strides of 4, 8, 16 and 32 respectively.

[0034] The column-aware feature reconstruction module operates during the shallow-to-high-level feature transfer process of the backbone feature extraction module, preserving the detailed features of small targets in the P2 feature.

[0035] The multi-scale feature fusion module adopts a top-down and bottom-up approach to fuse P2, P3, and P4 features with P5 features encoded by the multi-scale linear attention module.

[0036] The detection decoding module adopts an end-to-end ensemble prediction process, using object query, attention head and decoder layer to directly output detection results;

[0037] The training constraint module only participates in loss calculation during the training phase and does not add any additional network branches during the inference phase. During the inference phase, the detection decoding module directly outputs the target category, confidence level, and bounding box coordinates.

[0038] A UAV small target detection system based on feature reconstruction and distance constraints, comprising:

[0039] A drone image acquisition system is used to acquire images captured by drone cameras, drone video streams, or offline aerial image data.

[0040] A small target detection system, installed in a computing platform, is used to perform the method as described in any one of claims 1 to 7;

[0041] The small target detection system includes an image preprocessing module, a feature reconstruction detection module, a training constraint module, and a detection result output module. The image preprocessing module performs image resizing, pixel normalization, and bounding box coordinate transformation on the input image. The feature reconstruction detection module includes a backbone feature extraction unit, a column-aware feature reconstruction unit, a multi-scale linear attention unit, a multi-scale feature fusion unit, and a detection decoding unit. The training constraint module calculates the total loss based on the predicted category, predicted bounding box, and ground truth bounding box during the training phase and updates the trainable parameters of the feature reconstruction detection module based on the total loss. The detection result output module outputs the target category, confidence level, and predicted bounding box coordinates during the inference phase.

[0042] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a feature reconstruction and distance-constrained UAV small target detection method as described in any of the preceding claims.

[0043] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a feature reconstruction and distance-constrained method for small target detection in unmanned aerial vehicles as described in any of the preceding claims.

[0044] The core technical concept of this invention lies in the following: First, during the shallow-to-high-level feature transfer process, the details of the small target at level P2 are preserved. Then, within the backbone network, a column-aware feature reconstruction module is used to perform differentiated reconstruction of the suspected target column and the background interference column. Subsequently, multi-scale linear attention is used only on the high-level features at level P5 to obtain low-cost context. Finally, during the training phase, Gaussian combined distance is used to constrain the center offset and scale difference of the predicted bounding box of the small target. The above processing works continuously in the three stages of feature formation, semantic encoding, and box regression, forming a collaborative detection process for small targets in UAVs. The main innovation of this invention is not simply adding network modules, but constructing a "feature reconstruction-distance constraint" collaborative detection mechanism for small targets in UAVs: In the feature extraction stage, the suspected small target column is enhanced and continuous background columns are suppressed through column-aware feature reconstruction; in the high-level semantic stage, multi-scale linear attention is used to supplement the local context with low computational cost; in the training phase, Gaussian combined distance is used to jointly constrain the center offset and scale difference between the predicted bounding box of the small target and the real bounding box. The above mechanisms respectively address the problems of detail loss, background interference, and unstable non-overlapping box regression in UAV small target detection, thus forming a complete technical closed loop.

[0045] The UAV small target detection method and system based on column-aware feature reconstruction and Gaussian combined distance provided by this invention have the following beneficial effects:

[0046] (1) Improve the separability of small target features: By introducing a column-aware feature reconstruction module in the backbone network feature extraction process, the column of suspected small targets is enhanced and the continuous background column is suppressed, which effectively improves the separability of small target features in complex background and reduces background interference.

[0047] (2) Reduce computational overhead: By using a multi-scale linear attention module, multiple receptive fields are encoded only for high-level features, which introduces local contextual information while avoiding the high computational cost of the standard self-attention mechanism.

[0048] (3) Stable bounding box regression: By using Gaussian combined distance constraints, the center offset and scale difference between the predicted bounding box and the real bounding box are constrained simultaneously during the training phase. Even if the two do not overlap or overlap is extremely small, an effective regression signal can be provided, which improves the stability of small target box regression.

[0049] (4) Maintaining efficient inference: Gaussian combination distance constraint only participates in loss calculation during the training phase, and does not add extra network branches during the inference phase, thus maintaining the simplicity and efficiency of end-to-end detection.

[0050] (5) Wide range of applicable scenarios: This invention can identify and locate small-scale targets such as vehicles, pedestrians, and cyclists in drone aerial images or video frames. It is applicable to common drone visual perception scenarios such as drone road inspection, campus road monitoring, parking area monitoring, low-altitude security patrol, emergency rescue target search, and natural resource survey.

[0051] (6) Strong system compatibility: The present invention uses an end-to-end target detection network for identification, which is separate from specific UAV platforms, camera models and computing platforms. It can be applied to ground station servers, workstations, edge computing devices or airborne computing platforms. Since feature extraction, background interference suppression, multi-scale feature fusion and bounding box regression are all completed independently by the small target detection system, the dependence on UAV platform type and image acquisition device model is reduced. Attached Figure Description

[0052] Figure 1 This is the overall architecture diagram of the UAV small target detection system of the present invention.

[0053] Figure 2 This is an offline training structure diagram of the UAV small target detection system of the present invention.

[0054] Figure 3 This is a diagram of the online detection structure of the UAV small target detection system of the present invention.

[0055] Figure 4 This is a system structure diagram of the CLG-DETR detection network of the present invention.

[0056] Figure 5 This is a structural diagram of the column perception feature reconstruction module of the present invention.

[0057] Figure 6 This is a structural diagram of the multi-scale linear attention module of the present invention.

[0058] Figure 7 This is a schematic diagram of the Gaussian combination distance training constraint of the present invention.

[0059] Figure 8 This is a data flow diagram of the small target detection process of the UAV in this invention.

[0060] Figure 9 These are the feature responses and detection results of this invention in various drone aerial photography scenarios. Detailed Implementation

[0061] like Figure 1 As shown, this invention provides a method and system for small target detection on unmanned aerial vehicles (UAVs) based on feature reconstruction and distance constraints. The UAV small target detection system includes a UAV image acquisition module, an image preprocessing module, a feature reconstruction detection module, a training constraint module, and a detection result output module. Specifically, the UAV image acquisition module acquires images captured by a UAV camera, UAV video streams, or offline aerial image data; the image preprocessing module resizes and normalizes pixels in the input image; the feature reconstruction detection module performs backbone feature extraction, column-aware feature reconstruction, multi-scale linear attention encoding, multi-scale feature fusion, and detection decoding on the preprocessed image tensor, and outputs the detection decoding results; the training constraint module calculates classification loss, L1 bounding box regression loss, and Gaussian combined distance loss based on the predicted category, predicted bounding box, and ground truth bounding box during the training phase, and updates the trainable parameters of the feature reconstruction detection module based on the total loss; the detection result output module outputs the target category, confidence level, and predicted bounding box coordinates based on the detection decoding results during the inference phase.

[0062] The UAV small target detection method includes an offline training phase and an online detection phase. In the offline training phase, the trainable parameters of the feature reconstruction detection module are updated through a training constraint module. In the online detection phase, the training constraint module is not used; instead, the image to be detected is processed by an image preprocessing module and then input into the feature reconstruction detection module, with the detection result output by a detection result output module. In this process, the feature reconstruction chain is used to improve the feature representation capability of small targets in UAV images, and the distance constraint chain is used to improve the regression stability of the predicted bounding boxes of small targets; both work together to improve the UAV small target detection performance.

[0063] The method includes the following steps:

[0064] S1: Acquire image data collected by the drone and preprocess the image data;

[0065] Image data sources include real-time images captured by drone cameras, video frames extracted from drone video streams, aerial images transmitted from ground stations, and images from publicly available drone aerial datasets. The targets to be detected in the image data include vehicles, pedestrians, cyclists, and other target categories defined according to actual application needs. For publicly available datasets, the VisDrone-2019 dataset can be used as the primary training and validation data, or aerial images converted to horizontal bounding boxes using DOTA-v1.0 can be used as supplementary validation data. For self-collected data, the original aerial images, target categories, and corresponding bounding box annotations should be saved.

[0066] A small target detection annotation file for UAVs is established. This annotation file records the target's category, center point coordinates, width, and height in each image, or the coordinates of the top-left and bottom-right corners of the target's bounding box. Data from different sources is uniformly converted to an annotation format readable by the detection network. If the original data is a rotated bounding box annotation, the four vertices of the rotated box are converted into horizontal bounding rectangles to correspond with the horizontal bounding box detection process described in this invention.

[0067] Establish training, validation, and test sets. The training set is used to update the detection network parameters, the validation set is used to evaluate the detection performance during training, and the test set or external validation data is used to test the model's generalization ability in different drone scenarios. When dividing the data, it is necessary to cover different scenarios such as strong light, shadow, backlight, complex road textures, dense targets, target occlusion, and sparse small targets as much as possible to ensure that the trained model is suitable for real-world drone application environments.

[0068] The data preprocessing module converts the training image size, pixel values, and bounding box coordinates into a format that the CLG-DETR detection network can process. This involves resizing, pixel normalizing, and data augmentation of the UAV aerial images. The input image size can be set to 640x640. The module includes steps for constructing both the offline training structure and the online detection structure for the UAV small target detection system.

[0069] It should be noted that the CLG-DETR detection network referred to in this invention is an abbreviation for the feature reconstruction detection module described in this invention. CLG stands for Column-aware feature reconstruction, Linear attention, and Gaussian composite distance, corresponding to column-aware feature reconstruction, linear attention, and Gaussian composite distance, respectively. DETR stands for end-to-end object detection framework. The Gaussian composite distance is used for bounding box regression constraints during the training phase and is not used as an additional network branch in the inference phase. Therefore, the CLG-DETR detection network is not the name of any existing publicly available detection network, but rather a feature reconstruction and distance-constrained detection structure constructed in this application for the small target detection task of UAVs.

[0070] The feature reconstruction detection module corresponding to the CLG-DETR detection network includes a backbone feature extraction unit, a column-aware feature reconstruction unit, a multi-scale linear attention unit, a multi-scale feature fusion unit, and a detection decoding unit; the image preprocessing module is not part of the internal structure of the feature reconstruction detection module, and its output preprocessed image tensor is input to the backbone feature extraction unit.

[0071] like Figure 2 As shown, in the offline training structure, training images and labeled files are input to the image preprocessing module. The image preprocessing module converts the training images into image tensors of a preset size and normalized format, and converts the bounding box coordinates to a label format corresponding to the network input size. The preprocessed image tensors are input to the backbone feature extraction unit in the feature reconstruction and detection module; the converted ground truth bounding boxes are input to the training constraint module. The feature reconstruction and detection module outputs the predicted class and predicted bounding boxes. The training constraint module calculates the classification loss, L1 bounding box regression loss, and Gaussian combined distance loss based on the predicted class, predicted bounding boxes, and ground truth bounding boxes, respectively, and then weights and sums these losses to obtain the total loss. The trainable parameters of the feature reconstruction and detection module are then updated through backpropagation and an optimizer. After training, the system saves the detection network weights of the feature reconstruction and detection module.

[0072] like Figure 3 As shown, in the online detection structure, images transmitted from a UAV camera, UAV video stream, or ground station are input to the image preprocessing module. The image preprocessing module resizes and normalizes the pixels of the image to be detected, obtaining a preprocessed image tensor, which is then input into the backbone feature extraction unit of the feature reconstruction detection module. The feature reconstruction detection module sequentially performs backbone feature extraction, column-aware feature reconstruction, multi-scale linear attention encoding, multi-scale feature fusion, and detection decoding, outputting the target category, confidence score, and predicted bounding box coordinates. The training constraint module is not used in the online detection phase.

[0073] The CLG-DETR detection network directly outputs the target category, confidence score, and bounding box coordinates. The detection results are then overlaid on the original aerial image or video frame, or saved as structured data. This structure does not require a Gaussian combined distance module during the inference stage, thus avoiding the addition of extra inference branches.

[0074] S2: The backbone feature extraction module performs multi-scale feature extraction on the preprocessed image data and outputs a multi-scale feature layer containing shallow detail features and high-level semantic features.

[0075] like Figure 4 As shown, the CLG-DETR detection network includes an image preprocessing module, a backbone feature extraction module, a column-aware feature reconstruction module, a multi-scale linear attention module, a multi-scale feature fusion module, and a detection decoding module. The backbone feature extraction module extracts image features at different scales, and the input image size can be set to 640×640. For a 640×640 input image, the backbone feature extraction module outputs four scale feature layers: P2, P3, P4, and P5, corresponding to small target detail preservation, local structure representation, and high-level semantic representation, respectively. Their strides are 4, 8, 16, and 32, and the corresponding feature map sizes are 160×160, 80×80, 40×40, and 20×20. The P2 feature retains more edge, texture, and positional information of small targets, while the P3 to P5 features provide different levels of semantic information.

[0076] S3: During the feature extraction process, the column-aware feature reconstruction module is used to reconstruct the feature layer, and the output includes a feature layer with at least four scales from P2 to P5. The response of the suspected target column is enhanced and the interference of continuous background columns is suppressed by column-direction context aggregation.

[0077] like Figure 5 As shown, the column-aware feature reconstruction module includes a column-aware branch, a spatial attention branch, and a self-calibration branch. The column-aware branch is introduced because road boundaries, lane lines, building edges, and tree shadows in UAV aerial images often form continuous background responses along a certain direction, while small targets such as vehicles, pedestrians, and cyclists usually form discrete local responses within several adjacent columns. By aggregating the column direction context, the response of suspected target columns can be enhanced and the interference of continuous background columns can be reduced. Specifically, it includes:

[0078] Pooling is performed along the feature height direction by using a column-aware branch, which is to perform context aggregation in the column direction and generate column-direction weights to enhance the response of the column where the suspected target is located, that is, to enhance the response of small targets or narrow local areas in the image data.

[0079] Spatial location weights are generated by spatial attention branches to highlight the spatial locations where the target may appear, i.e., to highlight the target area.

[0080] By introducing a large range of contextual information through downsampling, convolution, and upsampling paths via a self-calibration branch, it is used to suppress background interference from road textures, tree shadows, or building edges; and to reduce background interference from road textures, tree shadows, building edges, and areas of strong light.

[0081] The outputs of the column-aware branch, spatial attention branch, and self-calibration branch are fused and then added to the input features as residuals to obtain the reconstructed features. The column-aware feature reconstruction module operates during the shallow-to-high-level feature transfer process of the backbone feature extraction module, preserving the detailed features of small targets in the P2 feature set.

[0082] The specific processing flow of the column-aware feature reconstruction module is as follows: Let the input feature be F∈R^{C×H×W}, where C, H, and W represent the number of channels, height, and width, respectively. The column-aware branch first performs average pooling and max pooling on F along the height direction to obtain two column response descriptions Mc_avg∈R^{1×1×W} and Mc_max∈R^{1×1×W}; then, the two are concatenated along the channel dimension, and after 1×k convolution, normalization, and sigmoid activation, the column weight Ac∈R^{1×1×W} is obtained, where k can be 3, 5, or 7; finally, Ac is broadcast along the height and channel directions and multiplied with the input feature to obtain the column enhancement feature Fc. This column weight is used to highlight the column where the suspected target is located and suppress continuous background columns.

[0083] The column-aware feature reconstruction module is embedded in the backbone feature extraction path, rather than being used as a separate image restoration module. This is because UAV small target detection focuses more on the separability of the target region in the feature space than on restoring the original image. By reconstructing features during the backbone feature extraction process, this invention can improve small target response and reduce background interference while maintaining the original detection network information transmission path.

[0084] S4: Use a multi-scale linear attention module to perform multi-receptive field context encoding on high-level semantic features to generate enhanced high-level semantic features.

[0085] The multi-scale linear attention module operates by dividing high-level semantic features into four branches based on channels. Each branch employs depthwise convolution with kernels of different sizes to extract local contextual information within different receptive fields. Each branch generates query features, key features, and value features, which are then encoded using linear attention. The outputs of each branch are weighted, concatenated, and fused using a 1×1 convolution to obtain the enhanced high-level semantic features. This module reduces the overhead of pairwise computation between spatial locations inherent in standard self-attention while retaining the local multi-scale information required for small target detection by UAVs.

[0086] A multi-scale linear attention module is set up in the high-level P5 feature encoding stage. For example... Figure 6 As shown, the multi-scale linear attention module divides the P5 high-level features into four branches according to channels. The four branches use 3×3, 5×5, 7×7, and 9×9 depth convolutions to extract local contextual information under different receptive fields. Subsequently, each branch generates query features Q, key features K, and value features V, and performs feature encoding using linear attention. The outputs of each branch are then weighted, concatenated, and fused using 1×1 convolutions to obtain the enhanced high-level semantic features.

[0087] Specifically, for the output feature Xi of the i-th scale branch, the query feature Qi=XiWQi, key feature Ki=XiWKi, and value feature Vi=XiWVi can be obtained through linear mapping. The linear attention encoding can be expressed as Yi=φ(Qi)(φ(Ki)T Vi) / (φ(Qi)(φ(Ki)T 1)+ε), where φ represents a non-negative feature mapping function, φ(x) can be elu(x)+1 or relu(x)+δ, ε represents a small constant to avoid zero denominators, and 1 represents a vector of all 1s. This calculation method first aggregates the keys and values, then multiplies them with the query, avoiding the direct formation of an attention matrix of N times the number of spatial locations, thereby reducing the computational overhead of high-level feature context modeling.

[0088] S5: The multi-scale feature fusion module fuses shallow detail features with enhanced high-level semantic features to obtain fused multi-scale features. Specifically, the multi-scale feature fusion module performs multi-scale feature fusion on features P2, P3, P4, and P5. During the fusion process, high-level semantic information is passed to low-level features through a top-down path, while low-level detail information is supplemented to high-level features through a bottom-up path. Features P2, P3, and P4 are then fused with feature P5, which has been encoded by the multi-scale linear attention module.

[0089] S6: The fused features are input into the detection decoding unit, which outputs the target category, confidence score, and predicted bounding box. Specifically, the detection decoding unit outputs the target category, confidence score, and predicted bounding box coordinates based on the fused multi-scale features. The detection decoding unit adopts an end-to-end ensemble prediction process, directly outputting the detection results using object queries, attention heads, and decoder layers. In one specific implementation, the detection decoding unit can use four feature layers, 300 object queries, 8 attention heads, 3 decoder layers, and 128-dimensional hidden features.

[0090] S7: During the model training phase, the Gaussian combined distance loss between the predicted bounding box and the ground truth bounding box is calculated using the training constraint module to constrain the center offset and scale difference of the bounding box, and the detection network parameters are updated by combining the classification loss and L1 bounding box regression loss.

[0091] The Gaussian combined distance loss between the predicted bounding box and the ground truth bounding box is calculated using the training constraint module, specifically including:

[0092] First, Gaussian representations of the predicted and ground truth bounding boxes are constructed. Let the predicted bounding box be bp=(xp,yp,wp,hp) and the ground truth bounding box be bg=(xg,yg,wg,hg), where x and y represent the coordinates of the bounding box center point, and w and h represent the width and height of the bounding box. The predicted and ground truth bounding boxes are represented as two-dimensional Gaussian distributions, with their mean being the coordinates of the bounding box center point and their covariance determined by the width and height of the bounding box. This allows the positional offset and scale difference of the bounding boxes to be expressed in the same distance metric.

[0093] Then, the center distance and scale difference terms between the predicted bounding box and the true bounding box are calculated. For example... Figure 7 As shown, the center distance term describes the lateral and vertical offsets between the center points of the predicted bounding box and the ground truth bounding box, while the scale difference term describes the differences between the width and height of the predicted bounding box and the ground truth bounding box. Since the proportion of pixels for small targets on UAVs is low, the predicted and ground truth bounding boxes are prone to non-overlapping in the early stages of training. Simply relying on IoU-type overlap area constraints does not provide a clear enough direction for center position and scale adjustment. Therefore, this invention uses a Gaussian combined distance to simultaneously constrain both center position and scale difference.

[0094] The square of the Gaussian combined distance, Dgc², is expressed as:

[0095] Dgc² = 1 / 2[(xp-xg)² / (wp²+ε) + (yp-yg)² / (hp²+ε) + (xp-xg)² / (wg²+ε) +(yp-yg)² / (hg²+ε)] + 1 / 8[(wp-wg)² / (wp²+ε) + (hp-hg)² / (hp²+ε) + (wp-wg)² / (wg²+ε) + (hp-hg)² / (hg²+ε)]

[0096] Where (xp, yp, wp, hp) are the center coordinates and width / height of the predicted bounding box, (xg, yg, wg, hg) are the center coordinates and width / height of the ground truth bounding box, and ε is a constant to prevent the denominator from being zero. In the above formula, the component with a coefficient of 1 / 2 is the center distance normalization term, used to measure the lateral and longitudinal center offsets between the predicted and ground truth bounding boxes; the component with a coefficient of 1 / 8 is the scale difference normalization term, used to measure the width and height differences between the predicted and ground truth bounding boxes. By normalizing using both the width / height of the predicted and ground truth bounding boxes simultaneously, this distance is more sensitive to the positional offset of small-scale targets and can reduce the regression bias caused by single-scale normalization.

[0097] The square root of Dgc² is used to obtain the Gaussian combined distance Dgc, and the Gaussian combined distance Dgc is mapped to a bounded Gaussian combined distance loss:

[0098] LGCD = 1 - exp(-Dgc)

[0099] The final L1 bounding box regression loss is expressed as:

[0100] Lreg = λL1·LL1 + λGCD·LGCD

[0101] Where Lcls represents the classification loss, LL1 represents the L1 bounding box regression loss, LGCD represents the Gaussian combined distance loss, and λcls, λL1, and λGCD represent the loss weight coefficients of the above loss terms in the total loss. These loss weight coefficients are used to adjust the proportion of different loss terms in the total loss and are not trainable parameters of the feature reconstruction detection module. In a preferred embodiment, λcls is 1, λL1 is 5, λGCD is 2, and ε is 10^-7. The classification loss Lcls is calculated based on the predicted class probability and the true class label output by the detection decoding unit, while the L1 bounding box regression loss LL1 and the Gaussian combined distance loss LGCD are calculated based on the predicted bounding box and the true bounding box.

[0102] During model training, the detection decoding module outputs the predicted class probability, confidence score, and predicted bounding box for each object query. The training constraint module matches the predicted results with the ground truth labeled targets based on the Hungarian matching results. For the matched predictions, it calculates the classification loss Lcls based on the predicted class probability and the ground truth class label, and calculates the L1 bounding box regression loss LL1 and the Gaussian combined distance loss LGCD based on the predicted bounding box and the ground truth bounding box. The total loss is expressed as:

[0103] Ltotal = λcls·Lcls + λL1·LL1 + λGCD·LGCD

[0104] The training constraint module performs backpropagation on the feature reconstruction detection module based on the total loss and updates the trainable parameters of the feature reconstruction detection module through the optimizer until the detection performance of the module on the validation set stabilizes. After training, the detection network weights of the feature reconstruction detection module are saved for subsequent online detection. The training constraint module only participates in loss calculation during the training phase and does not add additional network branches during the inference phase. During online detection, the trained detection network weights of the feature reconstruction detection module are loaded, and forward inference is performed on the image to be detected. The detection decoding unit directly outputs the target category, confidence score, and predicted bounding box coordinates.

[0105] The training constraint module only participates in loss calculation during the training phase and does not add any additional network branches during the inference phase. During online detection, the detection network weights of the trained feature reconstruction detection module are loaded, and forward inference is performed on the image to be detected. The detection decoding module directly outputs the target category, confidence score, and predicted bounding box coordinates.

[0106] During offline training, the detection network parameters are updated by backpropagation by calculating the total loss until the model converges. During online detection, the trained detection network weights are loaded, and forward inference is performed on the image to be detected to directly output the target category, confidence score, and bounding box coordinates.

[0107] The detection results output of the CLG-DETR detection network include the following steps:

[0108] Load the trained CLG-DETR detection network weights. Input drone camera images, drone video frames, or offline aerial images into the image preprocessing module, and obtain the network input by adjusting the size and normalizing the pixels in the same way as in the training phase.

[0109] The preprocessed image is input into the CLG-DETR detection network. The detection network sequentially performs backbone feature extraction, column-aware feature reconstruction, multi-scale linear attention encoding, multi-scale feature fusion, and detection decoding, directly outputting the target category, confidence score, and bounding box coordinates.

[0110] The detection results output the target category, confidence level, and bounding box location. The detection results can be overlaid on UAV images or video frames, or saved as a data file containing image number, target category, confidence level, center point coordinates, width, and height for use in UAV inspection logs, target statistics, and subsequent task analysis.

[0111] The Gaussian combination distance is only used in loss calculation during the training phase and does not add any additional network branches during the inference phase. The data flow during the inference phase is as follows: Figure 8 As shown, the image is preprocessed and then enters the detection network. The detection network outputs the target category, confidence score, and bounding box. During the training phase, the predicted bounding box and the ground truth bounding box enter the Gaussian combined distance training constraint module. This module does not participate in the calculation during the inference phase.

[0112] S8: Based on the updated detection network parameters, re-execute the process from S1 to S6 to obtain the detection results of the image data, including the target category, confidence score, and predicted bounding box. The detection results are then overlaid on the original image or saved as structured data to complete the detection task.

[0113] Specifically, the data stream of this invention is divided into two stages. The first stage is the training stage, such as... Figure 2As shown, the data flow during the training phase is as follows:

[0114] First, prepare the training data. Collect drone aerial images and annotate them to create annotation files containing target categories and bounding box information. Divide the dataset into training and validation sets.

[0115] Next, data preprocessing is performed. The training images are uniformly resized to 640×640 pixels and pixel normalized. Simultaneously, the bounding box coordinates are converted to a format that the network can process.

[0116] Next, the preprocessed image is input into the CLG-DETR detection network (e.g., Figure 4 (As shown). The network first extracts features at four scales: P2, P3, P4, and P5, through a backbone feature extraction module (such as ResNet).

[0117] During feature extraction, the column-aware feature reconstruction module (such as...) Figure 5 (As shown) is embedded in the backbone network. This module reconstructs the feature map through column-aware branches, spatial attention branches, and self-calibration branches, enhancing the response of small target regions and suppressing background interference such as roads and tree shadows.

[0118] Subsequently, the P5 high-level features are fed into a multi-scale linear attention module (e.g., Figure 6 (As shown). This module supplements high-level features with rich local contextual information at a low computational cost through parallel multi-scale deep convolution and linear attention mechanisms.

[0119] Subsequently, the reconstructed and enhanced P2 to P5 features are fed into a multi-scale feature fusion module (such as FPN-PAN) for fusion, which integrates deep semantic information and shallow detail information.

[0120] The fused features are input into the detection decoding module (such as the decoder of RT-DETR), and the output is the predicted target category and bounding box.

[0121] Finally, training constraint modules (such as...) Figure 7 (As shown) it begins operation. It receives the network's predicted bounding boxes and the ground truth bounding boxes from the data, and calculates the classification loss, L1 bounding box regression loss, and Gaussian combined distance (GCD) loss. The GCD loss uses an innovative distance metric that simultaneously constrains the center point offset and scale difference between the predicted and ground truth bounding boxes; its calculation formula is as described above. The total loss is used to update the parameters of the entire detection network through the backpropagation algorithm.

[0122] After training, save the detection network weights that achieve the best performance.

[0123] Phase Two: Online Reasoning Phase, such as Figure 3As shown, the data flow during the inference phase is as follows:

[0124] Load the trained detection network weights.

[0125] Acquire real-time images or video frames of the drone to be detected.

[0126] Perform the same preprocessing on the images as during the training phase (size adjustment, pixel normalization).

[0127] The preprocessed image is input into the CLG-DETR detection network. The network sequentially performs backbone feature extraction, column-aware feature reconstruction, multi-scale linear attention encoding, multi-scale feature fusion, and detection decoding.

[0128] It is important to note that during the inference phase, the training constraint module (GCD module) does not participate in the computation, and therefore does not add any additional inference time.

[0129] The detection and decoding module directly outputs the final target category, confidence level, and bounding box coordinates.

[0130] The detection results are overlaid on the original image or saved as structured data to complete the detection task.

[0131] Example 1

[0132] The system described in this invention is implemented using the parameters shown in Table 1. The VisDrone-2019 UAV target detection dataset is used as the main validation data, covering small target categories such as vehicles, pedestrians, and cyclists, to verify the detection capability of this invention in dense target, complex background, and scale-changing scenarios.

[0133] like Figure 9 As shown, in various drone aerial photography scenarios such as densely packed parking lots, road intersections, tree shadows, long-distance roads, and complex traffic areas, this invention can detect small targets such as vehicles, pedestrians, and cyclists, and the heat map shows that the model has a strong feature response to the target area. Figure 9 The original image is used to display the input scene, the heatmap of this invention is used to display the network's region of interest, and the detection results of this invention are used to display the target category and bounding box output.

[0134] Table 1 Main parameters of specific embodiments

[0135]

[0136] On the VisDrone-2019 dataset, compared with the RT-DETR-R18 baseline model, the comparison results are shown in Table 2. The AP of the complete system of this invention increased from 25.8% to 30.9%, and AP_S increased from 17.7% to 22.9%. This result shows that the combined effect of P2 to P5 multi-scale features, column-aware feature reconstruction, multi-scale linear attention, and Gaussian combined distance can improve the detection performance of small targets by UAVs.

[0137] Table 2 Detection Effect Verification Results

[0138]

[0139] Example 2

[0140] Supplementary verification was performed using the horizontal bounding box verification protocol converted from DOTA-v1.0. Since the original DOTA-v1.0 annotations are rotated boxes, this embodiment converts the rotated quadrilaterals into horizontal bounding rectangles before detection and verification. Under this verification protocol, the model of this invention achieved 27.1% AP and 42.5% AP50, representing improvements of 1.4, 1.2, and 2.7 percentage points in AP, AP50, and AP_S compared to RT-DETR-R18. This result demonstrates that the present invention remains effective in another type of aerial image scenario.

[0141] It should be noted that the speed and detection results mentioned above were obtained under specific experimental conditions. For example, 41.65 FPS was measured under experimental conditions of an NVIDIA A4000 GPU, an input size of 640×640, and a batch size of 4, and should not be directly equated to the real-time latency of the UAV's onboard embedded device. If claims regarding onboard real-time detection performance are to be made subsequently, additional measured data from onboard or edge devices should be provided.

Claims

1. A method for detecting small targets on unmanned aerial vehicles (UAVs) based on feature reconstruction and distance constraints, characterized in that, Includes the following steps: S1: Acquire image data collected by the drone and preprocess the image data; S2: The backbone feature extraction module performs multi-scale feature extraction on the preprocessed image data and outputs a multi-scale feature layer containing shallow detail features and high-level semantic features. S3: During the feature extraction process, the feature layer is reconstructed using the column-aware feature reconstruction module. The response of the suspected target column is enhanced and the interference of continuous background columns is suppressed by column-direction context aggregation. S4: Encode high-level semantic features using a multi-scale linear attention module to generate enhanced high-level semantic features; S5: The shallow detail features are fused with the enhanced high-level semantic features through the multi-scale feature fusion module to obtain the fused multi-scale features. S6: Input the fused features into the detection and decoding module, and output the target category, confidence level and predicted bounding box of the image data; S7: During the model training phase, the classification loss, L1 bounding box regression loss, and Gaussian combined distance loss between the predicted bounding box and the ground truth bounding box are calculated using the training constraint module. The classification loss, L1 bounding box regression loss, and Gaussian combined distance loss are weighted and summed according to preset loss weight coefficients to obtain the total loss. Backpropagation is performed on the feature reconstruction detection module based on the total loss, and the trainable parameters of the feature reconstruction detection module are updated through the optimizer. S8: Based on the updated parameters, re-execute the process from S1 to S6 to obtain the detection results of the image data, including the target category, confidence score and predicted bounding box. The detection results are then overlaid on the original image or saved as structured data to complete the detection task.

2. The method for detecting small targets in unmanned aerial vehicles (UAVs) based on feature reconstruction and distance constraints according to claim 1, characterized in that, In step S3, the column-aware feature reconstruction module includes a column-aware branch, a spatial attention branch, and a self-calibration branch. The reconstruction of the feature layer using the column-aware feature reconstruction module specifically includes: By performing pooling along the feature height direction through the column-aware branch, column-direction weights are generated to enhance the response of the column where the suspected target is located; Spatial location weights are generated through spatial attention branches to highlight the possible spatial locations of the target; A wide range of contextual information is introduced through a self-calibration branch to suppress background interference from road textures, tree shadows, or building edges; The outputs of the column perception branch, spatial attention branch, and self-calibration branch are fused together and then added to the input features as residuals to obtain the reconstructed features.

3. The method for detecting small targets in unmanned aerial vehicles (UAVs) based on feature reconstruction and distance constraints according to claim 1, characterized in that, In step S4, the operation of the multi-scale linear attention module includes: The high-level semantic features are divided into four branches according to channels, and the local context information is extracted by depthwise convolution with different kernel sizes in each of the four branches. Each branch generates query features, key features, and value features, and uses linear attention for feature encoding; The outputs of each branch are weighted, concatenated, and fused by convolution to obtain the enhanced high-level semantic features.

4. The method for detecting small targets in unmanned aerial vehicles (UAVs) based on feature reconstruction and distance constraints according to claim 1, characterized in that, In step S7, the Gaussian combined distance loss is used to constrain the center offset and scale difference of the predicted bounding box relative to the ground truth bounding box; the calculation of the Gaussian combined distance loss between the predicted bounding box and the ground truth bounding box using the training constraint module specifically includes: The predicted bounding box and the true bounding box are represented by two-dimensional Gaussian distributions, with the mean being the coordinates of the center point of the bounding box and the covariance being determined by the width and height of the bounding box. Calculate the center distance and scale difference terms between the predicted bounding box and the true bounding box; Gaussian combined distance is obtained by normalizing the width and height of both the predicted bounding box and the ground truth bounding box simultaneously. The Gaussian combined distance is mapped to a bounded loss function and used as a bounding box regression loss term in backpropagation to update network parameters.

5. The method for detecting small targets in unmanned aerial vehicles based on feature reconstruction and distance constraints according to claim 4, characterized in that, The square of the Gaussian combined distance, Dgc², The formula for calculating the square of the Gaussian combination distance, Dgc², is: Dgc² = 1 / 2[(xp-xg)² / (wp²+ε) + (yp-yg)² / (hp²+ε) + (xp-xg)² / (wg²+ε) + (yp-yg)² / (hg²+ε)] + 1 / 8[(wp-wg)² / (wp²+ε) +(hp-hg)² / (hp²+ε) + (wp-wg)² / (wg²+ε) + (hp-hg)² / (hg²+ε)]; Where (xp,yp,wp,hp) are the center coordinates and width and height of the predicted bounding box; (xg,yg,wg,hg) are the center coordinates and width and height of the true bounding box; and ε is a constant to prevent the denominator from being zero. The Gaussian combined distance Dgc is obtained by taking the square root of the square of the Gaussian combined distance Dgc², and then the Gaussian combined distance Dgc is mapped to the Gaussian combined distance loss LGCD = 1 - exp(-Dgc).

6. The method for detecting small targets in unmanned aerial vehicles (UAVs) based on feature reconstruction and distance constraints according to claim 1, characterized in that, The backbone feature extraction module outputs four scale feature layers, P2, P3, P4 and P5, with strides of 4, 8, 16 and 32 respectively. The column-aware feature reconstruction module operates during the shallow-to-high-level feature transfer process of the backbone feature extraction module, preserving the detailed features of small targets in the P2 feature. The multi-scale feature fusion module adopts a top-down and bottom-up approach to fuse P2, P3, and P4 features with P5 features encoded by the multi-scale linear attention module.

7. The method for detecting small targets in unmanned aerial vehicles based on feature reconstruction and distance constraints according to claim 1, characterized in that, The detection decoding module adopts an end-to-end ensemble prediction process, using object query, attention head and decoder layer to directly output detection results; The training constraint module only participates in loss calculation during the training phase and does not add any additional network branches during the inference phase. During the inference phase, the detection decoding module directly outputs the target category, confidence level, and bounding box coordinates.

8. A UAV small target detection system based on feature reconstruction and distance constraints, characterized in that, include: A drone image acquisition system is used to acquire images captured by drone cameras, drone video streams, or offline aerial image data. A small target detection system, installed in a computing platform, is used to perform the method as described in any one of claims 1 to 7; The small target detection system includes an image preprocessing module, a feature reconstruction detection module, a training constraint module, and a detection result output module. The image preprocessing module performs image resizing, pixel normalization, and bounding box coordinate transformation on the input image. The feature reconstruction detection module includes a backbone feature extraction unit, a column-aware feature reconstruction unit, a multi-scale linear attention unit, a multi-scale feature fusion unit, and a detection decoding unit. The training constraint module calculates the total loss based on the predicted category, predicted bounding box, and ground truth bounding box during the training phase and updates the trainable parameters of the feature reconstruction detection module based on the total loss. The detection result output module outputs the target category, confidence level, and predicted bounding box coordinates during the inference phase.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements a method for detecting small targets in unmanned aerial vehicles (UAVs) based on feature reconstruction and distance constraints as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements a method for small UAV target detection based on feature reconstruction and distance constraints as described in any one of claims 1 to 7.

Images