Lightweight low-altitude remote sensing image adaptive global feature enhancement small target detection network

Abstract

The invention provides a lightweight low-altitude remote sensing image adaptive global feature enhancement small target detection network, which comprises an input layer, a backbone network, a neck network and a detection head, the backbone network comprises a plurality of Conv modules, a plurality of C2F modules, an SPPF module and a GAM module; the neck network comprises a tiny target detection layer, a plurality of up-sampling modules, a plurality of Concat modules, a plurality of C2F modules and a plurality of Conv modules; the tiny target detection layer comprises an up-sampling module, a C2F module, a Conv module, a Concat module and a dynamic detection head; the neck network does not comprise a large target detection layer; the detection head comprises a plurality of dynamic detection heads, and the dynamic detection heads use a WIoUv3 loss function. According to the method, algorithm design is improved, the feature extraction capability, the multi-scale feature expression capability and the robustness of the model are improved, and the detection efficiency and accuracy are remarkably improved. Intelligent model design and optimization deployment strategies reduce model parameters as much as possible, and hardware cost is reduced while the automation level is improved.

Description

Technical Field

[0001] The present application relates to the technical field of low-altitude remote sensing image detection, and in particular to a lightweight low-altitude remote sensing image adaptive global feature enhancement small target detection network. Background Art

[0002] Low-altitude remote sensing is a key technology for acquiring high-resolution images of the Earth's surface using low-altitude platforms such as drones. With its advantages of high efficiency, low cost, and high degree of automation, it is widely used in a variety of fields, including disaster response, agricultural monitoring, forestry inspections, and border security. Drones, in particular, have become the primary means of low-altitude remote sensing due to their flexible operation and easy deployment.

[0003] However, the recognition and detection of small targets in complex backgrounds still face many challenges, such as image noise, small target scale, and limited resolution, which lead to insufficient detection accuracy and efficiency of traditional small target image detection methods.

[0004] Currently, low-altitude remote sensing target detection technology mainly relies on traditional methods and deep learning-based methods.

[0005] Traditional machine learning algorithms such as HOG (Histogram of Oriented Gradients), SIFT (Scale-Invariant Feature Transform), and LBP (Local Binary Patterns) are combined with classifiers such as SVM and decision trees to achieve object detection. However, these traditional methods require manual feature extraction and are computationally expensive and demanding. These algorithms perform poorly in aerial imagery with diverse object shapes and complex backgrounds. They struggle to adapt to complex scene variations such as lighting, scale, and perspective, resulting in lower detection accuracy than deep learning algorithms.

[0006] Deep learning eliminates the need for manual feature extraction and can autonomously learn complex features from raw data. Object detection algorithms based on deep learning can be broadly categorized into two types: two-stage and one-stage. The first stage of a two-stage algorithm generates candidate object regions. This typically involves passing the input image through a basic network (such as a CNN) to extract features. A region generator is then run on the extracted feature map to generate candidate object bounding boxes. The second stage then refines the localization and object detection of these candidate regions. Common two-stage object detection algorithms include Faster R-CNN and SPP-Net. The widely used Faster R-CNN, due to its two independent stages, typically requires more computing resources and time, resulting in relatively slow inference speed and inability to meet the requirements of real-time detection for drones. One-stage object detection algorithms perform object detection in a single forward pass, eliminating the need to generate candidate regions. They directly predict the object's class and location using a dense grid or anchor boxes. One-stage algorithms typically have faster inference speeds but lower accuracy than two-stage detection algorithms. Common one-stage object detection algorithms include YOLO, SSD, and RetinaNet. Currently commonly used deep learning detection models (such as YOLOv5 and SSD) require high computing resources, making them difficult to run efficiently on resource-constrained platforms such as drones. Furthermore, most existing deep learning algorithms are general-purpose models and lack optimized designs for small target detection in low-altitude remote sensing scenarios.

[0007] Existing object detection algorithms typically require high-performance computing devices to run efficiently, such as workstations equipped with GPUs or high-end embedded platforms. This high hardware requirement significantly increases equipment costs and limits the algorithm's application in resource-constrained scenarios such as drones.

[0008] like Figure 1 As shown in the figure, due to the high altitude of drones in aerial images, small objects occupy a very small proportion of the image pixel space and are densely distributed. As the feature extraction network gradually downsamples, the fine-grained information of small objects is gradually lost, resulting in a significant decline in feature representation capabilities. However, existing algorithms are unable to preserve the detailed information of small objects, thus reducing feature extraction capabilities.

[0009] Due to the drone's overhead perspective, nearby objects appear larger, while distant objects appear smaller, resulting in significant scale variations in the image. This phenomenon makes it difficult for models to adapt to multi-scale object detection. Existing algorithms have limited ability to perceive multi-scale features.

[0010] In practical application scenarios, aerial images are often affected by complex environmental factors, such as lighting changes, weather changes, shadow areas, and occlusion by buildings, etc. These factors can easily lead to the loss of target feature information or interference from noise.

[0011] In order to solve the above problems, scholars at home and abroad have conducted a series of studies.

[0012] Patent publication number "CN118115893A" discloses a small target detection method based on the YOLOv8n network. The introduction of the SPD-Conv module effectively improves the network's lightweight performance and enhances the model's small target detection capabilities by adding a small target detection layer. A global attention mechanism (GAM) is added to the underlying structure of the feature extraction network, allowing the model to focus more on target feature information. Simultaneously, the use of WIoU-v3 as a loss function further improves the model's positioning accuracy. However, this invention has the following deficiencies in practical applications: the SPD-Conv module has limited generalization capabilities and may degrade performance in complex scenarios; the design of the small target detection layer does not provide an adaptive optimization strategy, resulting in an increase in the number of parameters and insufficient flexibility; and the WIoU-v3 loss function is not integrated with the dynamic detection head, resulting in limited performance improvement when processing multi-scale targets and complex spatial characteristics.

[0013] Patent publication number "CN119229261A" discloses a corn pest identification method based on an improved YOLOv8, belonging to the field of corn pest detection technology. The method includes the following steps: first, acquiring a corn pest image; constructing an improved YOLOv8 model consisting of a backbone network, a neck network, and a head network; using the backbone network to extract features from the corn pest image and fuse the feature information using deformable convolution to generate a multi-scale feature map containing visual information at different levels; using the neck network to fuse the multi-scale feature map to extract features containing visual information at different levels; finally, using the head network to predict the corn pest identification result based on the feature map fused with multi-scale information. This method can provide accurate pest detection and identification for corn crops, with high-precision end-to-end detection capabilities, and is suitable for pest control needs in the agricultural sector.

[0014] However, the following shortcomings remain: While deformable convolution offers certain advantages in feature extraction flexibility, its high computational overhead may hinder the demand for real-time detection in actual agricultural scenarios. Multi-scale feature fusion is performed solely through the neck network, lacking support for dynamic weight adjustment or attention mechanisms, which may lead to insufficient extraction and fusion of feature information in complex backgrounds. The head network does not incorporate a dynamic detection head or task-aware mechanisms, which may limit its performance in small-target pest detection and classification recall. Despite the emphasis on high-precision detection, insufficient consideration has been given to lightweight design to reduce the number of model parameters and computational complexity, which may limit its application on resource-constrained devices.

[0015] Patent publication number "CN119152344A" discloses a method, device, and storage medium for abnormal behavior detection based on an improved YOLOv8, pertaining to the fields of computer vision and behavior detection. This method proposes a novel Feature Diffusion Pyramid Network (FDPN) structure, replacing the original neck network architecture of the YOLOv8 model. The advantages of the FDPN lie in its use of a feature focusing module and a feature diffusion mechanism. The feature focusing module captures and integrates cross-scale feature information through parallel depthwise convolution operations, while the feature diffusion mechanism facilitates the flow of contextual information between scales, enabling each feature map to contain more comprehensive semantic information. This design significantly improves the detection accuracy of small objects and enhances the network's adaptability to complex backgrounds and object occlusion. Furthermore, the invention introduces the RCSOSA module and the DyHead dynamic prediction head. The RCSOSA module extracts and enhances diverse features through the RepVGG module, while combining multiple RCS modules to optimize feature selection and channel shuffling, thereby enhancing inter-channel feature fusion. The DyHead dynamic prediction head further improves the recognition ability of occluded people and can achieve higher abnormal behavior detection accuracy and robustness in complex scenes.

[0016] However, the following shortcomings still exist: Although the feature diffusion mechanism enhances multi-scale feature fusion, its parallel computing and cross-scale flow may increase the computational cost of the model, making it less user-friendly for scenarios with high real-time requirements. Although the RepVGG and RCS modules have been introduced, the optimization of these modules mainly focuses on channel shuffling and feature enhancement, and may still have limitations in feature extraction capabilities in areas with dense small objects. The effectiveness of DyHead in abnormal behavior detection mainly depends on the balance of data distribution, and may not perform well in scenarios with sparse data or uneven category distribution. While emphasizing detection accuracy, the model does not fully consider lightweight design, which may limit its application in embedded devices or low-computing power scenarios. Summary of the Invention

[0017] In view of this, the present application provides a lightweight low-altitude remote sensing image adaptive global feature enhancement small target detection network to enhance the model's feature extraction capability, multi-scale feature expression capability and robustness, and significantly improve detection efficiency and accuracy.

[0018] To achieve the above objectives, the technical solutions adopted in this application are as follows: A lightweight low-altitude remote sensing image adaptive global feature enhancement small target detection network (KernelLightweight-YOLO, KL-YOLO), including: input layer, backbone network (Backbone), neck network (Neck) and detection head (Head); the backbone network is responsible for feature extraction, including multiple Conv modules (convolution modules), multiple C2F (CrossStage Partial with 2 convolutions and Feature split, with 2 convolutions and feature split local cross stage) modules, SPPF (Spatial Pyramid Pooling-Fast, spatial pyramid fast pooling) modules and GAM (Global Attention Mechanism, global attention mechanism) module, multiple Conv modules and multiple C2Fs in the backbone network are alternately connected, and then connected in sequence to the GAM module and the SPPF module; the neck network includes a small target detection layer, multiple upsampling modules, multiple Concat modules, multiple C2F modules and multiple Conv modules; the small target detection layer includes an upsampling module, a C2F module, a Conv module, a Concat module and a dynamic detection head (DyHead), and the Concat module is used for feature fusion. The upsampling module, the Concat module and the C2F module in the neck network are connected in sequence, and then connected to the small target detection layer. The output end of the small target detection layer is connected to the Conv module, the Concat module and the C2F module in sequence; the neck network does not include a large target detection layer; the detection head includes several dynamic detection heads (DyHead), and the dynamic detection head uses the WIoUv3 loss function; the dynamic detection head is connected to the C2F module in the neck network.

[0019] Furthermore, advanced data enhancement technology is introduced into the input layer, and the advanced data enhancement technology includes mosaic data enhancement and adaptive anchor box optimization.

[0020] Furthermore, the small target detection layer includes an upsampling module, two Concat modules, two C2F modules, a Conv module and a dynamic detection head (DyHead). The upsampling module, the first Concat module, the first C2F module, the convolution module, the second Concat module and the second C2F module in the small target detection layer are connected in sequence, and the output end of the first C2F module is connected to the input end of the dynamic detection head in the small target detection layer, wherein the first and second are defined according to the order of module connection.

[0021] Furthermore, the derivation of the WIoUv3 loss function is given by formulas (1)-(4): (1) (2) (3) (4) in,( , )and( , ) are the center point coordinates of the predicted box and the real box respectively; and Respectively represent the width and height of the minimum bounding box, and Represent the width and height of the overlapping area between the real box and the predicted box respectively; is the area of ​​the overlapping part, Represents the normalized distance between the center point of the predicted box and the real box is a constant, is the outlier degree, Is a positive number.

[0022] Furthermore, the attention calculation formula of the dynamic detection head is as follows: (5) in, 、 and They represent scale-aware attention function, space-aware attention function, and task-aware attention function respectively; (6) in is a linear function approximated by a 1x1 convolutional layer, is a hard Sigmoid function, is the feature tensor; (7) in, L represents the number of layers of the feature pyramid, K is the number of sparse sampling locations, is the position adjusted by self-learned spatial offset to focus on the discriminative area; It's location The self-learning importance scalar at , both of which are derived from the intermediate layer features It is learned from the input.

[0023] (8) in, is the feature slice of the Cth channel, is a hyperfunction used to learn and control the activation threshold.

[0024] Furthermore, the GAM module includes a channel attention submodule and a spatial attention submodule. Given an input feature map , then the intermediate state of the GAM module and output status The definition is as follows: (9) (10) in, and They are channel attention map and spatial attention map respectively; Represents element-by-element multiplication; Represent the number of channels, height and width of the feature map respectively; The channel attention submodule includes two layers of multi-layer perceptrons (MLPs) and a sigmoid function module; the spatial attention submodule includes two convolutional layers and a sigmoid function module.

[0025] Compared with the prior art, the present invention has the following advantages: 1. This application proposes a lightweight low-altitude remote sensing image adaptive global feature enhancement small target detection network: KL-YOLO. This network replaces CIoU with WIoUv3, which can effectively balance the gradient gain of high-quality and low-quality samples, enhance the model's positioning ability and bounding box regression accuracy, and thus make the model more flexible in various scenarios.

[0026] 2. This application replaces the original detection head used by YOLOv8n with a dynamic detection head (DyHead) combined with self-attention. DyHead unifies scale awareness, spatial awareness, and task awareness in a single framework, allowing the model to automatically adjust to the different characteristics of the input image. Through adaptive feature aggregation, spatial attention, contextual information capture, and dynamic weight adjustment, it effectively solves the problem of pixel overlap and loss of large amounts of spatial detail in the image caused by scale descent during iterative feature extraction, thereby improving the localization accuracy and detection recall of small objects.

[0027] 3. This application introduces a Global Attention Mechanism (GAM) into the backbone network. This mechanism better preserves the correlation between spatial and channel information by reducing information reduction and amplifying global interaction representations. This helps the model better focus on key information and ignore irrelevant information, thereby improving detection accuracy.

[0028] 4. Experiments have shown that the 20×20 large object detection layer has no significant effect on small objects. Therefore, this application removes the large object detection layer while adding a 160×160 small object detection layer to reduce the number of model parameters. The small object detection layer fuses shallow feature maps with deep feature maps, reducing the loss of semantic information during the sampling process and thus improving the model's detection ability for small objects.

[0029] 5. This application proposes a lightweight small target detection network based on deep learning. This algorithm improves the model's ability to recognize small targets through global feature enhancement. This algorithm can be widely used in military strikes, civilian patrols, rescue operations, logistics, and other scenarios. It offers high precision, low latency, and strong robustness, providing strong support for improving the intelligent perception capabilities of drones.

[0030] 6. This application significantly improves detection efficiency and accuracy by improving algorithm design, enhancing the model's feature extraction capabilities, multi-scale feature expression capabilities, and robustness. Intelligent model design and optimized deployment strategies minimize the number of model parameters, improving automation while reducing hardware costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0032] Figure 1 Schematic diagram of challenges in drone aerial images in background technology.

[0033] Figure 2 This is a structural diagram of a lightweight low-altitude remote sensing image adaptive global feature enhancement small target detection network (KL-YOLO) for this application.

[0034] Figure 3 Schematic diagram of the spatial relationship between the real box and the predicted box during the derivation of the WIoUv3 formula in this application.

[0035] Figure 4 This is a schematic diagram of the working principle of DyHead in this application.

[0036] Figure 5 This is a structural diagram of the DyHead of this application.

[0037] Figure 6 This is the overall workflow diagram of GAM for this application.

[0038] Figure 7 This is the workflow diagram of the channel attention submodule.

[0039] Figure 8 Workflow diagram of the spatial attention submodule of this application.

[0040] Figure 9 Figure 2 shows a visualization of the dataset and its sample images.

[0041] Figure 10 (a) is the mAP@0.5 change curve of the KL-YOLO network of this application and other models on the VisDrone2019 dataset.

[0042] Figure 10 (b) is the mAP@0.5 change curve of the KL-YOLO network and other models in this application on the UAVDT dataset.

[0043] Figure 11 These are the detection result diagrams using the YOLOv8n baseline algorithm and the KL-YOLO algorithm proposed in this application.

[0044] Figure 12 This is the visualization result of Grad-CAM.

[0045] Figure 13 This is the flow chart of target detection in UAV low-altitude remote sensing images.

[0046] Figure 14 This is a case diagram of target detection using the KL-YOLO network for this application.

[0047] Figure 15 This application shows a physical picture of the Jetson TX2 NX development board.

[0048] Figure 16 This is a physical picture of the IMX219 sensor.

[0049] Figure 17 This figure shows the results of real-time detection of the KL-YOLO network of this application using the hardware and software configurations in the specific implementation manner. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0051] The purpose of this application is to develop a lightweight, high-precision target detection network for efficient detection and recognition of small targets in low-altitude remote sensing scenes. In particular, it aims to provide a real-time and robust detection solution for complex scenarios such as high-density, strongly occluded, and multi-scale small target detection in drone applications. This application proposes a lightweight, adaptive global feature enhancement network called KL-YOLO. This algorithm combines model accuracy and operational efficiency, and can complete target detection tasks in low-altitude remote sensing images on drone platforms, achieving excellent detection performance with limited embedded device resources.

[0052] like Figure 2 As shown in the figure, a lightweight low-altitude remote sensing image adaptive global feature enhancement small target detection network, KL-YOLO, includes: input layer, backbone network (Backbone), neck network (Neck) and detection head (Head); the backbone network is responsible for feature extraction, including multiple Conv modules (convolution modules), multiple C2F (Cross Stage Partial with 2convolutions and Feature split, with 2 convolutions and feature split local cross stage) modules, SPPF (Spatial Pyramid Pooling-Fast, spatial pyramid fast pooling) modules and GAM (Global AttentionMechanism, global attention mechanism) module, multiple Conv modules and multiple C2Fs in the backbone network are alternately connected, and then connected in sequence to the GAM module and the SPPF module; the neck network includes a small target detection layer, multiple upsampling modules, multiple Concat modules, multiple C2F modules and multiple Conv modules; the small target detection layer includes an upsampling module, a C2F module, a Conv module, a Concat module and a dynamic detection head (DyHead), and the Concat module is used for feature fusion. The upsampling module, the Concat module and the C2F module in the neck network are connected in sequence, and then connected to the small target detection layer. The output end of the small target detection layer is connected to the Conv module, the Concat module and the C2F module in sequence; the neck network does not include a large target detection layer; the detection head includes several dynamic detection heads (DyHead), and the dynamic detection head uses the WIoUv3 loss function; the dynamic detection head is connected to the C2F module in the neck network.

[0053] This application proposes a lightweight low-altitude remote sensing image adaptive global feature enhancement small target detection network: KL-YOLO, such as Figure 2As shown. The backbone network is responsible for feature extraction, including Conv, C2f and SPPF modules. Among them, the C2f module draws on the ELAN module in YOLOv7 and uses the Bottleneck design to replace the C3 module, which improves the nonlinear representation capability and better handles complex image features; the SPPF module is an enhanced version of the SPP module in YOLOv5, which improves speed and efficiency while retaining the advantages of spatial pyramid pooling. The neck network performs multi-scale feature fusion through FPN and PAN, enhancing the model's ability to detect targets at different scales. At the same time, the C2f module improves the efficiency of feature fusion. The detection head is based on the design of YOLOX and adopts a decoupled head structure to process classification and regression tasks separately, thereby improving the detection efficiency and accuracy of different targets.

[0054] Figure 2 The middle blue area represents the modules added to the original model. During the feature extraction phase, feature information of various dimensions can be collected. By adding a global attention mechanism (GAM) to Backbone, this reduces feature loss during sampling, suppresses irrelevant information, and focuses on key features, thereby improving the model's accuracy and robustness. The addition of a small object detection layer to Neck allows for the focus on subtle feature information. By further extracting and integrating features from different scales, small objects stand out more clearly against complex backgrounds, effectively improving the model's detection accuracy for small objects. The red dotted line indicates the replacement of the CIoU in the original model with WIoUv3. By more appropriately weighing factors such as the position, size, and shape of the object bounding box, it can more accurately assess the overlap and match between the detection bounding box and the object, thereby improving the localization of small objects. Furthermore, the detection layer of the original model is replaced with DyHead. The dynamic head structure enables adaptive feature aggregation, allowing the detection layer to more effectively capture object details and edge information. The purple area represents the removal of the large object detection layer from the original model.

[0055] Furthermore, advanced data enhancement technology is introduced into the input layer, and the advanced data enhancement technology includes mosaic data enhancement and adaptive anchor box optimization.

[0056] Furthermore, the small target detection layer includes an upsampling module, two Concat modules, two C2F modules, a Conv module and a dynamic detection head (DyHead). The upsampling module, the first Concat module, the first C2F module, the convolution module, the second Concat module and the second C2F module in the small target detection layer are connected in sequence, and the output end of the first C2F module is connected to the input end of the dynamic detection head in the small target detection layer, wherein the first and second are defined according to the order of module connection.

[0057] The YOLOv8n model includes three detection layers, one for detecting small, medium, and large objects. However, in the VisDrone2019 and UAVDT datasets, which primarily contain small and micro objects, the large object detection layer does not significantly improve the accuracy of small object detection. Instead, it increases the number of model parameters and reduces detection efficiency. Large feature maps lose less spatial information during convolution and pooling, thus better preserving image details. To enhance detection capabilities, this application introduces a Tiny Object Detection Layer, the design of which is shown in Table 1.

[0058] Structural reference of small target detection layer Figure 2 In the blue area of ​​the Neck and Head networks, an upsampling module is added between layers 17-19. Layer 16 outputs an 80×80 feature map, which is upsampled to 160×160. This process increases the resolution of the feature map, enriching the spatial information and details, thereby improving the accuracy and robustness of the model. After feature fusion at layer 18, layer 19 outputs the 160×160 feature map to the small object detection layer (layer 29).

[0059] A downsampling module is added between layers 20 and 22. Layer 20 reduces the resolution of the feature map by half, outputting an 80×80 feature map. Layer 21 then performs feature fusion, and layer 22 then outputs the feature map to the small object detection layer (layer 30) through a convolution operation. Similarly, layer 25 outputs a 40×40 feature map to the medium object detection layer (layer 31).

[0060] Table 1 Network parameters of small target detection layer Furthermore, the derivation of the WIoUv3 loss function is given by formulas (1)-(4): (1) (2) (3) (4) Among them, such as Figure 3 As shown, ( , )and( , ) are the center point coordinates of the predicted box and the real box respectively; and Respectively represent the width and height of the minimum bounding box, and Represent the width and height of the overlapping area between the real box and the predicted box respectively; is the area of ​​the overlapping part, Represents the normalized distance between the center point of the predicted box and the real box is a constant, is the outlier degree, Is a positive number.

[0061] In small object detection, using CIoU can hinder network optimization. To address this issue, this paper improves CIoU and proposes a more effective version of IoU, WIoUv3. Most datasets contain low-quality samples, which can negatively impact the generalization ability of object detection models. WIoU is a dynamic, non-monotonic focusing mechanism (FM). By replacing CIoU with WIoUv3 and reducing the competitiveness of high-quality anchor boxes through a gradient gain allocation strategy, we minimize the impact of low-quality samples on the model's generalization ability and further improve model performance.

[0062] Furthermore, the attention calculation formula of the dynamic detection head is as follows: (5) in, 、 and They represent scale-aware attention function, space-aware attention function, and task-aware attention function respectively; (6) in is a linear function approximated by a 1x1 convolutional layer, is a hard Sigmoid function, is the feature tensor; (7) in, L represents the number of layers of the feature pyramid, K is the number of sparse sampling locations, is the position adjusted by self-learned spatial offset to focus on the discriminative area; It's location The self-learning importance scalar at , both of which are derived from the intermediate layer features It is learned from the input.

[0063] (8) in, is the feature slice of the Cth channel, is a hyperfunction used to learn and control the activation threshold.

[0064] In the YOLOv8n model, the output of the backbone network is distributed in three dimensions: spatial dimension, horizontal dimension, and channel dimension. In order to further improve the accuracy of small target detection, this paper replaces the original detection head with DY-Head. DY-Head integrates task perception, spatial perception, and scale perception into one detection head, which helps to better handle targets of different sizes and enhance the robustness of the model in detection. The working principle of DY-Head is as follows Figure 4 shown.

[0065] In DyHead, the attention function is transformed into three sequential attention mechanisms, each focusing on a specific dimension. The dimension is reduced by performing global average pooling on the L×S dimension, followed by two fully connected layers and a normalization layer. Finally, the output is normalized to [-1, 1] by a translated Sigmoid function. The structure of DyHead is as follows Figure 5 shown.

[0066] Furthermore, the GAM module includes a channel attention submodule and a spatial attention submodule. Given an input feature map , then the intermediate state of the GAM module and output status The definition is as follows: (9) (10) in, and They are channel attention map and spatial attention map respectively; Represents element-by-element multiplication; Represent the number of channels, height and width of the feature map respectively; The channel attention submodule includes two layers of multi-layer perceptrons (MLPs) and a sigmoid function module; the spatial attention submodule includes two convolutional layers and a sigmoid function module.

[0067] In the YOLOv8n model, the feature extraction module primarily extracts local features from the input image. However, in the VisDrone2019 dataset, detection becomes significantly more challenging due to the fact that almost all of the objects are small and micro. To improve the model's detection accuracy, this application introduces a global attention mechanism (GAM) into the backbone network. Based on a recurrent neural network model, GAM adaptively selects and processes "informative" node sequences in the graph. By introducing the attention mechanism, the model can dynamically focus on local features rather than the entire image, thereby enhancing its ability to identify key patterns. The model primarily consists of the following components: a structural attention layer, a stride layer, and a combination layer, which work together to emphasize important areas in the image. Furthermore, GAM employs an efficient training strategy and hyperparameter tuning to ensure optimal performance within a limited timeframe. By capturing important information across three dimensions, GAM significantly improves the detection of small objects.

[0068] GAM is an improvement on CBAM, inheriting the sequential channel-spatial attention mechanism of CBAM and redesigning the submodules. Its overall process is as follows Figure 6 is shown and defined in Equations (9) and (10).

[0069] The channel attention submodule retains information from three dimensions through three-dimensional arrangement, and then adopts a two-layer multi-layer perceptron (MLP) structure to expand the dependencies in the channel space. Figure 7 shown.

[0070] In the spatial attention submodule, two convolutional layers are used to fuse spatial information. In order to further retain local feature information, the structure chooses to cancel the pooling operation because the maximum pooling may cause the loss of some spatial information. Although this increases the number of parameters, it helps to obtain more complete spatial information and reduce the risk of ignoring certain feature maps. Figure 8 shown.

[0071] Experimental setup and result analysis: This section first introduces the dataset used in the experiment, then presents the experimental environment and parameter settings. It then compares the detection performance of KL-YOLO with other advanced object detection methods, demonstrating the overall performance advantage of the KL-YOLO algorithm. Finally, an ablation experiment verifies the effectiveness of the improved strategy proposed in this paper.

[0072] Dataset Introduction: This experiment uses the VisDrone2019 and UAVDT datasets, two mainstream benchmark datasets for drone aerial imagery. Experiments based on these two authoritative datasets fully validate the effectiveness and robustness of the proposed network.

[0073] The VisDrone2019 dataset, collected by the AISKYEYE team at the Machine Learning and Data Mining Laboratory of Tianjin University, consists of 10,209 static images, 6,471 of which are used for training, 548 for validation, and 1,610 for testing. The dataset has been manually annotated and contains over 34,600 object instances. The images, captured by various drone cameras, cover scenes from 14 different cities in China, both during the day and at night, and encompass 10 categories: ignored areas, pedestrians, people, bicycles, cars, vans, trucks, tricycles, sunshade tricycles, and buses.

[0074] Most of the targets are small, the scenes are crowded, and there is severe occlusion between targets. The large proportion of night scenes further increases the difficulty of detection. Figure 9 Part (a) shows the number of samples and feature distribution for each category, indicating that the "car" category has the largest number of samples. The figure also shows the shape, number, and distribution of bounding boxes. Most bounding boxes are small and concentrated in the center of the image. The aspect ratios of the bounding boxes are evenly distributed, indicating that most small objects in the dataset have been detected. Part (b) of Figure 9 shows some sample images from the dataset.

[0075] The UAVDT dataset, proposed by a research team from Nanjing University of Science and Technology in 2018, is specifically designed for drone imagery. The dataset contains images and videos captured vertically and obliquely, covering a variety of weather conditions (e.g., sunny, cloudy, and nighttime), totaling 100 videos and over 80,000 frames. Annotated categories include cars, trucks, buses, pedestrians, and others. The scenes primarily consist of urban roads, parking lots, and highways, with densely packed objects and severe occlusion, posing significant challenges. Due to the large size of the UAVDT dataset, which is not conducive to algorithm comparison experiments, this study randomly selected 2,000 images and divided them into training, validation, and test sets in an 8:1:1 ratio. Figure 9 Part (c) shows the distribution of object classes in the training data. The selected classes primarily include cars, buses, and trucks, with cars being the most numerous. The majority of detected objects are small. Part (d) of Figure 9 shows some sample images from the UAVDT dataset.

[0076] Experimental environment and parameter settings: All models were trained and tested on the same workstation (64-bit Windows 10 operating system, Intel(R) Core(TM) i9-14900K processor, NVIDIA GeForce RTX 4080 graphics card), using CUDA 11.8, PyTorch 2.1.0 + cu118, CUDNN 8.6, torchvision 0.16.0 + cu118, and Python 3.10. The experimental hyperparameter settings are shown in Table 2.

[0077] Table 2 Experimental environment and hyperparameters Evaluation indicators: The evaluation metrics used in this application's KL-YOLO object detection network include precision (P), recall (R), average precision (AP), mean average precision (mAP), confusion matrix (Confusion Matrix), and the number of model parameters. P and R are key metrics for evaluating model performance. mAP combines precision and recall, calculating the average precision across different object categories to assess overall model performance. The confusion matrix includes true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN). Furthermore, the number of model parameters and processing speed (FPS) are important metrics for evaluating model performance, storage requirements, and computational cost.

[0078] The formulas for precision and recall are shown in formulas (11) and (12) respectively: Where P represents precision, which is the proportion of instances predicted as positive by the model that are actually positive; R represents recall, which is the proportion of instances correctly predicted as positive by the model among all instances that are actually positive. When the IoU threshold is set to 0.5, the recall and precision values ​​obtained can be used to plot the precision-recall (PR) curve. The area under the curve represents the mean average precision (AP). Mean average precision (mAP) is defined as the average of all AP values ​​for all categories by calculating the mean precision for each category and taking the mean of these values. Indicates the number of positive samples predicted correctly; Indicates the number of positive samples with incorrect predictions; Indicates the number of negative samples predicted incorrectly; N indicates the total number of categories; AP i represents the average accuracy of the i-th category; FPS is the number of frames processed per second by the model, S is the total number of frames processed in each time period, and t represents the length of the time period.

[0079] Comparative experiments between KL-YOLO and mainstream detection models: To validate the advanced features and robustness of the KL-YOLO network, we conducted comprehensive comparative experiments comparing KL-YOLO with the mainstream YOLO model. These experiments were conducted on the same datasets (VisDrone2019 and UAVDT) and maintained consistent settings, including the number of training epochs and learning rate, to ensure rigorous evaluation. The evaluation metrics used for the comparison included parameters, mAP@0.5, precision, recall, and GFLOPs.

[0080] The experimental results, shown in Table 3, demonstrate that KL-YOLO outperforms other models in the mAP@0.5 metric on both datasets, demonstrating its superior performance in small object detection. Compared to other YOLO versions, KL-YOLO achieves higher mAP values ​​on both datasets while significantly reducing the number of parameters, resulting in a smaller model size and faster performance. This makes it suitable for small object detection in resource-constrained drone environments, providing technical support for the widespread application of drones in security monitoring, traffic management, disaster monitoring, and other fields.

[0081] Table 3 Performance comparison of mainstream detection models on VisDrone and UAVDT test datasets KL-YOLO was also compared with the two-stage Faster-RCNN algorithm, the anchor-free CenterNet algorithm, and the single-stage SSD algorithm. The experimental results are shown in Table 4, showing improvements in mAP by 17.16%, 16.83%, and 18.54%, respectively. These experiments demonstrate that KL-YOLO outperforms existing object detection algorithms and features a smaller model size, making it more suitable for deployment on resource-constrained drones.

[0082] Table 4 Comparative experiments with other models By comparing the mAP@0.5 curves of different models, we can visually demonstrate their differences in detection accuracy, convergence speed, and stability, particularly in small object detection. This curve comparison also assesses the adaptability of each model to different scene complexities and verifies the effectiveness of the optimization method. Figure 10 (a) shows the mAP@0.5 curve on the VisDrone2019 dataset. The proposed KL-YOLO model consistently outperforms other models after 75 iterations, demonstrating its superior accuracy and performance in high-density small object detection scenarios. Furthermore, KL-YOLO's faster convergence speed demonstrates its strong adaptability to this dataset and suitability for real-time applications. Furthermore, KL-YOLO's stable mAP@0.5 curve reflects its robustness and stability during training, avoiding overfitting and underfitting issues. Figure 10 (b) shows the mAP@0.5 curve on the UAVDT dataset. KL-YOLO maintains first place, demonstrating significant accuracy improvements, particularly when detecting small objects such as vehicles and pedestrians. KL-YOLO maintains high adaptability and stability in complex scenes and various weather conditions. Even in the presence of occlusion and illumination changes, KL-YOLO maintains excellent performance. Furthermore, KL-YOLO converges quickly in the early stages of training and stabilizes in the later stages, further validating the effectiveness and robustness of its design and optimization on the UAVDT dataset.

[0083] In order to verify the superiority of the KL-YOLO algorithm, a photo taken during the day and a photo taken at night were randomly selected from the VisDrone2019 dataset and the UAVDT dataset, and the YOLOv8n baseline algorithm and the KL-YOLO algorithm proposed in this application were used for detection respectively. The experimental environment remained consistent. The detection results are shown in the figure below. Figure 11The red box in the figure shows the difference in the detection results between the two models. As can be seen, the baseline algorithm suffers from significant missed detections and false detections, while the proposed KL-YOLO algorithm significantly improves these issues, demonstrating strong robustness and high detection accuracy. Therefore, it is concluded that the KL-YOLO model can improve the performance of small target detection on drones, validating the superiority of the improved algorithm.

[0084] Ablation experiment analysis: In order to verify the effectiveness of each improved strategy in KL-YOLO, this section conducts an ablation experiment analysis. The specific experimental results are shown in Table 5.

[0085] Table 5 Ablation experiment Ablation experiments were conducted on the VisDrone2019 dataset to verify the effectiveness of the improvements. These enhancement modules were sequentially added to the original YOLOv8n model, with added modules marked with "√" and removed modules marked with "-". The experimental environment configuration remained consistent, as shown in Table 5, where "TODL" indicates the addition of a small object detection layer and the removal of a large object detection layer.

[0086] The results in Table 5 show that each improvement strategy effectively improves the model's detection performance. Experiments show that compared to Experiment 1, Experiment 2 achieves a 2.6% increase in mAP, a 4.0% increase in Precision, a 1.7% increase in Recall, and a 31.2% reduction in parameters. This demonstrates that adding a small object detection layer and removing a large object detection layer effectively reduces irrelevant semantic information in the model, thereby increasing speed, reducing parameters, and improving detection accuracy. Experiment 3 further replaces Detect with DYHead, achieving 4.8% increases in mAP, 4.6% increases in Precision, and 3.7% increases in Recall, indicating that DYHead helps the network adapt to objects of varying scales and shapes, improving accuracy. Experiment 4 replaces CIoU with WIoUv3, achieving 0.1% increases in mAP and 0.2% increases in Precision, demonstrating that WIoUv3's dynamic non-monotonic evaluation mechanism improves box quality and localization capabilities.

[0087] Experiment 5 is the improved model proposed in this application, which adds GAMattention to Experiment 4. GAMattention calculates the similarity of each position and uses the similarity as a weight to create a global attention vector. This mechanism focuses attention on more important areas in the image, enhancing the accuracy of small object detection. Compared with the original YOLOv8n, KL-YOLO improves mAP by 8.0%, Precision by 9.7%, and Recall by 5.3%, with only a 5.2% increase in parameters. Therefore, KL-YOLO significantly improves the accuracy of small object detection and solves the problems of missed detection and false detection in drone missions.

[0088] Grad-CAM (Gradient Weighted Class Activation Mapping) is a technique for visualizing the importance of class-specific features in deep learning models. In deep learning, feature maps represent the activation patterns of neurons in different convolutional or other layers. Grad-CAM uses the gradients backpropagated from the final convolutional layer to weight the importance of each spatial location in the feature map, generating a heatmap related to the activation strength of the target class. This helps understand the areas where the model focuses when making predictions and reveals the rationale and key features behind the model's decisions.

[0089] The visualization results of Grad-CAM are as follows Figure 12 As shown in the figure, both models generate red heatmaps for nearby vehicles, indicating that both models focus on these areas. However, for distant vehicles and pedestrians, YOLOv8n's heatmaps are colder, indicating that YOLOv8n has difficulty processing small object areas. In contrast, the modified KL-YOLO heatmaps are warmer, indicating that KL-YOLO pays more attention to small object areas. When multiple small objects are close together, KL-YOLO's boundaries and predictions are more accurate, indicating that the model is more robust in small object detection. This shows that these modifications improve the performance of the original YOLO model in small object detection.

[0090] Implementation method: See also Figure 13 , the actual detection process of the KL-YOLO network model in this application is as follows: Step 1: Data Collection This step mainly involves collecting small targets in low-altitude remote sensing images. The data collection process includes carefully designed flight routes and the acquisition and annotation of low-altitude remote sensing images. In this implementation plan, the collection work mainly relies on efficient drone platforms and high-resolution sensors. This article uses a DJI drone, which has a long cruising time and a strong load capacity to ensure the efficiency and reliability of image acquisition. In addition, it is equipped with a high-precision camera module for multi-angle and multi-dimensional remote sensing image acquisition. These sensors include visible light, infrared imaging and laser ranging functions, which can fully capture target images and provide reliable data support for subsequent detection algorithms.

[0091] Step 2: Use lightweight low-altitude remote sensing image small target detection adaptive global feature enhancement network (KL-YOLO) After image acquisition, the acquired photovoltaic array image will be input into the KL-YOLO network of this application. KL-YOLO combines a variety of advanced technologies to enhance the model's ability to detect small targets. The network fully exploits the detailed features in the image through adaptive global feature enhancement, especially optimizing the high-precision detection of small targets in low-altitude remote sensing images. The KL-YOLO network outputs the position, category, and confidence score of each detection frame, which can effectively distinguish different types of small targets and accurately locate them. The detection results are as follows: Figure 14 shown

[0092] Through this implementation, ground targets can be accurately and efficiently detected in low-altitude remote sensing images, and this method has higher accuracy and stronger robustness when processing small targets than traditional models.

[0093] In specific implementation, this application solution can be implemented on the Jetson TX2 NX development board, such as Figure 15 shown.

[0094] Hardware requirements: Jetson TX2 NX development board, SD card (≥32GB) or NVMe SSD, power supply, monitor, mouse, and keyboard; Software preparation: Install JetPack SDK 4.6.1 (including TensorRT, CUDA, cuDNN), Ubuntu 18.04 / 20.04 (JetPack 4.x / 5.x), Python 3.6, PyTorch 1.10, OpenCV, NVIDIA DeepStream SDK, and pycuda installed; Model conversion and optimization: After training KL-YOLO on the PC, use PyTorch to export the model to ONNX format; transfer the ONNX model to the TX2 NX; install onnx2trt on the TX2 NX and perform the conversion; Optimize parameters: Use FP16 precision to improve inference speed, optimize single image inference, and allocate 2GB of video memory to TensorRT; Performance optimization: Jetson Power Mode tuning; adjust the power consumption mode of TX2 NX to improve inference speed.

[0095] Running inference on Jetson TX2 NX: 1. Test single image detection; 2. Test camera real-time detection.

[0096] This specific implementation uses the IMX219 sensor, which provides 8-megapixel image quality and a 79.3-degree field of view. Figure 16 The test results are shown as follows. Figure 17 shown.

[0097] This specific implementation method of KL-YOLO on Jetson TX2 NX greatly improves the inference speed through technologies such as TensorRT, CUDA, and DeepStream, and can achieve a target detection speed of 30fps. Compared with other popular target detection models, this application outperforms mainstream target detection algorithms in both detection speed and accuracy, meeting the needs of high real-time scenarios such as embedded intelligent detection, autonomous driving, and drone monitoring.

[0098] In summary, in response to the technical difficulties of small target detection, this application designed a feature extraction layer dedicated to small target detection on the basis of YOLOv8, which enhanced the model's ability to capture small targets; by introducing the dynamic attention mechanism (DYHead) and the global attention mechanism (GAM), efficient integration of global information was achieved, and the detection accuracy of the model in multi-target dense scenes was improved; a lightweight design was completed, including removing the large target detection layer and introducing the efficient WIoUv3 loss function, which reduced the redundant parameters of the model and optimized the inference speed; the network structure was improved, and combined with the characteristics of the embedded platform, the efficient deployment of the algorithm was achieved, meeting the requirements of real-time and deployment convenience in drone aerial photography, and providing an efficient and economical solution for drone low-altitude remote sensing tasks.

[0099] The above are only specific embodiments of the present application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A lightweight low-altitude remote sensing image adaptive global feature enhancement small target detection network, characterized by: include: Input layer, backbone network, neck network and detection head; the backbone network is responsible for feature extraction, including multiple Conv modules, multiple C2F modules, SPPF modules and GAM modules. The multiple Conv modules and multiple C2Fs in the backbone network are alternately connected, and then connected to the GAM module and SPPF module in sequence; the neck network includes a small target detection layer, multiple upsampling modules, multiple Concat modules, multiple C2F modules and multiple Conv modules; the small target detection layer includes an upsampling module, a C2F module, a Conv module, a Concat module and a dynamic detection head. The Concat module is used for feature fusion. The upsampling module, Concat module and C2F module in the neck network are connected in sequence and then connected to the small target detection layer. The output end of the small target detection layer is connected to the Conv module, Concat module and C2F module in sequence; the neck network does not include a large target detection layer; the detection head includes several dynamic detection heads, and the dynamic detection head uses the WIoUv3 loss function; the dynamic detection head is connected to the C2F module in the neck network.

2. A lightweight low-altitude remote sensing image adaptive global feature enhancement small target detection network according to claim 1, characterized in that: Advanced data enhancement technology is introduced into the input layer, and the advanced data enhancement technology includes mosaic data enhancement and adaptive anchor box optimization.

3. A lightweight low-altitude remote sensing image adaptive global feature enhancement small target detection network according to claim 1 or 2, characterized in that: The small target detection layer includes an upsampling module, two Concat modules, two C2F modules, a Conv module and a dynamic detection head. The upsampling module, the first Concat module, the first C2F module, the convolution module, the second Concat module and the second C2F module in the small target detection layer are connected in sequence. The output end of the first C2F module is connected to the input end of the dynamic detection head in the small target detection layer, wherein the first and second are defined according to the order of module connection.

4. A lightweight low-altitude remote sensing image adaptive global feature enhancement small target detection network as described in claim 3, characterized in that: The derivation of the WIoUv3 loss function is given by formulas (1)-(4): (1) (2) (3) (4) in,( , )and( , ) are the center point coordinates of the predicted box and the real box respectively; and Respectively represent the width and height of the minimum bounding box, and Represent the width and height of the overlapping area between the real box and the predicted box respectively; is the area of ​​the overlapping part, Represents the normalized distance between the center point of the predicted box and the real box is a constant, is the outlier degree, Is a positive number.

5. A lightweight low-altitude remote sensing image adaptive global feature enhancement small target detection network according to claim 4, characterized in that: The attention calculation formula of the dynamic detection head is as follows: （5） in, 、 and They represent scale-aware attention function, space-aware attention function, and task-aware attention function respectively; （6） in is a linear function approximated by a 1x1 convolutional layer, is a hard Sigmoid function, is the feature tensor; （7） in, L represents the number of layers of the feature pyramid, K is the number of sparse sampling locations, is the position adjusted by self-learned spatial offset to focus on the discriminative area; It's location The self-learning importance scalar at , both of which are derived from the intermediate layer features learned from the input; （8） in, is the feature slice of the Cth channel, is a hyperfunction used to learn and control the activation threshold.

6. A lightweight low-altitude remote sensing image adaptive global feature enhancement small target detection network according to claim 5, characterized in that: The GAM module includes a channel attention submodule and a spatial attention submodule. Given an input feature map , then the intermediate state of the GAM module and output status The definition is as follows: （9） （10） in, and They are channel attention map and spatial attention map respectively; Represents element-by-element multiplication; Represent the number of channels, height and width of the feature map respectively; The channel attention submodule includes two layers of multi-layer perceptrons (MLPs) and a sigmoid function module; the spatial attention submodule includes two convolutional layers and a sigmoid function module.

Images