A method for detecting small and medium targets in unmanned aerial vehicle inspection
By improving the YOLO-Ghost network, a dedicated detection network for small targets was constructed. By adopting ghost convolution and separable convolution, and designing a dual-mask method and a positive-negative sample balancing strategy, the redundant computation and adaptability problems of small target detection in UAV inspection were solved, thereby improving detection accuracy and efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- JIANGSU HOPERUN SOFTWARE CO LTD
- Filing Date
- 2026-04-13
- Publication Date
- 2026-07-14
AI Technical Summary
Existing drone inspection methods for small target detection suffer from serious redundant calculations, inaccurate feature extraction, complex processes leading to missed detections, and poor scene adaptability. Current technologies cannot meet the real-time inspection needs of drones.
Based on improvements to the YOLO-Ghost network, a dedicated detection network for small targets is constructed. Ghost convolution and separable convolution are adopted, and a dual-mask method is designed to handle large targets. A positive and negative sample balancing strategy is proposed to simplify the detection process and adapt to the lightweight deployment of embedded chips in UAVs.
It improves the accuracy and efficiency of small target detection, reduces the amount of computation, adapts to ultra-large high-resolution images, simplifies operation, and enhances detection performance and adaptability.
Smart Images
Figure CN122391924A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a detection method, specifically a detection method for small targets in drone inspections, belonging to the field of target detection technology. Specifically, it relates to an efficient detection method for small targets of 5x5~48x48 pixels that is specifically adapted to drone inspection scenarios. It can be widely applied to drone inspection scenarios such as high-altitude vehicle inspection, small defect detection of power lines, and high-altitude small obstacle detection for security. Background Technology
[0002] With the rapid development of drone technology, drone inspection has been widely used in many fields such as power, transportation, and security. During the inspection process, it is often necessary to detect small targets such as high-altitude vehicles, line defects, and small obstacles. The pixel scale of these targets is usually between 5x5 and 48x48 pixels. They are characterized by small size, indistinct features, complex background, high image resolution, and the need for rapid processing, which brings great challenges to the inspection work.
[0003] Currently, small target detection in drone inspections mostly employs general-purpose target detection algorithms. However, existing technologies and general-purpose target detection algorithms (such as the YOLO series and Faster R-CNN) suffer from the following core shortcomings, failing to effectively address the pain points of small target detection in drone inspections:
[0004] Redundant computation is a serious problem: Most algorithms retain multi-scale output layers to adapt to targets of different sizes. When detecting small targets of 5x5 to 48x48 pixels, the medium and large-sized output layers cannot play a role and instead consume a lot of computing resources, resulting in a decrease in detection performance and failing to meet the real-time inspection needs of UAVs.
[0005] Inaccurate feature extraction: Traditional convolutional layers have a large number of parameters and low computational efficiency. Even with the existing improved scheme of GhostConv convolution, the downsampling and upsampling times of the network are not optimized in combination with the characteristics of small target scale, which can easily lead to the loss of small target features and insufficient localization accuracy.
[0006] The process is complex and prone to missing detections: Most algorithms require NMS (non-maximum suppression) to remove duplicate detection boxes, but in small target detection, it is easy to miss detections due to unreasonable threshold settings, which affects the detection accuracy. At the same time, the sliding window detection method requires high-resolution image segmentation, which is cumbersome and prone to duplicate detections or missed detections.
[0007] Poor scene adaptability: Existing technologies are not specifically optimized for drone inspection scenarios. They either cannot adapt to ultra-large, high-resolution images or cannot adapt to the lightweight deployment requirements of drone embedded chips. Furthermore, they do not clearly define the scale range of small targets, resulting in weak detection targeting.
[0008] The YOLO-Ghost network, as a lightweight target detection network, uses GhostConv convolutions to reduce computational redundancy. However, it is essentially still a general-purpose detection network (as shown in prior art 1), and it is not specifically optimized for small targets of 5x5~48x48 pixels in UAV inspections, thus failing to fully meet the accuracy and performance requirements for small target detection. Therefore, there is an urgent need for a dedicated detection method specifically adapted to UAV inspection scenarios and targeting small targets of 5x5~48x48 pixels, to solve the problems of low detection accuracy, insufficient performance, poor adaptability, and redundant computation in existing technologies. Summary of the Invention
[0009] This invention addresses the technical problems existing in the prior art by providing a dedicated detection method for small targets in UAV inspections. It addresses the pain point of detecting small targets of 5x5 to 48x48 pixels in UAV inspections by making targeted improvements based on the YOLO-Ghost network, simplifying the detection process, avoiding redundant calculations, and adapting to the lightweight deployment requirements of UAV embedded chips.
[0010] To achieve the above objectives, the technical solution of this invention is as follows: a dedicated detection method for small targets in UAV inspections. This method is based on improvements to YOLO and GhostNet networks and is specifically designed for detecting small targets with a scale of less than 32x32 pixels. It does not detect medium to large-sized targets. The core improvements revolve around "specialization, efficiency, and adaptability," and specifically include the following steps.
[0011] Step 1: Construct a UAV small target detection dataset.
[0012] The system can collect inspection videos of drones at different altitudes (20 meters, 40 meters, and 80 meters). If flight conditions are limited, it can also collect inspection videos at a fixed altitude (80 meters) with different optical zoom magnifications (1x, 2x, 4x).
[0013] The inspection video is subjected to interval frame extraction to obtain sample images. The frame extraction interval should meet the requirement that the overlap rate of two consecutive sample images along the flight direction is no greater than 0.1. The calculation formula is as follows:
[0014]
[0015] In the formula, Indicates rounding up. Indicates the sensor height. Indicates the drone's flight altitude. Indicates the focal length of the drone camera. For video frame rate, This refers to the flight speed of the drone.
[0016] The extracted frames were labeled using a labeling tool in a rectangular bounding box format to obtain a UAV small target detection dataset. Then, according to... The sets are randomly divided into training, validation, and test sets.
[0017] Step 2: Use the double mask method to process targets with larger sizes (target box length or width greater than 48) in the dataset. The specific process is as follows:
[0018] For any training image, design two masks of the same size as the image. and ,in Fill with 0, Fill with 1;
[0019] Calculate the area of all bounding boxes and sort them in ascending order, then iterate through all bounding boxes.
[0020] If the length and width of the current target bounding box are both no greater than 48, then the area corresponding to the target bounding box... Fill with 0, as shown below
[0021]
[0022] If the length or width of the current target bounding box is greater than 48, create a fill region with a known size to the target bounding box. The default fill is 1, then with The corresponding regions are intersected, as shown below.
[0023]
[0024] Update and As shown below
[0025]
[0026]
[0027] After iterating through all detection boxes and repeating the above operation, finally fill the image with 0s. The corresponding area is shown below.
[0028]
[0029] Step 2 is the first to use a dual-mask design, which solves the problem of selection when targets of different sizes overlap, ensures the consistency of target size in training data, and improves the stability of model training.
[0030] Step 3: Construct a dedicated detection network for small targets.
[0031] The small target-specific detection network (such as) Figure 1The main structure consists of the Backbone, Neck, and STHead (small target parsing layer), which are built from basic operators such as Conv2d convolutional layers, DWconv2d separable convolutional layers, residual connection layers, Upsample layers, Silu activation layers, and BatchNorm normalization layers. The construction process is as follows:
[0032] We propose a separable ghost convolution module, DGBlock, which consists of a convolutional module ConvBlock, a ghost convolutional module GhostConvBlock, a separable convolutional layer DWconv2d, and residual connections. For the input feature layer... After being processed sequentially by GhostConvBlock, DWconv2d, and GhostConvBlock, the result is... , After being processed sequentially by DWconv2d and ConvBlock, the following is obtained: Through residual connection, i.e. + 2. As the output of DGBlock, ConvBlock consists of a Conv2d convolutional layer, a Silu activation layer, and a BatchNorm normalization layer; GhostConvBlock consists of two ConvBlocks and channel concatenation operations.
[0033] The backbone is used for image feature extraction and consists of stacked layers: ConvBlock, GhostConvBlock, DGBlock (output feature layer P2), GhostConvBlock, DGBlock (output feature layer P3), and GhostConvBlock, DGBlock (output feature layer P4), with a downsampling rate of 1 / 16. The size is... Image After passing through Backbone, the dimensions obtained are... Feature layer P2, size is Feature layer P3 and size are Feature layer P4.
[0034] The Neck is used for feature fusion and consists of DGBlock, an upsampling layer (Upsample), and residual connections. For feature layer P4, it first undergoes an upsampling process before being fused with feature layer P3. The fused result undergoes a DGBlock to align the channel count, then another upsampling process before being fused with feature layer P2. The fused result then undergoes another DGBlock, followed by a GhostConvBlock for downsampling, and then another DGBlock, finally yielding a final size of [size missing]. Small-scale target output feature layer .
[0035] STHead is used to parse the category and coordinate information of small targets. It adopts a decoupled head design, enabling both one-to-many and one-to-one detection heads during the training phase (a dual-head design), while only retaining the results of the one-to-one detection head during the inference phase. Figure 1 As shown in the STHead module, it is mainly implemented through basic convolution operators.
[0036] A positive and negative sample balancing strategy based on small-sized targets is proposed. Since the dedicated small-target detection network does not output multi-scale features, a unified scale matching mechanism is employed, the specific process of which is as follows:
[0037] In the one-to-many loss calculation process, a size of [missing information] is constructed. Classification label target matrix A size is Target coordinate frame matrix The default value is 0. The area of all bounding boxes is calculated and sorted in descending order. The bounding boxes are then traversed.
[0038] For the goal Its category information is The coordinate frame information is , , , , , , , The coordinates of the center point, the ordinate of the center point, the width of the coordinate frame, the length of the coordinate frame, the x-coordinate of the top-left corner, the y-coordinate of the top-left corner, the x-coordinate of the bottom-right corner, and the y-coordinate of the bottom-right corner are respectively used to calculate the range of the feature layer position corresponding to the target, as shown below.
[0039]
[0040]
[0041]
[0042]
[0043] Use target Fill in the information as shown below.
[0044]
[0045]
[0046] After processing all targets, the classification label matrix is converted into a one-hot vector. Then, the category prediction head outputs features. The cross-entropy calculation is used to calculate the classification loss of the detected target. As shown below
[0047]
[0048] Output features of the bounding box prediction head Calculate the target bounding box loss based on boundary regression loss CIOU As shown below,
[0049]
[0050] In the one-to-one loss calculation, the main process is the same as that in the one-to-many loss calculation, the difference being the matching of positive and negative samples, which is as follows.
[0051] Construct a size of Classification label target matrix A size is Target coordinate frame matrix The default value is 0. It calculates the area of all bounding boxes and sorts them in ascending order, then iterates through the bounding boxes.
[0052] like For the target have
[0053]
[0054]
[0055] Otherwise, in Find a distance within the range The nearest non-zero position Fill in the labels.
[0056]
[0057]
[0058] After processing all targets, calculate the classification loss and frame loss using a one-to-many loss calculation method.
[0059] Step 3 proposes a small target detection network based on ghost convolution and separable convolution. Compared with the YOLOv8s network, its parameter count is reduced by 92%, and the computational cost is reduced by 68% for the same input size, making it a more efficient detection network.
[0060] Step 3 proposes a positive and negative sample balancing strategy based on small-sized targets. This method consumes less and more GPU memory during training, and the memory usage is constant, thus improving training stability. It solves the problem that the GPU memory usage of detection algorithms such as YOLOv8 increases significantly with the number of targets in a single image during training, making it more suitable for drone-based dense small target detection scenarios.
[0061] Step 4: Use the training and validation sets of the processed UAV small target detection dataset for training and weight updates of the small target-specific detection network. The specific settings are as follows:
[0062] Data augmentation includes random mirroring, random color perturbation, random size scaling (0.25~1.0), and random offset;
[0063] The optimizer selected is MuSGD optimizer, the initial learning rate is set to 0.01, the cosine annealing decay strategy is adopted, the momentum parameter is set to 0.9, the weight penalty coefficient is 0.0001, the batch size is 32, the number of training epochs is set to 300, and the input image size is 1280x1280.
[0064] The training server is equipped with an RTX 3090 graphics card, 64 GB of video memory, and an Intel(R) Core(TM) i7-12700 CPU. The system environment is Ubuntu 20.04.4 LTS, and the training framework is Torch 2.1 + cuda 12.1.
[0065] Step 5: Adaptive size input inference.
[0066] Based on information such as the actual target size, UAV inspection altitude, and camera parameters, the normalized image size is determined and estimated using the following formula.
[0067]
[0068] In the formula, Input image size to the model. The altitude at which the drone flies. For the drone camera focal length, , For the width and height of the drone camera sensor, , The actual width and height of the target are represented by 24, which is an empirical value related to the distribution of small target sizes, and 16 is the maximum downsampling factor of the network.
[0069] After the collected UAV small target detection dataset test set images are normalized in size and mean according to the above calculation results, they are sent into the detection network to obtain the one-to-one detection head result in STHead, which is the target detection result.
[0070] Step 5 incorporates the characteristics of UAV inspection operations. By setting parameters for UAV inspection, appropriate input dimensions are calculated for inference detection, thereby improving detection efficiency and accuracy.
[0071] Compared with the prior art, the advantages of the present invention are as follows:
[0072] 1. By designing and constructing a small target-specific detection network based on ghost convolution and separable convolution, the accuracy and efficiency of small target detection are improved. Compared with existing mainstream algorithms, the detection accuracy (mAP) is improved by more than 4%, and the detection performance is improved by more than 2 times.
[0073] 2. By designing a positive and negative sample balancing strategy based on small-sized targets, we adopt one-to-many and one-to-one multi-head training prediction, and only use one-to-one prediction in the inference stage, which simplifies the detection process, avoids redundant calculations, and adapts to the lightweight deployment requirements of UAV embedded chips.
[0074] 3. Based on the low computational cost of this algorithm, it is particularly suitable for ultra-large, high-resolution images in UAV inspections, requiring no cropping or processing and is easy to operate;
[0075] 4. A formula for calculating image input resolution based on information such as the actual size of the target, the drone inspection altitude, and camera parameters is proposed. This clarifies the specialization of small target detection, avoids training bias, and ensures the practicality and reproducibility of the technical solution. Attached Figure Description
[0076] Figure 1 A network structure diagram for detection of small targets. Detailed Implementation
[0077] To enhance understanding of the present invention, the embodiments will be described in detail below with reference to the accompanying drawings.
[0078] Example 1: See Figure 1 A method for detecting small targets during unmanned aerial vehicle (UAV) inspections, comprising the following steps:
[0079] Step 1: Construct a UAV small target detection dataset.
[0080] The system collects inspection videos from drones at different altitudes (20 meters, 40 meters, and 80 meters). If flight conditions are limited, inspection videos can also be collected at a fixed altitude (80 meters) with different optical zoom magnifications (1x, 2x, 4x). In practice, we used a fixed altitude with different optical zoom magnifications for video acquisition and adaptation. We used a DJI M4 series drone equipped with a Matrice 4T camera system, flying at an altitude of 80 meters and a speed of 10 meters per second, to inspect a certain area three times. We collected three videos at optical zoom magnifications of 1x, 2x, and 4x, with durations of 80 minutes, 40 minutes, and 20 minutes respectively, for a total of 140 minutes.
[0081] The inspection video is subjected to interval frame extraction to obtain sample images. The frame extraction interval should meet the requirement that the overlap rate of two consecutive sample images along the flight direction is no greater than 0.1. The calculation formula is as follows:
[0082]
[0083] In the formula, Indicates rounding up. This indicates the sensor height, with an equivalent height of 24 millimeters. This indicates the drone's flight altitude, which is 80 meters. Indicates the focal length of the drone camera. This represents the video frame rate, with a value of 30. The value represents the drone's flight speed, which is 10 meters per second. The specific results are as follows:
[0084]
[0085] After frame extraction, a total of 1996 samples were obtained. The Labelme tool was used to label the extracted images using rectangular bounding boxes, and two-wheeled vehicles (including bicycles and electric scooters) in the images were also labeled, resulting in a UAV small target detection dataset. The dataset contains 3423 targets, with the majority of targets ranging in size from 10 to 80 pixels, and targets smaller than 32 pixels accounting for 53%.
[0086] Finally according to The proportions were randomly divided into a training set (1198 images), a validation set (399 images), and a test set (399 images).
[0087] Step 2: Use the double mask method to process targets with larger dimensions (target box length or width greater than 48) in the dataset.
[0088] In the aforementioned UAV small target detection dataset, the target size distribution is mainly concentrated between 10 and 76. Considering that operations such as cropping and randomizing sizes occur during training, the target size distribution will become even wider. The dedicated small target detection network lacks multi-scale detection capabilities, and large target supervision signals can lead to training instability. Therefore, it is necessary to perform a masking operation on targets exceeding the specified size. The specific process is as follows:
[0089] For any training image, design two masks of the same size as the image. and ,in Fill with 0, Fill with 1;
[0090] Calculate the area of all bounding boxes and sort them in ascending order, then iterate through all bounding boxes.
[0091] If the length and width of the current target bounding box are both no greater than 48, then the area corresponding to the target bounding box... Fill with 0, as shown below
[0092]
[0093] If the length or width of the current target bounding box is greater than 48, create a fill region with a known size to the target bounding box. The default fill is 1, then with The corresponding regions are intersected, as shown below.
[0094]
[0095] Update and As shown below
[0096]
[0097]
[0098] After iterating through all detection boxes and repeating the above operation, finally fill the image with 0s. The corresponding area is shown below.
[0099]
[0100] Step 3: Construct a dedicated detection network for small targets.
[0101] The small target-specific detection network (such as) Figure 1The main structure consists of the Backbone, Neck, and STHead (small target parsing layer), built from basic operators such as Conv2d convolutional layers, DWconv2d separable convolutional layers, residual connection layers, Upsample layers, Silu activation layers, and BatchNorm normalization layers. The construction process is as follows:
[0102] We propose a separable ghost convolution module, DGBlock, which consists of a convolutional module ConvBlock, a ghost convolutional module GhostConvBlock, a separable convolutional layer DWconv2d, and residual connections. For the input feature layer... After being processed sequentially by GhostConvBlock, DWconv2d, and GhostConvBlock, the result is... , After being processed sequentially by DWconv2d and ConvBlock, the following is obtained: Through residual connection, i.e. + 2. As the output of DGBlock, ConvBlock consists of a Conv2d convolutional layer, a Silu activation layer, and a BatchNorm normalization layer; GhostConvBlock consists of two ConvBlocks and channel concatenation operations.
[0103] The backbone is used for image feature extraction and consists of stacked layers: ConvBlock, GhostConvBlock, DGBlock (output feature layer P2), GhostConvBlock, DGBlock (output feature layer P3), and GhostConvBlock, DGBlock (output feature layer P4), with a downsampling rate of 1 / 16. The size is... Image After passing through Backbone, the dimensions obtained are... Feature layer P2, size is Feature layer P3 and size are Feature layer P4.
[0104] The Neck is used for feature fusion and consists of DGBlock, an upsampling layer (Upsample), and residual connections. For feature layer P4, it first undergoes an upsampling process before being fused with feature layer P3. The fused result undergoes a DGBlock to align the channel count, then another upsampling process before being fused with feature layer P2. The fused result then undergoes another DGBlock, followed by a GhostConvBlock for downsampling, and then another DGBlock, finally yielding a final size of [size missing]. Small-scale target output feature layer .
[0105] STHead is used to parse the category and coordinate information of small targets. It adopts a decoupled head design, enabling both one-to-many and one-to-one detection heads during the training phase (a dual-head design), while only retaining the results of the one-to-one detection head during the inference phase. Figure 1 As shown in the STHead module, it is mainly implemented through basic convolution operators.
[0106] A positive and negative sample balancing strategy based on small-sized targets is proposed. The dedicated small-target detection network does not output multi-scale features, but instead employs a unified scale matching mechanism. This method is simple in rules, stable in training, and significantly saves GPU memory resources used during training. The specific process is as follows:
[0107] In the one-to-many loss calculation process, a size of [missing information] is constructed. Classification label target matrix A size is Target coordinate frame matrix The default value is 0. The area of all bounding boxes is calculated and sorted in descending order. The bounding boxes are then traversed.
[0108] For the goal Its category information is The coordinate frame information is , , , , , , , The coordinates of the center point, the ordinate of the center point, the width of the coordinate frame, the length of the coordinate frame, the x-coordinate of the top-left corner, the y-coordinate of the top-left corner, the x-coordinate of the bottom-right corner, and the y-coordinate of the bottom-right corner are respectively used to calculate the range of the feature layer position corresponding to the target, as shown below.
[0109]
[0110]
[0111]
[0112]
[0113] Use target Fill in the information as shown below.
[0114]
[0115]
[0116] After processing all targets, the classification label matrix is converted into a one-hot vector. Then, the category prediction head outputs features. The cross-entropy calculation is used to calculate the classification loss of the detected target. As shown below
[0117]
[0118] Output features of the bounding box prediction head Calculate the target bounding box loss based on boundary regression loss CIOU As shown below,
[0119]
[0120] In the one-to-one loss calculation, the main process is the same as that in the one-to-many loss calculation, the difference being the matching of positive and negative samples, which is as follows.
[0121] Construct a size of Classification label target matrix A size is Target coordinate frame matrix The default value is 0. It calculates the area of all bounding boxes and sorts them in ascending order, then iterates through the bounding boxes.
[0122] like For the target have
[0123]
[0124]
[0125] Otherwise, in Find a distance within the range The nearest non-zero position Fill in the labels.
[0126]
[0127]
[0128] After processing all targets, calculate the classification loss and frame loss using a one-to-many loss calculation method.
[0129] Step 4: Use the training and validation sets of the processed UAV small target detection dataset for training and weight updates of the small target-specific detection network. The specific settings are as follows:
[0130] Data augmentation includes random mirroring, random color perturbation, random size scaling (0.25~1.0), and random offset;
[0131] The optimizer selected is MuSGD optimizer, the initial learning rate is set to 0.01, the cosine annealing decay strategy is adopted, the momentum parameter is set to 0.9, the weight penalty coefficient is 0.0001, the batch size is 32, the number of training epochs is set to 300, and the input image size is 1280x1280.
[0132] The training server is equipped with an RTX 3090 graphics card, 64 GB of video memory, and an Intel(R) Core(TM) i7-12700 CPU. The system environment is Ubuntu 20.04.4 LTS, and the training framework is Torch 2.1 + cuda 12.1.
[0133] Step 5: Adaptive Size Input Inference
[0134] Based on information such as the actual target size, UAV inspection altitude, and camera parameters, the normalized image size is determined and estimated using the following formula.
[0135]
[0136] In the formula, Input image size to the model. The value is 80 meters, representing the drone's flight altitude. The focal length of the drone camera is 24 mm. , The width and height of the drone camera sensor are 36 mm and 24 mm, respectively. , Here, represents the actual width and height of the target, with reference values of 1 meter and 1 meter respectively. 24 is an empirical value related to the distribution of small target sizes, and 16 is the maximum downsampling factor of the network.
[0137] The normalized dimensions of the test set images in the UAV small target detection dataset are obtained using the above calculation formula, as shown in the table below.
[0138]
[0139] After normalizing the test set images according to the recommended size, they are fed into the detection network to obtain the one-to-one detection head results in STHead, which are the detection target results.
[0140] To verify the effectiveness of this method, we also trained a YOLOv8 model based on the official YOLOv8 implementation for comparison. The experimental results are shown in the table below.
[0141]
[0142] Experimental results show that our model reduces the number of parameters by 92.1% compared to the Yolov8s model, reduces the computational cost by 68.9% for the same input size, improves the overall inference performance by more than 2 times, and improves the detection accuracy by 4%.
[0143] It should be noted that the above embodiments are not intended to limit the scope of protection of the present invention. Equivalent transformations or substitutions made based on the above technical solutions all fall within the scope of protection of the claims of the present invention.
Claims
1. A method for detecting small targets during unmanned aerial vehicle (UAV) inspections, characterized in that, The method includes the following steps: Step 1: Construct a small target detection dataset for drones; Step 2: Use the double mask method to process targets with larger sizes (target box length or width greater than 48) in the dataset; Step 3: Construct a dedicated detection network for small targets; Step 4: Use the training and validation sets of the processed UAV small target detection dataset for training and weight updates of the small target-specific detection network; Step 5: The adaptive size input detection network performs inference.
2. The method for detecting small targets during UAV inspection according to claim 1, characterized in that: In step 1, a small target detection dataset for UAVs is constructed. First, inspection videos of the UAV at different altitudes (20m, 40m, and 80m) are collected. If flight conditions are limited, inspection videos at a fixed altitude of 80m with different optical zoom ratios (1x, 2x, 4x) are collected. Frame extraction is performed on the videos to ensure that the overlap rate along the flight direction between two consecutive sample images is no greater than 0.1, with a frame extraction interval of [missing information]. The calculation method is as follows: In the formula, Indicates rounding up. Indicates the sensor height. Indicates the drone's flight altitude. Indicates the focal length of the drone camera. For video frame rate, This refers to the flight speed of the drone. The extracted frames were labeled using a labeling tool in a rectangular bounding box format to obtain a UAV small target detection dataset. Then, according to... The sets are randomly divided into training, validation, and test sets.
3. The method for detecting small targets during UAV inspection according to claim 1, characterized in that: The process of using a double mask method in step 2 to process targets with larger dimensions (target box length or width greater than 48) in the dataset is as follows: For any training image, design two masks of the same size as the image. and ,in Fill with 0, Fill with 1; Calculate the area of all bounding boxes and sort them in ascending order, then iterate through all bounding boxes. If the length and width of the current target bounding box are both no greater than 48, then the area corresponding to the target bounding box... Fill with 0, as shown below [y1:y2,x1:x2]=0 If the length or width of the current target bounding box is greater than 48, create a fill region with a known size to the target bounding box. The default fill is 1, then with The corresponding regions are intersected, as shown below. Update and As shown below After iterating through all detection boxes and repeating the above operation, finally fill the image with 0s. The corresponding area is shown below. 。 4. The method for detecting small targets during UAV inspection according to claim 1, characterized in that: The process of constructing the dedicated detection network for small targets in step 3 is as follows. The dedicated small target detection network mainly consists of the Backbone, Neck, and STHead (small target parsing layer) built from the basic operators of Conv2d convolutional layer, DWconv2d separable convolutional layer, residual connection layer, Upsample layer, Silu activation layer, and BatchNorm normalization layer. We propose a separable ghost convolution module, DGBlock, which consists of a convolutional module ConvBlock, a ghost convolutional module GhostConvBlock, a separable convolutional layer DWconv2d, and residual connections. For the input feature layer... After being processed sequentially by GhostConvBlock, DWconv2d, and GhostConvBlock, the result is... , After being processed sequentially by DWconv2d and ConvBlock, the following is obtained: Through residual connection, i.e. + 2. As the output of DGBlock, ConvBlock consists of a Conv2d convolutional layer, a Silu activation layer, and a BatchNorm normalization layer; GhostConvBlock consists of two ConvBlocks and channel concatenation operations. The backbone is used for image feature extraction and consists of stacked layers: ConvBlock, GhostConvBlock, DGBlock (output feature layer P2), GhostConvBlock, DGBlock (output feature layer P3), and GhostConvBlock, DGBlock (output feature layer P4), with a downsampling rate of 1 / 16. The size is... Image After passing through Backbone, the dimensions obtained are... Feature layer P2, size is Feature layer P3 and size are Feature layer P4, The Neck is used for feature fusion and consists of DGBlock, an upsampling layer (Upsample), and residual connections. For feature layer P4, it first undergoes an upsampling process before being fused with feature layer P3. The fused result undergoes a DGBlock to align the channel count, then another upsampling process before being fused with feature layer P2. The fused result then undergoes another DGBlock, followed by a GhostConvBlock for downsampling, and then another DGBlock, finally yielding a final size of [size missing]. Small-scale target output feature layer , STHead is used to parse the category and coordinate information of small targets. It adopts a decoupled head design, enabling both one-to-many and one-to-one detection heads during the training phase (a dual-head design), while only the one-to-one detection head results are retained during the inference phase. A positive and negative sample balancing strategy based on small-sized targets is proposed. The dedicated small-target detection network does not output multi-scale features, so a unified scale matching mechanism is adopted. The specific process is as follows. In the one-to-many loss calculation process, a size of [missing information] is constructed. Classification label target matrix A size is Target coordinate frame matrix The default value is 0. The area of all bounding boxes is calculated and sorted in descending order. The bounding boxes are then traversed. For the goal Its category information is The coordinate frame information is , , , , , , , The coordinates of the center point, the ordinate of the center point, the width of the coordinate frame, the length of the coordinate frame, the x-coordinate of the top-left corner, the y-coordinate of the top-left corner, the x-coordinate of the bottom-right corner, and the y-coordinate of the bottom-right corner are respectively used to calculate the range of the feature layer position corresponding to the target, as shown below. Use target Fill in the information as shown below. After processing all targets, the classification label matrix is converted into a one-hot vector. Then, the category prediction head outputs features. The cross-entropy calculation is used to calculate the classification loss of the detected target. As shown below Output features of the bounding box prediction head Calculate the target bounding box loss based on boundary regression loss CIOU As shown below, In the one-to-one loss calculation, the main process is the same as that in the one-to-many loss calculation, the difference being the matching of positive and negative samples, which is as follows. Construct a size of Classification label target matrix A size is Target coordinate frame matrix The default value is 0. It calculates the area of all bounding boxes and sorts them in ascending order, then iterates through the bounding boxes. like For the target have Otherwise, in Find a distance within the range The nearest non-zero position Fill in the labels. After processing all targets, calculate the classification loss and frame loss using a one-to-many loss calculation method.
5. A method for detecting small targets during UAV inspections according to claim 1, characterized in that: The adaptive size input inference step 5 is as follows: Based on information such as the actual target size, UAV inspection altitude, and camera parameters, the normalized image size is determined and estimated using the following formula. In the formula, Input image size to the model. The altitude at which the drone flies. For the drone camera focal length, , For the width and height of the drone camera sensor, , The actual width and height of the target are represented by 24, which is an empirical value related to the distribution of small target sizes, and 16 is the maximum downsampling factor of the network. After the collected UAV small target detection dataset test set images are normalized in size and mean according to the above calculation results, they are sent into the detection network to obtain the one-to-one detection head result in STHead, which is the target detection result.
6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements a special detection method for small targets in UAV inspection as described in any one of claims 1 to 5.
7. A computer-readable storage medium storing computer instructions thereon, characterized in that, When the computer instruction is executed by the processor, it implements a special detection method for small targets in UAV inspection as described in any one of claims 1-5.