UAV small target detection method, deep learning network model, system and readable storage medium

Abstract

The present application relates to the field of deep learning technology, and in particular to a method for detecting small targets in unmanned aerial vehicles, a deep learning network model, a system, and a readable storage medium. A more lightweight deep spatial feature fusion module is used to replace the deep residual block to reduce channel interaction, thereby avoiding the redundancy and inefficiency of computing resources caused by a large number of stacked residual blocks in deep feature extraction. In the feature alignment task, a lightweight upsampling operator DySample is introduced, which enables the present application to further save computing resources without sacrificing detection accuracy. This invention aims to solve the problem of how to reduce the computational overhead of small target detection methods and improve the ability to extract semantic information.

Description

Technical Field

[0001] The present application relates to the field of deep learning technology, and in particular to a method for detecting small targets in unmanned aerial vehicles, a deep learning network model, a system, and a readable storage medium. Background Art

[0002] Detecting small objects (also known as lightweight objects) from drones is a technical challenge that needs to be overcome. A common approach currently uses a feature pyramid network (FPN) to detect small objects in drone images. FPN fuses feature maps of different scales through horizontal concatenation and vertical addition, combining shallow features with deep features to enhance their semantic information and improve small object detection performance.

[0003] However, in the process of conceiving and implementing this application, the applicant discovered that although deep semantic information is effective for small target detection, the effect of directly detecting small targets based on deep features is not good - because traditional FPN has a large number of stacked residual blocks in deep feature extraction, which easily causes redundancy and inefficiency of computing resources.

[0004] Based on this, this application proposes a small target detection method for drones using a new FPN architecture. Summary of the Invention

[0005] The main purpose of this application is to provide a small target detection method for drones, aiming to solve the problem of how to reduce the computational overhead of the small target detection method and improve the semantic information extraction capability.

[0006] To achieve the above objectives, the present application provides a method for detecting small targets on drones, the method comprising:

[0007] Obtain an image dataset containing multiple small targets collected by a drone;

[0008] Inputting the image dataset into the trained deep learning network for feature extraction;

[0009] The deep learning network uses multiple deep spatial feature fusion modules to fuse semantic features of different scales, and aligns the fused feature map to the same resolution as the second convolutional layer through a lightweight upsampling operator DySample. The aligned feature map is input into the feature extraction layer and the residual block is used to fuse it with the shallow features to extract the detail features.

[0010] Obtain the predicted category and predicted box coordinates corresponding to each image output by the deep learning network;

[0011] Target prediction frames without overlap are screened out according to the prediction frame coordinates, so as to mark target positions and target categories corresponding to the respective small targets on the image based on the target prediction frames.

[0012] Optionally, the deep spatial feature fusion module includes:

[0013]

[0014]

[0015] Where, Represents the output of semantic features after deep hole convolution processing, is the output after the previous layer of hole convolution processing, is the learnable parameter of the deep dilated convolution, represents the depth-wise dilated convolution operation, is the expansion rate, is the final output feature map, represents point-wise convolution, Indicates channel splicing.

[0016] Optionally, the depthwise dilated convolution operation is performed by a standard convolution layer, a depthwise separable convolution layer, and a dilated convolution layer, wherein:

[0017] The convolution kernel of the standard convolution is 3;

[0018] The convolution kernel of the depthwise separable convolution is 3, the number of group convolutions is the number of hidden channels, and the number of input channels is 4;

[0019] The convolution kernel of the dilated convolution is 3, and the dilation rates are 1, 2, and 3 respectively.

[0020] Optionally, the fusion strategy of the deep learning network is a two-headed asymptotic fusion strategy, and the two-headed asymptotic fusion strategy specifically includes:

[0021] In the initial stage of integration, Indicates being in layer, and below it The feature maps of the first stage (with BackBone as the starting stage) are fused in parallel with the output from BackBone:

[0022]

[0023]

[0024] in, Corresponding to BackBone output , represents the strided convolution downsampling operation, represents the upsampling operation, and Indicates that it is in the middle stage of the downsampling phase;

[0025] In the later stage of fusion, the semantics of the generated intermediate layers are gradually integrated:

[0026]

[0027]

[0028] in, Integration in the middle layer ,and Then further integrate the output from the deepest layer of BackBone with the output of the previous stage of P3 layer ,DSF is the deep spatial feature fusion module;

[0029] In the dual-headed progressive fusion strategy, the shallow layer gradually acquires the semantics of the deep layer. ,when Sometimes:

[0030]

[0031] when Sometimes:

[0032]

[0033] Where, Indicates splicing by channel dimension, Indicates that the feature map is adjusted in resolution by upsampling.

[0034] Optionally, the step of aligning the fused feature map to the same resolution as the second convolutional layer by using a lightweight upsampling operator DySample specifically includes:

[0035] Generate a dynamic sampling offset for each pixel using a sampling point generator, and then add the original grid position to the dynamic sampling offset to obtain a sampling set;

[0036] Resampling the input fused feature map through a grid sampling function to obtain the upsampled feature;

[0037] Among them, the generation strategy of the sampling point generator includes a static range factor strategy and a dynamic range factor strategy. The static range factor strategy includes: generating a dynamic sampling offset through a linear layer; the dynamic range factor strategy includes: first generating a range factor through content perception, and then generating the dynamic sampling offset by passing the range factor through a linear layer and pixel transformation.

[0038] Optionally, the hyperparameters in the training process of the deep learning network are set as:

[0039] The number of iterations is set to 300, the learning rate is set to 0.01, and the number of images input at a time is set to 8.

[0040] Optionally, filtering out non-overlapping target prediction boxes according to the prediction box coordinates includes:

[0041] The non-maximum suppression method is used to screen out overlapping prediction boxes, and the remaining prediction boxes are used as the target prediction boxes.

[0042] In addition, to achieve the above objectives, the present application also provides a deep learning network model, which includes:

[0043] YOLOv8 architecture;

[0044] Multiple deep spatial feature fusion modules are used to fuse semantic features of different scales in image datasets containing multiple small objects collected by drones;

[0045] The dual-headed progressive fusion module is used to align the fused feature map to the same resolution as the second convolutional layer through the lightweight upsampling operator DySample, and input the aligned feature map into the feature extraction layer to extract detailed features after fusing it with the shallow features using the residual block.

[0046] In addition, to achieve the above-mentioned purpose, the present application also provides a drone small target detection system, which includes: a memory, a processor, and a computer program stored on the memory and runnable on the processor. When the computer program is executed by the processor, the steps of the drone small target detection method as described in any one of the above items are implemented.

[0047] In addition, to achieve the above-mentioned purpose, the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the drone small target detection method as described in any one of the above items are implemented.

[0048] This application has at least the following beneficial effects:

[0049] 1. Use a lighter-weight deep spatial feature fusion module to replace the deep residual block to reduce channel interaction, thereby avoiding the redundancy and inefficiency of computing resources caused by a large number of stacked residual blocks in deep feature extraction;

[0050] 2. In the feature alignment task, the lightweight upsampling operator DySample is introduced, which enables this application to further save computing resources without sacrificing detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a flow chart of the first embodiment of the UAV small target detection method of the present application;

[0052] Figure 2 Schematic diagram of the construction process of the deep spatial feature fusion module involved in the embodiment of the present application;

[0053] Figure 3 The feature extraction layer involved in the embodiment of the present application uses a residual block to fuse it with shallow features to extract detail features;

[0054] Figure 4 Schematic diagram of a multi-scale L-shaped pyramid involved in an embodiment of the present application;

[0055] Figure 5 This is a diagram of the deep learning network structure of an embodiment of the present application;

[0056] Figure 6 This is a schematic diagram of the architecture of the hardware operating environment of the drone small target detection system involved in the embodiments of the present application.

[0057] The realization of the objectives, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0058] To better understand the above technical solutions, exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0059] First embodiment

[0060] Reference Figure 1 In this embodiment, the UAV small target detection method includes the following steps:

[0061] Step S10, obtaining an image dataset containing multiple small targets collected by a drone;

[0062] In this embodiment, a small target refers to a target with a pixel area smaller than or equal to 1024 pixels.

[0063] Step S20, inputting the image data set into the trained deep learning network for feature extraction;

[0064] The deep learning network uses multiple deep spatial feature fusion modules to fuse semantic features of different scales, and aligns the fused feature map to the same resolution as the second convolutional layer through a lightweight upsampling operator DySample. The aligned feature map is input into the feature extraction layer and the residual block is used to fuse it with the shallow features to extract the detail features.

[0065] In this embodiment, YOLOv8 is used as the basic model of the deep learning network, and the backbone network of YOLOv8 is used as the backbone network of this implementation.

[0066] In the deep learning network architecture of this embodiment, on the one hand, the residual blocks in the deep layer of the network are replaced with a lightweight deep spatial feature fusion module, while the residual blocks are still used to extract features in the shallow layer.

[0067] Alternatively, in some embodiments, Figure 2 As shown in Figure 2, the construction process of the deep spatial feature fusion module is as follows:

[0068] (1) Build a standard convolutional layer, a depth-wise separable convolutional layer, and a dilated convolutional layer, and cascade them in series to form a deep spatial feature fusion module;

[0069] (2) Set the input parameters: c1: number of input channels, c2: number of output channels, e: channel scaling factor (set the default value to 4);

[0070] (3) Divide the number of input channels c1 by the scaling factor e to obtain the number of hidden channels c_;

[0071] (4) Create the first layer of standard convolution, reducing the number of input channels from c1 to the number of hidden channels c_, with a convolution kernel size of 1 and a stride of 1;

[0072] (5) Create the second layer of standard convolution, transform the number of channels after splicing c_*4+c1 to the number of output channels c2, the convolution kernel size is 1, and the step size is 1;

[0073] (6) Create three depth-wise separable convolutional layers with a kernel size of 3 and the number of group convolutions equal to the number of hidden channels;

[0074] (7) Create three dilated convolutional layers with a kernel size of 3 and dilation rates of 1, 2, and 3 respectively;

[0075] (8) Perform a 1×1 convolution on the input x, reducing the number of channels from c1 to c_, and output y0;

[0076] (9) Pad y0 (pad = 1) to ensure that the output size remains unchanged, and use depth-wise separable convolution and dilation rate 1 dilated convolution to output y1;

[0077] (10) Pad y1 (pad = 2) to ensure that the output size remains unchanged, and use depth-wise separable convolution and dilation rate 2 dilated convolution to output y2;

[0078] (11) Pad y2 (pad=3) to ensure that the output size remains unchanged, and use depthwise separable convolution and dilation rate 3 dilated convolution to output y3;

[0079] (12) Concatenate x, y0, y1, y2, y3 along the channel dimension, perform feature transformation through the convolutional layer cv2, and output the final result.

[0080] On the other hand, in the process of fusing semantic features of different scales in the deep spatial feature fusion module, the fusion strategy of the deep learning network is a two-headed asymptotic fusion strategy, which specifically includes:

[0081] In the initial stage of integration, Indicates being in layer, and below it The feature maps of the first stage (with BackBone as the starting stage) are fused in parallel with the output from BackBone:

[0082]

[0083]

[0084] in, Corresponding to BackBone output , represents the strided convolution downsampling operation, represents the upsampling operation, and Indicates that it is in the middle stage of the downsampling phase;

[0085] In the later stage of fusion, the semantics of the generated intermediate layers are gradually integrated:

[0086]

[0087]

[0088] in, Integration in the middle layer ,and Then further integrate the output from the deepest layer of BackBone with the output of the previous stage of P3 layer ,DSF is the deep spatial feature fusion module;

[0089] In the dual-headed progressive fusion strategy, the shallow layer gradually acquires the semantics of the deep layer. ,when Sometimes:

[0090]

[0091] when Sometimes:

[0092]

[0093] Where, Indicates splicing by channel dimension, Indicates that the feature map is adjusted in resolution by upsampling.

[0094] Optionally, the deep spatial feature fusion module includes:

[0095]

[0096]

[0097] Where, Represents the output of semantic features after deep hole convolution processing, is the output after the previous layer of hole convolution processing, is the learnable parameter of the deep dilated convolution, represents the depth-wise dilated convolution operation, is the expansion rate, is the final output feature map, represents point-wise convolution, Indicates channel splicing.

[0098] Furthermore, in some specific implementation schemes, the depthwise dilated convolution operation is performed by a standard convolution layer, a depthwise separable convolution layer, and a dilated convolution layer, wherein: the convolution kernel of the standard convolution is 3; the convolution kernel of the depthwise separable convolution is 3, the number of group convolutions is the number of hidden channels, and the number of input channels is 4; the convolution kernel of the dilated convolution is 3, and the expansion rates are 1, 2, and 3, respectively.

[0099] On the other hand, the fused feature map is aligned to the same resolution as the second convolutional layer through the lightweight upsampling operator DySample, and the aligned feature map is input into the feature extraction layer to extract detail features after being fused with shallow features using the residual block.

[0100] Specifically, a sampling point generator is used to generate a dynamic sampling offset for each pixel point, and then the original grid position is added to the dynamic sampling offset to obtain a sampling set;

[0101] Resampling the input fused feature map through a grid sampling function to obtain the upsampled feature;

[0102] The generation strategy of the sampling point generator includes a static range factor strategy and a dynamic range factor strategy. The static range factor strategy includes: generating a dynamic sampling offset through a linear layer.

[0103] The dynamic range factor strategy includes: first generating a range factor through content perception, and then applying the range factor to a linear layer and pixel transformation to generate the dynamic sampling offset.

[0104] Optionally, in a specific embodiment, referring to Figure 3 , the steps of extracting detail features by using residual blocks to fuse them with shallow features can be as follows:

[0105] (1) Use DySample to upsample C5 to C4 resolution, and use 2*2 convolution to downsample C2 to C3 resolution;

[0106] (2) Use the deep spatial feature fusion module to fuse the sampled C5 and C4, C2 and C3 to obtain C4´ and C3´;

[0107] (3) Use DySample to upsample C4´ to C3 resolution, and use the deep spatial feature fusion module to fuse C4´ and C3´ to obtain C3´´;

[0108] (4) Use DySample to upsample C5 to C3 resolution, and use the deep spatial feature fusion module to fuse C5 and C3´´ to obtain C3´´´;

[0109] (5) Use DySample to upsample C3´, C3´´, and C3´´´ to C2 resolution respectively, and use the residual block to extract features to obtain C2´, C2´´, and C2´´´ respectively.

[0110] Furthermore, Concat is used to fuse C2 and C2´, C2 and C2´´, C2 and C2´´´, C2´ and C2´´, and C2´´ and C2´´´ respectively, and the residual block is used to extract the features in the fused feature map;

[0111] Use the Concat architecture to concatenate the above fused feature maps;

[0112] Arrange the feature maps of each layer from C2 to C5 in sequence to obtain a multi-scale L-shaped pyramid, as shown in Figure 4 shown.

[0113] Step S30, obtaining the predicted category and predicted box coordinates corresponding to each image output by the deep learning network;

[0114] Step S40 , filtering out target prediction frames without overlap according to the prediction frame coordinates, and marking target positions and target categories corresponding to each small target on the image based on the target prediction frames.

[0115] In this embodiment, the test set images in the dataset are resized to 640×640 and input into the trained deep learning network for forward propagation to obtain the predicted category and predicted box coordinates of each image.

[0116] Optionally, the non-maximum suppression method can be used to filter out overlapping prediction boxes, and the final prediction box can be used to mark the location and category of the target on the image to complete target detection.

[0117] In the technical solution provided in this embodiment, on the one hand, a more lightweight deep spatial feature fusion module is used to replace the deep residual block to reduce channel interaction, thereby avoiding the redundancy and inefficiency of computing resources caused by a large number of stacked residual blocks in deep feature extraction; on the other hand, in the feature alignment task, a lightweight upsampling operator DySample is introduced, so that this application can further save computing resources without sacrificing detection accuracy.

[0118] Second embodiment

[0119] As an optional embodiment, this embodiment provides a method for constructing and training a deep learning network based on the first embodiment.

[0120] Specifically, for the construction phase, refer to Figure 5 The deep learning network architecture shown is based on YOLOv8. It uses the prediction module of the fully convolutional one-stage object detection framework, which includes a classification subnetwork and a bounding box regression subnetwork. The YOLOv8 backbone network adopts a modular design. It uses components such as convolutional layers to extract features and downsample, encoding the input image into a multi-level feature map for subsequent object localization and classification by the detection head. The convolutional layers use a 3×3 kernel size with a stride of 1 or 2, with downsampling occurring when the stride is 2. The number of channels increases gradually: 64 → 128 → 256 → 512 → 1024.

[0121] During training, the data was normalized to the range [0, 1] and the image size was resized to 640×640. Deep learning network training hyperparameters were set: the number of epochs was set to 300, the learning rate was set to 0.01, and the batch size of images was set to 8. A batch of images from the training set was then fed into the deep learning network for forward propagation, yielding predicted categories and bounding box coordinates. The CIoU loss function was used to calculate the cross-entropy loss between the target category and the predicted category. The cross-entropy loss was used to measure the accuracy of the model's predictions. Finally, epoch-by-epoch metrics such as mAP50 and mAP50-95 were calculated on the validation set to measure detection accuracy. The performance change at each epoch was recorded to determine whether the model was overfitting or underfitting. Training was terminated after the specified number of epochs.

[0122] Third embodiment

[0123] This embodiment further illustrates the effect of the deep learning network model proposed in the above embodiment:

[0124] (1) Simulation experiment conditions:

[0125] The experiments in this example were conducted on a Windows operating system. The experimental code was written in Python (3.9), implemented based on the deep learning framework PyTorch (2.4.0+cu121), and run on an NVIDIA 4060 Ti GPU workstation.

[0126] (2) Simulation experiment content and result analysis

[0127] On the public dataset VisDrone2019, we conducted a simulation comparative analysis using the deep learning network model proposed in this application and existing object detection methods. We obtained the object detection bounding boxes and categories, and used the mean average precision (mAP50) and the mean average precision (mAP50-95) calculated at IoU thresholds ranging from 0.50 to 0.95 (increments of 0.05) as evaluation metrics. The simulation results are shown in Table 1:

[0128] Table 1 Test results of the present invention and prior art in simulation experiments

[0129]

[0130] As can be seen from Table 1, compared with the existing technologies GFL and RetinaNet, when the image input size is the same, the accuracy of the present invention is higher, and the computational complexity is much smaller than the two. Compared with YOLOv5n, although the computational complexity of the present invention is larger, the detection accuracy is significantly improved: mAP50 is 11.9% higher than YOLOv5n, and mAP50-95 is 9.4% higher than YOLOv5n. This is enough to show that the present invention can significantly improve the inference speed while ensuring high accuracy. Especially in small target detection tasks, the present invention not only has advantages in computational efficiency, but also achieves higher accuracy in small target detection tasks, can effectively reduce computational costs without sacrificing the detection performance of the model, meet the strict requirements for computational efficiency in practical applications, have good practical application value, and is particularly suitable for embedded or edge computing devices with limited resources.

[0131] In addition, as an implementation solution, the present application also proposes a deep learning network model, which includes:

[0132] YOLOv8 architecture;

[0133] Multiple deep spatial feature fusion modules are used to fuse semantic features of different scales in image datasets containing multiple small objects collected by drones;

[0134] The dual-headed progressive fusion module is used to align the fused feature map to the same resolution as the second convolutional layer through the lightweight upsampling operator DySample, and input the aligned feature map into the feature extraction layer to extract detailed features after fusing it with the shallow features using the residual block.

[0135] In addition, as an implementation solution, Figure 6 This is a schematic diagram of the architecture of the hardware operating environment of the drone small target detection system involved in the embodiment of the present application.

[0136] like Figure 6As shown, the drone small target detection system may include: a processor 1001, such as a CPU, a memory 1005, a user interface 1003, a network interface 1004, and a communication bus 1002. Communication bus 1002 is used to enable communication between these components. User interface 1003 may include a display and an input unit, such as a keyboard. Optionally, user interface 1003 may also include a standard wired interface or a wireless interface. Network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). Memory 1005 may be high-speed RAM or non-volatile memory, such as disk storage. Memory 1005 may also be a storage device independent of processor 1001.

[0137] Those skilled in the art will understand that Figure 6 The UAV small target detection system architecture shown in the figure does not constitute a limitation on the UAV small target detection system, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0138] like Figure 6 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a computer program. The operating system is a program that manages and controls the hardware and software resources of the drone small target detection system, and the computer program and other software or programs are executed.

[0139] exist Figure 6 In the drone small target detection system shown, the user interface 1003 is mainly used to connect to the terminal and communicate data with the terminal; the network interface 1004 is mainly used for the background server and communicates data with the background server; the processor 1001 can be used to call the computer program stored in the memory 1005.

[0140] In this embodiment, the UAV small target detection system includes: a memory 1005, a processor 1001, and a computer program stored in the memory and executable on the processor, wherein:

[0141] When the processor 1001 calls the computer program stored in the memory 1005, it performs the following operations:

[0142] Obtain an image dataset containing multiple small targets collected by a drone;

[0143] Inputting the image dataset into the trained deep learning network for feature extraction;

[0144] The deep learning network uses multiple deep spatial feature fusion modules to fuse semantic features of different scales, and aligns the fused feature map to the same resolution as the second convolutional layer through a lightweight upsampling operator DySample. The aligned feature map is input into the feature extraction layer and the residual block is used to fuse it with the shallow features to extract the detail features.

[0145] Obtain the predicted category and predicted box coordinates corresponding to each image output by the deep learning network;

[0146] Target prediction frames without overlap are screened out according to the prediction frame coordinates, so as to mark target positions and target categories corresponding to the respective small targets on the image based on the target prediction frames.

[0147] When the processor 1001 calls the computer program stored in the memory 1005, it performs the following operations:

[0148] In the initial stage of integration, Indicates being in layer, and below this layer The feature maps of the first stage (with BackBone as the starting stage) are fused in parallel with the output from BackBone:

[0149]

[0150]

[0151] in, Corresponding to BackBone output , represents the strided convolution downsampling operation, represents the upsampling operation, and Indicates that it is in the middle stage of the downsampling phase;

[0152] In the later stage of fusion, the semantics of the generated intermediate layers are gradually integrated:

[0153]

[0154]

[0155] in, Integration in the middle layer ,and Then further integrate the output from the deepest layer of BackBone with the output of the previous stage of P3 layer ,DSF is the deep spatial feature fusion module;

[0156] In the dual-headed progressive fusion strategy, the shallow layer gradually acquires the semantics of the deep layer. ,when Sometimes:

[0157]

[0158] when Sometimes:

[0159]

[0160] Where, Indicates splicing by channel dimension, Indicates that the feature map is adjusted in resolution by upsampling.

[0161] When the processor 1001 calls the computer program stored in the memory 1005, it performs the following operations:

[0162] Generate a dynamic sampling offset for each pixel using a sampling point generator, and then add the original grid position to the dynamic sampling offset to obtain a sampling set;

[0163] Resampling the input fused feature map through a grid sampling function to obtain the upsampled feature;

[0164] Among them, the generation strategy of the sampling point generator includes a static range factor strategy and a dynamic range factor strategy. The static range factor strategy includes: generating a dynamic sampling offset through a linear layer; the dynamic range factor strategy includes: first generating a range factor through content perception, and then generating the dynamic sampling offset by passing the range factor through a linear layer and pixel transformation.

[0165] When the processor 1001 calls the computer program stored in the memory 1005, it performs the following operations:

[0166] The non-maximum suppression method is used to screen out overlapping prediction boxes, and the remaining prediction boxes are used as the target prediction boxes.

[0167] Furthermore, those skilled in the art will appreciate that all or part of the steps in the method of the above-described embodiment can be implemented by instructing the relevant hardware through a computer program. The computer program includes program instructions, which can be stored in a computer-readable storage medium. The program instructions are executed by at least one processor in the drone small target detection system to implement the steps of the above-described method embodiment.

[0168] Therefore, the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the various steps of the drone small target detection method described in the above embodiment.

[0169] The computer-readable storage medium may be any computer-readable storage medium that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk.

[0170] It should be noted that since the storage medium provided in the embodiments of this application is the storage medium used to implement the method of the embodiments of this application, based on the method described in the embodiments of this application, those skilled in the art will be able to understand the specific structure and deformation of the storage medium, and therefore will not be described in detail here. All storage media used in the method of the embodiments of this application fall within the scope of protection to be provided by this application.

[0171] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0172] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0173] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0174] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0175] It should be noted that in the claims, any reference signs placed between parentheses shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claim. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The present application may be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names.

[0176] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0177] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A method for detecting small targets of unmanned aerial vehicles, characterized in that: The method comprises the following steps: Obtain an image dataset containing multiple small targets collected by a drone; Inputting the image dataset into the trained deep learning network for feature extraction; The deep learning network uses multiple deep spatial feature fusion modules to fuse semantic features of different scales, and aligns the fused feature map to the same resolution as the second convolutional layer through a lightweight upsampling operator DySample. The aligned feature map is input into the feature extraction layer and the residual block is used to fuse it with the shallow features to extract the detail features. Obtain the predicted category and predicted box coordinates corresponding to each image output by the deep learning network; Filtering non-overlapping target prediction frames according to the prediction frame coordinates, so as to mark target positions and target categories corresponding to each small target on the image based on the target prediction frames; The fusion strategy of the deep learning network is a two-headed asymptotic fusion strategy, which specifically includes: In the initial stage of integration, Indicates being in layer, and below this layer The feature maps of the stage are fused in parallel with the output from BackBone: ； ； in, Corresponding to BackBone output , represents the strided convolution downsampling operation, represents the upsampling operation, and Indicates that it is in the middle stage of the downsampling phase; In the later stage of fusion, the semantics of the generated intermediate layers are gradually integrated: ； ； in, Integration in the middle layer ,and Then further integrate the output from the deepest layer of BackBone with the output of the previous stage of P3 layer ,DSF is the deep spatial feature fusion module; In the dual-headed progressive fusion strategy, the shallow layer gradually acquires the semantics of the deep layer. ,when Sometimes: ； when Sometimes: ； Where, Indicates splicing by channel dimension, Indicates that the feature map is adjusted in resolution by upsampling.

2. The method according to claim 1, wherein The deep spatial feature fusion module includes: ； ； Where, Represents the output of semantic features after deep hole convolution processing, is the output after the previous layer of hole convolution processing, is the learnable parameter of the deep dilated convolution, represents the depth-wise dilated convolution operation, is the expansion rate, is the final output feature map, represents point-wise convolution, Indicates channel splicing.

3. The method according to claim 2, wherein The depthwise dilated convolution operation is performed by a standard convolution layer, a depthwise separable convolution layer, and a dilated convolution layer, where: The convolution kernel of the standard convolution is 3; The convolution kernel of the depthwise separable convolution is 3, the number of group convolutions is the number of hidden channels, and the number of input channels is 4; The convolution kernel of the dilated convolution is 3, and the dilation rates are 1, 2, and 3 respectively.

4. The method according to claim 1, wherein The step of aligning the fused feature map to the same resolution as the second convolutional layer through the lightweight upsampling operator DySample specifically includes: Generate a dynamic sampling offset for each pixel using a sampling point generator, and then add the original grid position to the dynamic sampling offset to obtain a sampling set; Resampling the input fused feature map through a grid sampling function to obtain the upsampled feature; Among them, the generation strategy of the sampling point generator includes a static range factor strategy and a dynamic range factor strategy. The static range factor strategy includes: generating a dynamic sampling offset through a linear layer; the dynamic range factor strategy includes: first generating a range factor through content perception, and then generating the dynamic sampling offset by passing the range factor through a linear layer and pixel transformation.

5. The method according to claim 1, wherein The hyperparameters during the training of the deep learning network are set as follows: The number of iterations is set to 300, the learning rate is set to 0.01, and the number of images input at a time is set to 8.

6. The method according to claim 1, wherein The step of filtering out the target prediction frame without overlap according to the prediction frame coordinates includes: The non-maximum suppression method is used to screen out overlapping prediction boxes, and the remaining prediction boxes are used as the target prediction boxes.

7. A deep learning network model for implementing the UAV small target detection method according to claim 1, characterized in that: The deep learning network model includes: YOLOv8 architecture; Multiple deep spatial feature fusion modules are used to fuse semantic features of different scales in image datasets containing multiple small objects collected by drones; The dual-headed progressive fusion module is used to align the fused feature map to the same resolution as the second convolutional layer through the lightweight upsampling operator DySample, and input the aligned feature map into the feature extraction layer to extract detailed features after fusing it with the shallow features using the residual block.

8. A UAV small target detection system, characterized in that: The drone small target detection system includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the steps of the drone small target detection method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the method for detecting small targets of a drone according to any one of claims 1 to 6.

Images