End-to-end small target detection method with foreground enhancement for UAV aerial images

Abstract

The present invention proposes a foreground-enhanced end-to-end small target detection method for drone aerial images, comprising: inputting the image taken by the drone into a lightweight backbone network ShuffleNetV2 to obtain feature maps of different sizes; adding an FPN feature pyramid structure after the feature map to fuse multi-scale features to obtain a fused multi-scale feature map; inputting the multi-scale feature map into a single-stage detection head to obtain a rough detection result, and obtaining a foreground map from the rough detection result; passing a layer of convolution on the foreground map to obtain a foreground-enhanced multi-scale feature map; inputting the foreground-enhanced multi-scale feature map into the single-stage detection head again to obtain a fine detection frame; using a dynamic soft-weighted redundant detection frame suppression algorithm to eliminate redundant redundant frames of the same target in the fine detection frame to obtain the final detection result. The method proposed by the present invention has a good detection effect for drone aerial scenes with dense local targets and sparse global foregrounds.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision target detection, and in particular relates to a foreground enhanced end-to-end small target detection method for unmanned aerial vehicle aerial images. Background Art

[0002] The drone industry is developing rapidly and plays an important role in the industrial, agricultural, and military sectors. Drone aerial images can provide information that ground-based photography cannot. With the popularity of consumer drones, target detection in drone images has gradually become a popular research direction. Currently, there are two main problems with small target detection methods in drone aerial photography scenarios: (1) Drones are often located at a high position for shooting, so a notable feature of drone images is the presence of a large number of small-sized targets; (2) Although drone high-definition cameras can capture high-resolution images, the proportion of foreground targets to be detected (pedestrians, bicycles, cars, etc.) is very small and unevenly distributed, and most areas are background areas that do not contain any targets, which poses a huge challenge to target detection.

[0003] Although object detection on natural images, such as PASCAL VOC and MS COCO, has made some progress, the performance on drone images is still unsatisfactory. As mentioned above, the high proportion of small objects and high image resolution make it more challenging. Inspired by the region search strategy in general object detection, most current drone detection research adopts a coarse-to-fine framework to solve the above problems. This framework successively adopts a simple unsupervised selection strategy or a coarse detector to roughly estimate the area where small objects are densely distributed, and uses a fine detector to accurately locate the objects on it. For example, the PTS network uses a tiling-based method for uniform segmentation to generate sub-regions, but it often destroys the integrity of the target when truncating the image; ClutDet proposes to use a sub-network to obtain the cropped sub-regions from the original input; DMNet proposes to estimate the object density in the original image and then separate the sub-regions into the smallest area of ​​connected possible blocks. The above methods have promoted object detection in drone aerial images. However, the sub-regions generated based on unsupervised methods are not effective. Due to the lack of foreground information of the drone target instance, the cropped sub-regions contain a lot of background information. The methods based on sub-networks or rough detection networks require the construction of multiple network models to infer the image, and multiple models need to be trained separately, which makes training more difficult and will result in more video memory usage and higher inference latency during the inference process.

[0004] As mentioned above, most of the current mainstream methods mainly use the method of manually designing foreground clustering strategies or building sub-networks to detect small drone targets from coarse to fine. However, the current technical solutions still have two problems:

[0005] 1) The foreground extraction module is difficult to cope with complex environments. Current UAV small target detection methods usually use manually designed clustering algorithms to aggregate sub-regions that may contain foreground targets. However, there are a large number of manually designed parameters in the clustering process, including the number of clustering regions, scaling factors, intersection-over-union ratio thresholds, etc. The search space for these hyperparameters is very large, and the clustering effect is very sensitive to them. Static parameters are difficult to cope with complex and changing UAV detection environments. Therefore, the manually designed foreground extraction strategy will make the model less robust and difficult to cope with complex and changing environments.

[0006] 2) Multiple serial network modules increase the inference burden. The current two-step method for detecting small drone targets from coarse to fine requires the design of a separate sub-network for coarse target detection, then foreground area clustering, and then fine target detection. The two networks are trained independently, making it difficult to obtain the optimal training network. The method of stacking multiple network serial inferences at the same time, as well as the inefficient foreground extraction module that cannot be executed in parallel in the middle, will cause serious memory usage and high inference latency. Summary of the invention

[0007] The present invention aims to solve one of the technical problems in the related art at least to a certain extent.

[0008] To this end, the purpose of the present invention is to propose a foreground enhanced end-to-end small target detection method for UAV aerial images, so as to realize true end-to-end UAV small target detection.

[0009] To achieve the above-mentioned purpose, the first embodiment of the present invention proposes a foreground enhanced end-to-end small target detection method for drone aerial images, comprising:

[0010] The images taken by the drone are input into the lightweight backbone network ShuffleNetV2 to obtain feature maps of different sizes; an FPN feature pyramid structure is added after the feature map to fuse multi-scale features to obtain a fused multi-scale feature map;

[0011] Inputting the multi-scale feature map into a single-stage detection head to obtain a rough detection result, and obtaining a foreground map from the rough detection result;

[0012] The foreground image is subjected to a layer of convolution to obtain a multi-scale feature map with foreground enhancement;

[0013] The foreground-enhanced multi-scale feature map is inputted into the single-stage detection head again to obtain a refined detection frame;

[0014] A redundant detection frame suppression algorithm with dynamic soft weights is used to eliminate redundant frames of the same target in the fine detection frame to obtain the final detection result.

[0015] In addition, the foreground enhancement end-to-end small target detection method for drone aerial images according to the above embodiment of the present invention may also have the following additional technical features:

[0016] Furthermore, in one embodiment of the present invention, before the image taken by the drone is input into the lightweight backbone network ShuffleNetV2, the image is also preprocessed:

[0017] Scaling the image to a fixed size;

[0018] performing standardization processing on the image;

[0019] The image size is padded to a multiple of 32 to accommodate the stride of the model.

[0020] Further, in one embodiment of the present invention, the step of subjecting the foreground image to a layer of convolution to obtain a foreground enhanced multi-scale feature map includes:

[0021] The spatial offset and mask of the foreground area are obtained from the foreground image, and the deformable convolution is guided to focus on the foreground area; wherein the deformable convolution is as follows:

[0022]

[0023] Where x represents the feature map, w k represents the weight of the convolution kernel at index k, where k represents the coordinates of a pixel in the feature map, and p k represents the coordinates of the feature map to be convolved with the convolution kernel position k, Δp k represents the learned spatial offset, Δm k represents the learned spatial mask;

[0024] The deformable convolution is used to perform foreground enhancement on the multi-scale feature map to obtain the foreground enhanced multi-scale feature map.

[0025] Further, in one embodiment of the present invention, the redundant detection frame suppression algorithm using dynamic soft weights eliminates redundant frames of the same target in the fine detection frame, including:

[0026] S201: Input the initial detection frame coordinates B = {b1, ..., b N}, detection box score S = {s1,…,s N}, N is the number of detection boxes; define the non-redundant box set D to be returned, which is initially an empty set;

[0027] S202: sorting the detection box scores in S from high to low, and arranging the detection box coordinates B in the order;

[0028] S203: The detection box b with the highest score m Save it in D and remove it from B and S;

[0029] S204: Traverse all remaining detection boxes b in B i and its corresponding score s i , update the score s of each detection box according to the following formula i ;

[0030]

[0031] Among them, σ Gaussian variance parameter is set to 1 by default, and iou is the intersection-over-union ratio of two detection boxes. The specific implementation is as follows:

[0032]

[0033] S205: If B is not empty at this time, return to execute S201, otherwise execute S206;

[0034] S206: Return the detection box D and the corresponding score S after redundant box suppression.

[0035] To achieve the above-mentioned purpose, the second embodiment of the present invention proposes an end-to-end small target detection device for foreground enhancement of drone aerial images, comprising the following modules:

[0036] The extraction module is used to input the image taken by the drone into the lightweight backbone network ShuffleNetV2 to obtain feature maps of different sizes; add the FPN feature pyramid structure after the feature map to fuse multi-scale features to obtain a fused multi-scale feature map;

[0037] A roughness detection module, used for inputting the multi-scale feature map into a single-stage detection head to obtain a roughness detection result, and obtaining a foreground map from the roughness detection result;

[0038] An enhancement module, used for obtaining a foreground enhanced multi-scale feature map by performing a layer of convolution on the foreground image;

[0039] A fine detection module, which inputs the foreground enhanced multi-scale feature map into the single-stage detection head again to obtain a fine detection frame;

[0040] The detection module is used to use a redundant detection frame suppression algorithm with dynamic soft weights to eliminate redundant frames of the same target in the fine detection frame to obtain a final detection result.

[0041] Furthermore, in one embodiment of the present invention, the extraction module further includes a preprocessing unit, which is used to:

[0042] Scaling the image to a fixed size;

[0043] performing standardization processing on the image;

[0044] The image size is padded to a multiple of 32 to accommodate the stride of the model.

[0045] Furthermore, in one embodiment of the present invention, the enhancement module is further used to:

[0046] The spatial offset and mask of the foreground area are obtained from the foreground image, and the deformable convolution is guided to focus on the foreground area; wherein the deformable convolution is as follows:

[0047]

[0048] Where x represents the feature map, w k represents the weight of the convolution kernel at index k, where k represents the coordinates of a pixel in the feature map, and p k represents the coordinates of the feature map to be convolved with the convolution kernel position k, Δp k represents the learned spatial offset, Δm k represents the learned spatial mask;

[0049] The deformable convolution is used to perform foreground enhancement on the multi-scale feature map to obtain the foreground enhanced multi-scale feature map.

[0050] To achieve the above-mentioned purpose, the third aspect of the present invention proposes a computer device, characterized in that it includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the foreground enhanced end-to-end small target detection method for drone aerial images as described above.

[0051] To achieve the above-mentioned objectives, the fourth aspect of the present invention proposes a computer-readable storage medium on which a computer program is stored, characterized in that when the computer program is executed by a processor, it implements the foreground enhanced end-to-end small target detection method for drone aerial images as described above.

[0052] The foreground enhancement end-to-end small target detection method for drone aerial images proposed in the embodiment of the present invention uses a single-stage model architecture to achieve a multi-stage detection effect from coarse to fine, and eliminates the manually designed foreground extraction module, thus realizing a true end-to-end drone small target detection model. Due to its ingenious design, it takes into account both speed and accuracy, and is suitable for efficient operation on low-power devices such as drones. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0054] Figure 1 A flowchart of an end-to-end small target detection method for foreground enhancement of drone aerial images provided by an embodiment of the present invention.

[0055] Figure 2 A schematic diagram comparing an existing mainstream multi-model multi-stage detection process provided by an embodiment of the present invention and the single-stage model detection process of the present invention.

[0056] Figure 3 A schematic diagram of an end-to-end small target detection device for foreground enhancement of drone aerial images provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0057] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0058] The following describes an end-to-end small target detection method for foreground enhancement of drone aerial images according to an embodiment of the present invention with reference to the accompanying drawings.

[0059] Figure 1 A flowchart of an end-to-end small target detection method for foreground enhancement of drone aerial images provided by an embodiment of the present invention.

[0060] like Figure 1 As shown, the foreground enhancement end-to-end small target detection method for UAV aerial images includes the following steps:

[0061] S101: Input the image taken by the drone into the lightweight backbone network ShuffleNetV2 to obtain feature maps of different sizes; add the FPN feature pyramid structure after the feature map to fuse multi-scale features to obtain a fused multi-scale feature map;

[0062] like Figure 2As shown in the left half, existing methods usually use multiple models and multi-stage serial processing. First, the image is input into a rough backbone network and a rough detection head, and a rough detection result is obtained through post-processing; then, rough target clustering is performed to obtain the foreground area, and the foreground image is cropped on the original image; finally, the foreground image is input into the fine backbone network and the fine detection head one by one, and the final prediction result is obtained through post-processing. Compared with existing methods, the present invention only uses a single-stage detector, uses a backbone network to perform feature extraction to obtain a feature map, and obtains a rough prediction and foreground map through a single-stage detection head; then, a learnable foreground enhancement module is used to perform foreground enhancement on the feature map obtained by the backbone network at the feature level to obtain a feature map after foreground enhancement; finally, a single-stage detection head that shares weights with the rough detection stage is used to perform fine detection, and the final prediction result is obtained through post-processing of dynamic soft weights. As shown Figure 2 As shown in the right half, the structure of the method of the present invention is very concise and efficient, and can significantly reduce the inference delay and memory usage of the model.

[0063] Furthermore, in one embodiment of the present invention, before the image taken by the drone is input into the lightweight backbone network ShuffleNetV2, the image is also preprocessed:

[0064] Scaling the image to a fixed size;

[0065] performing standardization processing on the image;

[0066] The image size is padded to a multiple of 32 to accommodate the stride of the model.

[0067] S102: inputting the multi-scale feature map into a single-stage detection head to obtain a rough detection result, and obtaining a foreground map from the rough detection result;

[0068] In order to solve the problem that the existing foreground extraction methods are poor in robustness and separated from the detection network, the present invention designs a foreground enhancement module that can learn offset convolution. The module first obtains the foreground image from the rough prediction, and then obtains the spatial offset and mask of the foreground area from the foreground image to guide the deformable convolution to focus on the foreground area.

[0069] Further, in one embodiment of the present invention, the step of subjecting the foreground image to a layer of convolution to obtain a foreground enhanced multi-scale feature map includes:

[0070] The spatial offset and mask of the foreground area are obtained from the foreground image, and the deformable convolution is guided to focus on the foreground area; wherein the deformable convolution is as follows:

[0071]

[0072] Among them, x represents the feature map, w k represents the weight of the convolution kernel at index k, where k represents the coordinates of a pixel in the feature map, and p k represents the coordinates of the feature map to be convolved with the convolution kernel position k, Δp k represents the learned spatial offset, Δm k represents the learned spatial mask;

[0073] The deformable convolution is used to perform foreground enhancement on the multi-scale feature map to obtain the foreground enhanced multi-scale feature map.

[0074] S103: performing a layer of convolution on the foreground image to obtain a foreground enhanced multi-scale feature map;

[0075] S104: inputting the foreground enhanced multi-scale feature map into the single-stage detection head again to obtain a fine detection frame;

[0076] S105: Using a redundant detection frame suppression algorithm with dynamic soft weights, the redundant frames of the same target in the fine detection frame are eliminated to obtain a final detection result.

[0077] Target detection uses a dense prediction method, and usually uses non-maximum suppression (NMS) to eliminate redundant frames. However, there are often dense and crowded target frames in drone aerial images, and it is difficult to determine whether they are redundant frames by setting a fixed threshold. To solve this problem, the present invention designs a dynamic soft-weighted dense target frame suppression algorithm, which gives each detection frame a soft weight to measure its importance, rather than directly judging it as a redundant frame.

[0078] Further, in one embodiment of the present invention, the redundant detection frame suppression algorithm using dynamic soft weights eliminates redundant frames of the same target in the fine detection frame, including:

[0079] S201: Input the initial detection frame coordinates B = {b1, ..., b N}, detection box score S = {s1,…,s N}, N is the number of detection boxes; define the non-redundant box set D to be returned, which is initially an empty set;

[0080] S202: sorting the detection box scores in S from high to low, and arranging the detection box coordinates B in the order;

[0081] S203: The detection box b with the highest score m Save it in D and remove it from B and S;

[0082] S204: Traverse all remaining detection boxes b in B i and its corresponding score s i, update the score s of each detection box according to the following formula i ;

[0083]

[0084] Among them, σ Gaussian variance parameter is set to 1 by default, and iou is the intersection-over-union ratio of two detection boxes. The specific implementation is as follows:

[0085]

[0086] S205: If B is not empty at this time, return to execute S201, otherwise execute S206;

[0087] S206: Return the detection box D and the corresponding score S after redundant box suppression.

[0088] The foreground enhancement end-to-end small target detection method for drone aerial images proposed in the embodiment of the present invention uses a single-stage model architecture to achieve a multi-stage detection effect from coarse to fine, and eliminates the manually designed foreground extraction module, thus realizing a true end-to-end drone small target detection model. Due to its ingenious design, it takes into account both speed and accuracy, and is suitable for efficient operation on low-power devices such as drones.

[0089] Compared with the prior art, the present invention has the following advantages:

[0090] 1) Mainstream UAV aerial image small target detection technology usually adopts a coarse-to-fine detection paradigm. Current methods usually use multiple models and multi-stage serial processing. First, a coarse detector is used to obtain a rough prediction result; then foreground clustering is performed to crop the foreground area on the original image; then the foreground image is input into a fine detector to obtain the final prediction output. The method invented in the present application only uses a single-stage detector, and the two detections are only performed on the detection head with shared weights, which is very lightweight. Compared with the prior art, the method of the present invention has a simpler structure, can be trained end-to-end, occupies less memory and computing resources in the inference process, and has the same detection accuracy as the prior art, and can achieve high speed and high precision on low-power UAV devices.

[0091] 2) Existing UAV aerial image small target detection technology usually uses manually designed clustering algorithms to extract foreground areas. There are a large number of manually designed parameters in the clustering process, including the number of clustering areas, scaling factors, intersection-over-union ratio thresholds, etc. At the same time, different scene images may require different clustering parameters, so existing methods are difficult to apply to complex UAV aerial photography environments. The method invented in the present application establishes an end-to-end detection paradigm, uses a neural network to learn to focus on the foreground area from a large amount of data, and adaptively enhances the feature foreground. Compared with existing methods, the method of the present invention has a stronger adaptability to complex environments and meets the needs of UAV aerial photography detection.

[0092] In order to implement the above embodiments, the present invention also proposes an end-to-end small target detection device for foreground enhancement of drone aerial images.

[0093] Figure 3 A schematic structural diagram of an end-to-end small target detection device for foreground enhancement of drone aerial images provided by an embodiment of the present invention.

[0094] like Figure 3 As shown, the end-to-end small target detection device for foreground enhancement of drone aerial images includes: an extraction module 100, a rough detection module 200, an enhancement module 300, a fine detection module 400, and a detection module 500, wherein:

[0095] The extraction module is used to input the image taken by the drone into the lightweight backbone network ShuffleNetV2 to obtain feature maps of different sizes; add the FPN feature pyramid structure after the feature map to fuse multi-scale features to obtain a fused multi-scale feature map;

[0096] A roughness detection module, used for inputting the multi-scale feature map into a single-stage detection head to obtain a roughness detection result, and obtaining a foreground map from the roughness detection result;

[0097] An enhancement module, used for obtaining a foreground enhanced multi-scale feature map by performing a layer of convolution on the foreground image;

[0098] A fine detection module, which inputs the foreground enhanced multi-scale feature map into the single-stage detection head again to obtain a fine detection frame;

[0099] The detection module is used to use a redundant detection frame suppression algorithm with dynamic soft weights to eliminate redundant frames of the same target in the fine detection frame to obtain a final detection result.

[0100] Furthermore, in one embodiment of the present invention, the extraction module further includes a preprocessing unit, which is used to:

[0101] Scaling the image to a fixed size;

[0102] performing standardization processing on the image;

[0103] The image size is padded to a multiple of 32 to accommodate the stride of the model.

[0104] Furthermore, in one embodiment of the present invention, the enhancement module is further used to:

[0105] The spatial offset and mask of the foreground area are obtained from the foreground image, and the deformable convolution is guided to focus on the foreground area; wherein the deformable convolution is as follows:

[0106]

[0107] Where x represents the feature map, w k represents the weight of the convolution kernel at index k, where k represents the coordinates of a pixel in the feature map, and p k represents the coordinates of the feature map to be convolved with the convolution kernel position k, Δp k represents the learned spatial offset, Δm k represents the learned spatial mask;

[0108] The deformable convolution is used to perform foreground enhancement on the multi-scale feature map to obtain the foreground enhanced multi-scale feature map.

[0109] To achieve the above-mentioned purpose, the third aspect of the present invention proposes a computer device, characterized in that it includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the foreground enhanced end-to-end small target detection method for drone aerial images as described above is implemented.

[0110] To achieve the above-mentioned objectives, the fourth aspect of the present invention proposes a computer-readable storage medium on which a computer program is stored, characterized in that when the computer program is executed by a processor, the foreground enhanced end-to-end small target detection method for drone aerial images as described above is implemented.

[0111] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0112] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0113] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A foreground enhanced end-to-end small target detection method for UAV aerial images, characterized in that: The following steps are involved: The images taken by the drone are input into the lightweight backbone network ShuffleNetV2 to obtain feature maps of different sizes; an FPN feature pyramid structure is added after the feature map to fuse multi-scale features to obtain a fused multi-scale feature map; Inputting the multi-scale feature map into a single-stage detection head to obtain a rough detection result, and obtaining a foreground map from the rough detection result; The foreground image is subjected to a layer of convolution to obtain a multi-scale feature map with foreground enhancement; The foreground-enhanced multi-scale feature map is inputted into the single-stage detection head again to obtain a refined detection frame; Using a redundant detection frame suppression algorithm with dynamic soft weights, the redundant frames of the same target in the fine detection frame are eliminated to obtain the final detection result; The redundant detection frame suppression algorithm using dynamic soft weights eliminates redundant frames of the same target in the fine detection frame, including: S201: Input the initial detection frame coordinates B = {b1, ..., b N }, detection box score S = {s1,…,s N }, N is the number of detection boxes; define the non-redundant box set D to be returned, which is initially an empty set; S202: Sort the detection box scores in S from high to low, and arrange the detection box coordinates B in order; S203: The detection box b with the highest score m Save it in D and remove it from B and S; S204: Traverse all remaining detection boxes b in B i and its corresponding score s i , update the score s of each detection box according to the following formula i ; Among them, σ Gaussian variance parameter is set to 1 by default, and iou is the intersection-over-union ratio of two detection boxes. The specific implementation is as follows: S205: If B is not empty at this time, return to execute S201, otherwise execute S206; S206: Return the detection box D and the corresponding score S after redundant box suppression.

2. The method according to claim 1, characterized in that Before the images taken by the drone are input into the lightweight backbone network ShuffleNetV2, the images are also preprocessed: Scaling the image to a fixed size; performing standardization processing on the image; The image size is padded to a multiple of 32 to accommodate the stride of the model.

3. The method according to claim 1, characterized in that The step of obtaining a foreground enhanced multi-scale feature map by performing a layer of convolution on the foreground map comprises: The spatial offset and mask of the foreground area are obtained from the foreground image, and the deformable convolution is guided to focus on the foreground area; wherein the deformable convolution is as follows: Where x represents the feature map, w k represents the weight of the convolution kernel at index k, where k represents the coordinates of a pixel in the feature map, and p k represents the coordinates of the feature map to be convolved with the convolution kernel position k, Δp k represents the learned spatial offset, Δm k represents the learned spatial mask; The deformable convolution is used to perform foreground enhancement on the multi-scale feature map to obtain the foreground enhanced multi-scale feature map.

4. A foreground enhanced end-to-end small target detection device for drone aerial images, characterized in that: Includes the following modules: The extraction module is used to input the image taken by the drone into the lightweight backbone network ShuffleNetV2 to obtain feature maps of different sizes; add the FPN feature pyramid structure after the feature map to fuse multi-scale features to obtain a fused multi-scale feature map; A roughness detection module, used for inputting the multi-scale feature map into a single-stage detection head to obtain a roughness detection result, and obtaining a foreground map from the roughness detection result; An enhancement module, used for obtaining a foreground enhanced multi-scale feature map by performing a layer of convolution on the foreground image; A fine detection module, which inputs the foreground enhanced multi-scale feature map into the single-stage detection head again to obtain a fine detection frame; A detection module, used to use a redundant detection frame suppression algorithm with dynamic soft weights to eliminate redundant frames of the same target in the fine detection frame to obtain a final detection result; The detection module is also used for: S201: Input the initial detection frame coordinates B = {b1, ..., b N }, detection box score S = {s1,…,s N }, N is the number of detection boxes; define the non-redundant box set D to be returned, which is initially an empty set; S202: Sort the detection box scores in S from high to low, and arrange the detection box coordinates B in order; S203: The detection box b with the highest score m Save it in D and remove it from B and S; S204: Traverse all remaining detection boxes b in B i and its corresponding score s i , update the score s of each detection box according to the following formula i ; Among them, σ Gaussian variance parameter is set to 1 by default, and iou is the intersection-over-union ratio of two detection boxes. The specific implementation is as follows: S205: If B is not empty at this time, return to execute S201, otherwise execute S206; S206: Return the detection box D and the corresponding score S after redundant box suppression.

5. The device according to claim 4, characterized in that The extraction module further includes a preprocessing unit, which is used to: Scaling the image to a fixed size; performing standardization processing on the image; The image size is padded to a multiple of 32 to accommodate the stride of the model.

6. The device according to claim 4, characterized in that The enhancement module is further used for: The spatial offset and mask of the foreground area are obtained from the foreground image, and the deformable convolution is guided to focus on the foreground area; wherein the deformable convolution is as follows: Where x represents the feature map, w k represents the weight of the convolution kernel at index k, where k represents the coordinates of a pixel in the feature map, and p k represents the coordinates of the feature map to be convolved with the convolution kernel position k, Δp k represents the learned spatial offset, Δm k represents the learned spatial mask; The deformable convolution is used to perform foreground enhancement on the multi-scale feature map to obtain the foreground enhanced multi-scale feature map.

7. A computer device, characterized in that: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the foreground enhanced end-to-end small target detection method for drone aerial images as described in any one of claims 1 to 3.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the end-to-end small target detection method for foreground enhancement in drone aerial images is implemented as described in any one of claims 1 to 3.

Images