An unmanned aerial vehicle small target detection method and device based on fuzzy object enhancement

Abstract

The application belongs to the technical field of computer vision, and discloses a kind of unmanned aerial vehicle small target detection method and device based on fuzzy object enhancement, including the pretreatment and feature extraction of input unmanned aerial vehicle image frame, generate multi-scale feature map, and output the preliminary target detection result of each scale feature map through preliminary target detection;Based on the fuzzy object region extracted, the region type label is assigned to the unmanned aerial vehicle image frame, and the region type label is fuzzy object region or ordinary region;Discrete scale adaptive processing and cross-frame feature compensation mechanism are carried out on the fuzzy object region, and fine target detection is carried out;The fine target detection result and the candidate target of the preliminary target detection result in the ordinary region are merged as the complete small target detection result.The application obtains high-precision and low-overhead small target detection result, and is especially suitable for airborne or edge computing resource limited scene.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically relating to a method and apparatus for detecting small targets in unmanned aerial vehicles based on fuzzy object enhancement. Background Technology

[0002] Small target detection in drone aerial images has significant application value in fields such as security monitoring, border patrol, and disaster emergency response. Currently, mainstream target detection technologies mainly employ single-stage or two-stage detectors based on deep learning, and these methods are relatively mature in target detection at conventional scales.

[0003] However, in real-world drone applications, existing detection technologies face significant limitations. First, because drones typically operate at high altitudes or long distances, the pixel proportion of target objects in images is extremely low. When the target size is smaller than 32×32 pixels, the features are severely degraded after multiple downsampling cycles by the convolutional neural network, making effective detection difficult. Second, existing methods generally employ a globally uniform processing strategy, applying the same feature enhancement and detection process to the entire image without considering the differences in target quality across different regions. This results in unreasonable allocation of computational resources: redundant enhancement processing is performed on clear, ordinary regions, wasting computational resources; while there is a lack of targeted enhancement methods for blurred object regions, leading to limited performance improvement. Furthermore, single-frame detection methods struggle to address the instantaneous feature loss caused by motion blur and partial occlusion of small targets. Additionally, scale-adaptive processing for small targets often employs a continuous scale pyramid structure, which suffers from high computational complexity and inaccurate scale selection, making it difficult to dynamically match the optimal feature representation scale based on the actual target size.

[0004] In summary, existing UAV small target detection technologies have significant shortcomings in areas such as fuzzy object recognition, computational resource allocation, cross-frame feature utilization, and scale adaptive processing. They are unable to effectively improve the accuracy of small target detection while ensuring real-time performance. There is an urgent need for a detection technology that can achieve differentiated enhancement and efficient feature compensation. Summary of the Invention

[0005] The purpose of this invention is to provide a method and apparatus for small target detection on unmanned aerial vehicles (UAVs) based on fuzzy object enhancement. This method is suitable for real-time small-scale target detection tasks on UAVs in complex backgrounds, and is particularly suitable for scenarios with limited airborne or edge computing resources. This invention intelligently identifies regions requiring enhancement by determining fuzzy object regions and triggers the enhancement process only on these regions. By combining discrete scale adaptive processing and cross-frame feature compensation mechanisms, targeted enhancement is performed on fuzzy object regions to obtain high-precision and low-overhead small target detection results.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] Firstly, a method for detecting small targets on unmanned aerial vehicles (UAVs) based on fuzzy object enhancement is provided, comprising the following steps:

[0008] The input UAV image frames are preprocessed and feature extracted to generate multi-scale feature maps, and preliminary target detection results of each scale feature map are output through preliminary target detection.

[0009] Based on the multi-scale feature map and preliminary target detection results, the blurred object region is extracted according to the confidence level, target size and target response intensity;

[0010] Based on the extracted blurred object region, a region type label is assigned to the UAV image frame, which is either a blurred object region or a normal region.

[0011] Multi-scale regional features of each fuzzy object region are extracted, and the weights corresponding to each discrete scale are calculated. Based on the discrete scale, the multi-scale regional features are scaled to obtain preliminary enhanced regional features.

[0012] Filter historical secondary enhancement region features that match the fuzzy object region, and perform weighted fusion of historical secondary enhancement region features and initial enhancement region features to obtain the current secondary enhancement region features;

[0013] Based on the current secondary enhancement region features, fine target detection is performed. The fine target detection results are then merged with the candidate targets from the preliminary target detection results in the ordinary region to form a complete small target detection result.

[0014] Several alternative methods are provided below, but they are not intended as additional limitations on the overall solution above. They are merely further additions or optimizations. Provided there are no technical or logical contradictions, each alternative method can be combined individually with respect to the overall solution above, or multiple alternative methods can be combined with each other.

[0015] Preferably, the step of extracting the blurred object region based on the multi-scale feature map and preliminary target detection results, according to confidence level, target size, and target response intensity, includes:

[0016] Condition 1: If the detection confidence of the candidate target is lower than the first threshold and the spatial size of the candidate target is smaller than the second threshold, then the candidate target is marked as a fuzzy object and the spatial location of the fuzzy object is taken as the fuzzy object region.

[0017] Condition 2: Calculate the target response intensity map based on the highest resolution feature map in the multi-scale feature map, sort the spatial positions in the highest resolution feature map according to the target response intensity, and perform clustering on the spatial positions within the scale threshold to generate the minimum bounding rectangle that surrounds the connected components. Map the minimum bounding rectangle back to the UAV image frame coordinate space to obtain the blurred object region.

[0018] The fuzzy object regions obtained from conditions one and two are deduplicated to obtain the final set of fuzzy object regions.

[0019] Preferably, the extraction of multi-scale region features for each blurred object region includes:

[0020] Extract the bounding box of the blurred object region, and expand the bounding box by a preset multiple based on the center of the bounding box to generate a bounding box;

[0021] If the intersection-union ratio of two bounding boxes exceeds the merging threshold, the two bounding boxes will be spatially merged to form a new bounding box.

[0022] Based on the coordinates of the bounding box, for each blurred object region, feature blocks at corresponding locations are extracted from all corresponding resolution levels of the multi-scale feature map to form a multi-scale region feature set.

[0023] Preferably, the calculation of the weights corresponding to each discrete scale includes:

[0024] Take a predefined set of discrete scales, which contains several discrete scales;

[0025] Multi-scale regional features are fused and encoded into a fixed-dimensional scale strategy feature vector;

[0026] After the scale strategy feature vector passes through a multilayer perceptron structure, the weights corresponding to each discrete scale are output by the probability normalization function.

[0027] Preferably, the step of performing scale sampling on multi-scale regional features based on discrete scales to obtain preliminary enhanced regional features includes:

[0028] A soft selection strategy is adopted: for the regional features at each scale, scale sampling is performed at all discrete scales to obtain the scale sampling features corresponding to each discrete scale; using the weights corresponding to each discrete scale, the scale sampling features corresponding to each discrete scale are weighted and fused to obtain single-scale enhanced regional features; the single-scale enhanced regional features at all scales are combined to obtain multi-scale preliminary enhanced regional features.

[0029] Alternatively, a hard selection strategy can be adopted: based on the weights corresponding to each discrete scale, the discrete scale with the largest weight is selected as the master scale, and the regional features of each scale are scale-sampled using the master scale. The scale-sampled features of all scales are combined to obtain the preliminary enhanced regional features of multiple scales.

[0030] Preferably, the historical secondary enhancement region features are stored under the following conditions:

[0031] After the current UAV image frame is processed, the secondary enhancement region features corresponding to targets with confidence scores greater than the storage confidence score threshold and target sizes greater than the storage size threshold are selected and stored as historical secondary enhancement region features.

[0032] Preferably, the historical secondary enhancement region features that match the blurred object region include:

[0033] Historical secondary enhancement region features with timestamps within a range of several frames prior to the current frame are selected as candidate historical features;

[0034] Calculate the spatial overlap between the current blurred object region and the target spatial coordinates corresponding to the candidate historical features, and use the historical secondary enhanced region features with a spatial overlap greater than the matching threshold as the final filtering result.

[0035] Preferably, the weighted fusion of historical secondary enhancement region features and initial enhancement region features to obtain the current secondary enhancement region features includes:

[0036] Calculate the time decay weights corresponding to the features of each selected historical secondary enhancement region;

[0037] Based on the time decay weight, the features of each selected historical secondary enhancement region are weighted and fused to obtain the historical fused features;

[0038] By introducing a historical fusion coefficient, the historical fusion features and the current preliminary enhanced regional features are weighted and fused to obtain the secondary enhanced regional features corresponding to the current fuzzy object region.

[0039] Secondly, a small target detection device for unmanned aerial vehicles (UAVs) based on fuzzy object enhancement is provided, including a processor and a memory storing a number of computer instructions, wherein the computer instructions are executed by the processor to implement the steps of the small target detection method for UAVs based on fuzzy object enhancement.

[0040] Existing technologies lack differentiated enhancement mechanisms for small target detection on UAVs, and most existing UAV small target detection methods employ fixed input scale strategies, making it difficult to adapt to the drastic fluctuations in target scale with flight altitude and viewing angle in UAV aerial photography scenarios. This is especially true for ambiguous target regions with low confidence, small size, and insufficient feature representation, which are prone to missed or false detections. Therefore, this invention provides a UAV small target detection method and apparatus based on fuzzy object enhancement. It intelligently identifies regions requiring enhancement by determining fuzzy object regions and triggers the enhancement process only for these regions. Combining discrete scale adaptive processing and cross-frame feature compensation mechanisms, it performs targeted enhancement on fuzzy object regions to obtain high-precision and low-overhead small target detection results. Compared with existing technologies, this invention has the following advantages:

[0041] (1) By using the fuzzy object region determination and sparse enhancement scheduling mechanism, only the fuzzy object region is targeted for enhancement processing, avoiding uniform magnification of the entire image, significantly reducing the consumption of computing resources, and making it more suitable for airborne and edge computing environments;

[0042] (2) A discrete scale adaptive processing unit is introduced to dynamically adjust the scale mapping method according to the feature state of the target region of the fuzzy object, thereby improving the distinguishability of small-scale targets in the feature space and effectively reducing the false detection rate and the missed detection rate.

[0043] (3) Construct a cross-frame feature compensation mechanism based on short-term memory, use historical high-confidence target features to make up for the lack of features in the current frame, significantly enhance the robustness to interference such as motion blur and partial occlusion, make the detection results more continuous and reliable, and improve the practicality in complex scenarios. Attached Figure Description

[0044] Figure 1 A flowchart illustrating a method for detecting small targets in unmanned aerial vehicles (UAVs) based on fuzzy object enhancement provided by this invention;

[0045] Figure 2 This is a schematic diagram illustrating the principle of the scale-adaptive processing of this invention. Detailed Implementation

[0046] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to limit the invention.

[0048] To overcome the shortcomings of existing technologies, this embodiment provides a method for small target detection in unmanned aerial vehicles (UAVs) based on fuzzy object enhancement, such as... Figure 1 As shown, it includes the following steps:

[0049] (1) Basic feature extraction: The input UAV image frames are preprocessed and feature extracted to generate multi-scale feature maps, and the basic detection network outputs the preliminary detection results.

[0050] (1-1) Image preprocessing: First, the raw image frames captured by the UAV are acquired, and preprocessing operations are performed on the raw image frames. Preprocessing operations include, but are not limited to, image size normalization, pixel value standardization, noise suppression, and brightness and contrast adjustment, in order to reduce the impact of ambient light changes and imaging noise on the detection results.

[0051] (1-2) Backbone Feature Extraction: A lightweight convolutional neural network is used as the backbone network to extract features from the preprocessed image. Then, through FPN (Feature Pyramid Network), multi-scale feature maps are output. Lightweight convolutional neural networks include, but are not limited to, the MobileNet (Mobile Network) series, the CSPNet (Cross Stage Partial Network) series, and the EfficientNet (Efficient Network) series. In this embodiment, a lightweight variant of CSPNet, namely CSPDarknet (Cross Stage Partial Darknet), is used as the backbone network. The backbone network contains multiple downsampling stages, generating single-path deep features with different spatial resolutions. FPN receives the output of the backbone network, performs multi-scale fusion, and generates multi-scale feature maps. Low-level feature maps have higher spatial resolution but weaker semantic information, while high-level feature maps have lower spatial resolution but richer semantic information. Multi-scale feature maps are used to simultaneously represent the spatial structure and semantic information of large-scale and small-scale targets.

[0052] (1-3) Preliminary detection: Based on the multi-scale feature map, the potential targets in the image are preliminarily detected by the detection head of the basic detection network for each scale of the feature map, and the corresponding preliminary detection results are output.

[0053] Specifically, the detection head is a network structure capable of transforming feature maps into predicted data, including at least one of concatenated convolutional layers, fully connected layers, or self-attention mechanism layers. In this embodiment, the detection head adopts a convolution-based decoupled head network structure, including a 3×3 convolutional layer for feature fusion and a 1×1 convolutional layer for classification and regression prediction. The preliminary detection results include at least the spatial location of the candidate target, target size information, target detection category, and corresponding detection confidence, wherein the spatial location and target size information of the target constitute the target detection box.

[0054] (2) Blurred Object Region Determination: Based on the preliminary detection results and multi-scale feature maps, the blurred object region is determined for the candidate target region. In this embodiment, the blurred object refers to the target in the UAV image whose feature representation is insufficient due to factors such as small target scale, motion blur, partial occlusion, or lighting interference. It exhibits weak signal characteristics during the detection process, specifically manifested as quantifiable indicators such as low detection confidence, small spatial size, or weak feature response. Furthermore, the spatial bounding box range occupied by the blurred object in the feature space, that is, the set of its spatial location coordinates, is called the blurred object region (i.e., the weak signal region).

[0055] (2-1) Candidate Region Screening: Based on the preliminary detection results, candidate regions containing potential targets are screened out as the basic input for fuzzy object region determination, forming an effective candidate set. Specifically, low-quality detection results that are obviously background noise are filtered out. If the confidence level of a candidate target in the preliminary detection results is lower than the background noise filtering threshold, for example, lower than the set value of 0.01, it can be considered as background noise or interference. After removing the above low-quality detection results, the remaining candidate regions constitute the effective candidate set.

[0056] (2-2) Fuzzy object determination based on confidence and target size (condition 1): When the detection confidence of a candidate target is lower than the preset first threshold, the target is considered to be insufficiently represented under the current detection conditions, and the corresponding region is marked as a fuzzy object candidate region. At the same time, when the spatial size of the candidate target is also smaller than the preset second threshold, the fuzzy object candidate region is determined as the fuzzy object region to highlight the detection requirements of small-scale targets.

[0057] (2-3) Supplementary judgment based on feature response (condition 2): For blurry objects that may be missed in the preliminary detection stage, extract the highest resolution feature map from the obtained multi-scale feature map, and further analyze the target response intensity to explore potential blurry object regions.

[0058] In this embodiment, the target response intensity at each spatial location is calculated based on multi-channel feature saliency fusion. Specifically, let the highest resolution feature map in the multi-scale feature map be... ,in Indicates spatial location, To represent the feature channel index, the features of each channel are first aggregated to obtain the basic response map. :

[0059]

[0060] in, This represents the maximum value operation in the channel dimension, used to extract the most significant channel response at that spatial location.

[0061] Based on this, spatial context information is introduced to smooth or weight the base response map. In one embodiment, the standard deviation is used as... Two-dimensional Gaussian kernel Convolution operations are performed on the base response map to obtain the context-enhanced response map. :

[0062]

[0063] in, This represents a convolution operation used to fuse response information within a spatial neighborhood, thereby enhancing the response of a continuous target region and suppressing local noise.

[0064] Finally, the target response intensity map is obtained by weighted fusion of the basic response map and the context-enhanced response map. :

[0065]

[0066] in, and The weighting coefficients are non-negative and satisfy the following conditions: This is used to balance the contribution of local salient response and spatial context information to the final target response intensity.

[0067] It should be noted that the above-described method for calculating the target response intensity is merely an exemplary implementation, and the present invention is not limited to this specific form. In other optional embodiments, channel-weighted summation, attention mechanisms, local contrast measurements, or combinations thereof can also be used to process the high-resolution feature map to obtain the target response intensity map.

[0068] After obtaining the target response intensity map, it is globally sorted, and spatial locations with response values ​​within a preset scale threshold are selected as high-response feature points. These high-response feature points will then be used to construct potential blurred object regions. For example, when the scale threshold is set to the previous value... At that time, the response intensity ranking will be at the top of the entire map. Spatial regions are marked as high-response feature points, even if the spatial location of the point is not detected in the initial detection stage.

[0069] Next, clustering is performed on the high-response feature points. First, morphological operations (such as closing and dilation operations) are used to enhance the connectivity of discrete high-response feature points, so that adjacent points are merged into continuous high-response region blocks. Then, spatial connectivity analysis is performed on the high-response region blocks to remove small connected regions with an area smaller than a preset noise threshold, and the minimum bounding rectangle of the retained connected regions is generated to form candidate regions based on feature responses.

[0070] In practice, the ratio threshold is adaptively adjusted based on the number of blurred object regions confirmed in Condition 1, and a maximum upper limit is set for the number of blurred object regions to prevent too many regions from entering the subsequent enhancement process. Specifically, a base ratio is set (e.g., When the number of initially detected blurry object regions When less, increase The value is used to uncover more potential targets; the calculation formula can be set as an exponential decay function, such as... ,in and For adjustment coefficients, The maximum compensation coefficient is used for control. The upper limit, typically 10, is a good starting point. To control the attenuation rate coefficient, a typical value range is between 0.01 and 0.1. Simultaneously, an upper limit is set for the maximum number of blurred object regions, generally between 50 and 100, to avoid excessive noise regions slowing down inference. Subsequently, based on the spatial correspondence between the multi-scale feature maps and the original image, the candidate regions based on feature responses are mapped back to the original image coordinate space, generating corresponding blurred object regions (corresponding to the minimum bounding rectangle, which contains one or more candidate targets from the preliminary detection results), for further detection or confirmation processing.

[0071] (2-4) Fuzzy Object Region Confirmation: Based on the above judgment results, regions that meet any of the following conditions are confirmed as the fuzzy object region set: Condition 1, the confidence of the candidate target is lower than the first threshold and the target size is smaller than the second threshold; Condition 2, it is located in front of the feature response map. The high-response region is defined as the region with the highest response rate. The remaining regions constitute the set of ordinary regions. Considering that the same fuzzy target may be captured by two mechanisms simultaneously, there may be cases where the regions generated by the two conditions overlap in spatial location, requiring deduplication processing when merging. Specifically, the intersection-union ratio (IUGR) between the two sets of regions is calculated. If the IUGR of a supplementary region generated in step (2-3) and an existing region generated in step (2-2) is greater than a preset overlap threshold, such as 0.5, the supplementary region is discarded; if there is no overlap or the overlap is extremely low, it is added to the set as a new fuzzy object region. In this embodiment, after obtaining the region determination results of the two conditions, they are merged to form the final fuzzy object region, reducing the influence of the background region.

[0072] (3) Sparse augmentation scheduling: Based on the determination results of the fuzzy object region, implement differentiated processing strategies to avoid computational waste caused by global augmentation.

[0073] (3-1) Sparse scheduling decision: Assign region type labels to UAV image frames. Region types include blurred object regions and ordinary regions. Based on the blurred object regions obtained in step (2), regions outside the blurred object regions are labeled as ordinary regions, and no further splitting is performed inside the ordinary regions. If a clearly defined target and a blurred target in the scene occlude each other or overlap spatially, they will appear as two independent candidate regions with spatial overlap at the algorithm data level. During the scheduling phase, the two do not interfere with each other and will be processed independently according to their respective type labels in the subsequent ordinary region pass-through process or blurred object region enhancement process. The enhancement process only applies to the blurred object regions to avoid performing uniform enhancement processing on the entire image or all candidate regions.

[0074] (3-2) Normal region pass-through: For each candidate target in the normal region set, its detection result is directly retained to the final output queue without any enhancement processing, which significantly reduces the computational overhead.

[0075] (3-3) Blurred Object Region Enhancement Trigger: For each blurred object region in the set of blurred object regions, calculate the spatial coordinates used for subsequent feature block extraction. Specifically, based on the center of the bounding box, expand the bounding box into a square region larger than 1 times its size, generating bounding box coordinates. Region expansion is used to introduce contextual information, ensure target integrity, and facilitate the model to use contextual information to help determine the target type. Here, the bounding box refers to the target detection box of the candidate target confirmed in step (2-2) and the minimum bounding rectangle confirmed in step (2-3).

[0076] To avoid unlimited expansion of the region due to dense overlap of targets, a merging mechanism based on overlap is designed here. If there are adjacent or overlapping blurred object regions in space, the expanded bounding boxes will only be spatially merged to form a cluster if the intersection-union ratio of their bounding boxes exceeds a preset merging threshold. Independent regions that are spatially close but do not reach the merging threshold will not be merged. The generated bounding box coordinates are then sent to subsequent steps (4) and (5) for scale adaptation and cross-frame feature compensation. At the same time, a unique spatiotemporal identifier is assigned to each blurred object region. This identifier contains the current frame number and the spatial center coordinates of the region, which is used for cross-frame feature matching in subsequent step (5).

[0077] (4) Scale-adaptive processing: such as Figure 2 As shown, the scale of the feature representation is dynamically adjusted according to the size characteristics of the blurred object region to enhance the spatial resolution of small targets.

[0078] (4-1) Region feature construction: Based on the bounding box coordinates passed in step (3-3), for each fuzzy object region, extract the corresponding feature blocks from all corresponding resolution levels in the multi-scale feature map to form a multi-scale region feature set. This set contains the feature representation of the same spatial region under different resolution levels.

[0079] (4-2) Definition of Discrete Scale Set: A predefined discrete scale set, containing multiple scale factors, is used for super-resolution resampling of blurred object regions in feature maps, covering the size range of common small targets in UAV scenarios. Assume the total number of discrete scales is... ,For example The discrete scale set can include five discrete scales: the original scale, 1.5x magnification, 2x magnification, 3x magnification, and 5x magnification, avoiding the computational overhead of continuous scale search.

[0080] (4-3) Scale-based feature encoding: The multi-scale region feature set obtained in step (4-1) is fused and encoded into a fixed-dimensional scale-based feature vector, which serves as the input to the subsequent scale-based network. In practice, each scale feature block (region feature) can first undergo global average pooling to obtain low-dimensional features, and then these low-dimensional features are concatenated and input into the scale-based network. Assume the number of channels in the feature block is... First, perform global average pooling on each feature block separately, then compress them uniformly to a dimension of . The low-dimensional feature vectors are then concatenated along the channel dimension and input into the scale policy network. In scenarios with sufficient computing power, the scale policy network can further include one or more lightweight convolutional layers, such as depthwise separable convolutions, before the global average pooling layer, to capture the fine-grained spatial topology and micro-texture within the feature blocks before pooling, thereby outputting more accurate scale-aware features.

[0081] (4-4) Scale Weight Prediction: Based on the feature encoding results of the scale policy, the encoded data is input into the scale policy network to predict the weight distribution corresponding to each discrete scale level, which reflects the degree of adaptation of different scales to the current region. The scale policy network in this embodiment includes an MLP (Multi-Layer Perceptron) structure: the first layer serves as a dimensionality reduction bottleneck layer, compressing the input and activating it through the ReLU (Rectified Linear Unit) function; the second layer maps the features to the output dimension. The final output of the scale policy network Each original predicted value (logits) is then converted into a scale weight distribution that satisfies the normalization constraint using the softmax function. One scale weight distribution corresponds to one fuzzy object region.

[0082] In one embodiment, a soft selection strategy is used to distribute the scale weights for weighted fusion in subsequent scale remapping. In another embodiment, a hard selection strategy is used to select the scale with the largest weight as the master scale label. This feature block then only undergoes remapping operations at the master scale and is directly output without requiring calculations at other scales, thus further reducing computational overhead during the inference stage. In this embodiment, a soft selection strategy is used for subsequent weighted fusion.

[0083] (4-5) Neighborhood scale consistency constraint: To avoid discontinuity in detection results due to large differences in scale selection between adjacent fuzzy object regions, a neighborhood consistency mechanism is introduced.

[0084] For soft selection strategies, the scale weight distributions of adjacent blurred object regions are compared. This can be achieved by calculating the Wasserstein (bulldozer) distance between the two regions. If the distributions differ significantly, a penalty is applied to the network during training to encourage it to learn spatial isomorphic priors. Specifically, this distance metric is multiplied by a preset weight coefficient and added as a loss term to the network's total loss function. Through backpropagation, the network learns three-dimensional spatial isomorphic priors, adaptively smoothing the scale weights of adjacent regions.

[0085] For a hard selection strategy, the scale selection difference between adjacent fuzzy object regions needs to be constrained to no more than one level. The hard selection strategy does not impose a training-period penalty, but instead uses it as a smoothing post-processing rule during the inference phase. Specifically, during the inference phase, regions that violate this constraint are scaled, forcing the main scale level span between adjacent fuzzy object regions to be no more than one level (e.g., one region cannot choose 1.5 times, and the adjacent one cannot jump directly to 5 times). If a pair of regions violates this constraint, the scale selection result of the region with higher scale weight reset confidence in step (4-4) is prioritized, meaning that the feature of that region is more reliable. The main scale level of the region with the lower value is aligned with it, adjusted until the difference is no more than one level. For example, if among adjacent fuzzy object regions, one region A has a main scale label of 1.5 times and its main scale weight is 0.8; another region B has a main scale label of 5 times and its main scale weight is 0.5. In this case, the main scale weight reset confidence of region A is higher, so the main scale label of region B is adjusted to 2 times.

[0086] (4-6) Scale Remapping: Based on the scale weight distribution, a weighted scale remapping operation is performed on the regional features to generate enhanced regional features, i.e., preliminary enhancement of regional features. Specifically, feature blocks extracted from the multi-scale regional feature set in step (4-1) are selected, and one level of feature block is denoted as the feature map. ,in and This represents the original spatial dimensions of the feature block. This represents the number of channels. In scenarios with sufficient computing power, the following scale remapping operation is performed for each level of the feature block; in lightweight scenarios with insufficient computing power, the scale remapping operation is only performed on the shallowest feature map with the highest resolution that is sensitive to small targets, and the operation is performed according to the actual settings.

[0087] Take a predefined set of discrete scales , For the first Discrete scale For each discrete scale, a corresponding scale sampling operator is constructed. This scale sampling operator is used to resample regional features under different sampling densities or receptive field ranges, while maintaining the output feature size. A preferred implementation of the scaling operator is as follows: first, the original features are processed by bilinear interpolation. Magnification in space times; then through a layer The convolutional layer extracts microscopic textures and high-frequency details at the magnified resolution; finally, average pooling is used to downsample and map them back to the original size. The scale sampling operator is not limited to a specific implementation; as long as the scale adjustment can be achieved by changing the feature sampling density or the receptive field range while keeping the output feature size unchanged, it should fall within the protection scope of this invention.

[0088] Take the scale weight distribution corresponding to each discrete scale The weights satisfy normalization constraints. Based on the scale weight distribution, the sampled features at different scales are weighted and fused to obtain the enhanced region features:

[0089]

[0090] in, For enhanced regional features (preliminary enhanced regional features). For the first Each weight, For the first Scale sampling operators corresponding to discrete scales, based on discrete scales The calculation is performed. In this embodiment, the number of layers in the multi-scale feature map is not related to the number of discrete scales, and each feature in the multi-scale region feature set can be processed by different scale sampling operators. In this embodiment, the features of each scale are first processed by all discrete scales, and then merged using a soft selection strategy (in other embodiments, a hard selection strategy can be used to select the best multiple) to obtain the merged features corresponding to each scale. Then, the merged features of all scales are further combined to obtain the final multi-scale preliminary enhanced region features.

[0091] By using the above-mentioned scale remapping method, without changing the output size of the regional features, the preliminary enhanced regional features are simultaneously integrated with target information under the multi-scale receptive field, thereby improving the discriminability of small targets with blurred objects in the feature space.

[0092] (5) Cross-frame feature compensation: The cross-frame feature compensation step is used to improve the stability of blurred object target detection by taking advantage of the continuity of the target in the time dimension. The features of several past frames are merged with the features of the current frame to supplement the missing features.

[0093] (5-1) Short-term memory unit: Construct a circular buffer with a capacity of several frames as a short-term memory unit. After each frame is processed, that is, after step (6) is completed, store the final features of the high-confidence targets in this frame whose confidence (the confidence of the candidate targets in the ordinary region in the preliminary detection and the confidence of the candidate targets in the blurred object region in the fine detection) is greater than the high confidence threshold and whose size is greater than the size threshold into this unit (for the blurred object region, store its secondary enhanced region features; for the ordinary region, directly store the corresponding initial region features in step (1)). The stored features are the enhanced features obtained after the scale adaptation processing in step (4) and the cross-frame feature compensation in step (5). The stored content includes: normalized feature vector (for the blurred region, it is obtained by normalizing the secondary enhanced region features output in step (5); for the ordinary region, it is obtained by normalizing the corresponding initial region features), bounding box position, scale information and timestamp. In order to control the storage overhead, the feature vector can also be reduced in dimensionality.

[0094] (5-2) Historical High-Confidence Target Screening: When processing the current frame, candidate historical features with timestamps within a range of several frames prior to the current frame are screened from the short-time memory unit. To improve matching accuracy, the spatial overlap between the current blurred object region and the candidate historical targets is calculated (the intersection-union ratio of the spatial coordinates of the bounding box or leading box). Only candidate historical features with an overlap greater than the matching threshold are retained to form the candidate set. Only high-confidence target features are stored to avoid introducing noisy or falsely detected targets into the cross-frame compensation process, ensuring the stability and reliability of historical features.

[0095] (5-3) Current frame blurred object region: Obtain the enhanced region features corresponding to the current frame blurred object region after the scale adaptation processing in step (4), that is, the preliminary enhanced region features. Each feature block is accompanied by its spatial location information and spatiotemporal identifier.

[0096] (5-4) Calculation of Time Decay Weight: For each preliminary enhanced region feature corresponding to the current blurred object region, calculate its time decay weight relative to the matching historical features. This weight is calculated using an exponential decay function, with the decay coefficient controlling the decay rate of the historical feature contribution. Specifically, the formula for calculating the time decay weight is: ,here It is the first The difference between a historical feature and the current frame, i.e., the number of frames between them. This is the attenuation coefficient, typically set between 0.1 and 1.0. To ensure weight normalization, the weights of all historical features corresponding to the same fuzzy object region are normalized.

[0097] (5-5) Cross-frame feature fusion: For each current blurred object region, if a matching historical feature set exists, weighted fusion is performed to obtain secondary enhanced region features. The fusion formula is as follows:

[0098]

[0099]

[0100] in, This refers to the enhanced features after fusion (secondary enhanced region features). This provides the initial enhanced region features corresponding to the blurred object region in the current frame. For the first A historical characteristic, For the first Each normalized time decay weight, and , The historical feature fusion coefficient (ranging from 0 to 1) controls the maximum contribution ratio of historical features, ensuring that the current frame features dominate. If no matching historical features exist, the enhanced features degenerate into the current frame features themselves. The secondary enhanced region features retain the immediate information of the current frame while incorporating stable features from historical frames, effectively mitigating the effects of motion blur and momentary occlusion.

[0101] (6) Fine detection output: The final detection result is generated based on the enhanced features after fusion.

[0102] (6-1) Enhanced Region Detection: The enhanced region features output from step (5) are input into the fine detection module, and the fine detection results are output. In this embodiment, the fine detection module includes a shared convolutional layer for feature integration, and two parallel sub-network branches, including a classification branch and a bounding box regression branch. The classification branch is used to output the probability score and confidence of the target belonging to each preset category in the region, while the bounding box regression branch outputs the bounding box position of each target in the region. This embodiment adopts a decoupled head network structure without anchor boxes. The classification branch includes multiple 3×3 convolutional layers for feature fusion and extraction, and finally connects to a 1×1 convolutional layer for classification prediction, and outputs a probability score vector through an activation function. The bounding box regression branch also includes multiple 3×3 convolutional layers for location feature extraction, and finally connects to a 1×1 convolutional layer for regression prediction, and finally determines the final bounding box coordinates of each target.

[0103] (6-2) Merging Detection Results: The fuzzy object region detection results output by the fine detection are merged with the ordinary region detection results retained in step (3-2) to form a complete set of detection results. At the same time, in order to avoid duplicate detection, non-maximum suppression can be applied to the detection boxes with an overlap greater than the threshold to retain results with higher confidence.

[0104] (6-3) Output Result Generation: Outputs the final detection result of the current UAV image frame. The output result can be directly used for downstream tasks, such as target tracking, behavior analysis, or alarm triggering.

[0105] (7) Training setup: To ensure the effective collaboration between the above-mentioned scale policy network and cross-frame feature compensation network and the backbone network, the lightweight convolutional neural network, detection head, scale policy network and cross-frame feature compensation network in this embodiment form a unified framework for end-to-end joint training during the training phase.

[0106] In this embodiment, the training data uses VisDrone2019-VID (VisDrone2019 Video ObjectDetection Dataset), which contains object detection samples in various complex scenes, covering 10 types of targets including pedestrians and vehicles. During data preprocessing, the input images are uniformly scaled to 640×640 pixels and data augmentation is performed, including random flipping, scale perturbation, and color space perturbation, to improve the model's generalization ability. For training parameter settings, a mini-batch stochastic gradient descent (SGD) optimizer is used to optimize the model, with an initial learning rate of 0.01, a momentum coefficient of 0.95, and a weight decay coefficient of 0.0005. Nesterov momentum is used to accelerate convergence. During training, the batch size is set to 16, the maximum number of training epochs is set to 300, and a warm-up strategy is used in the first 10 epochs to gradually increase the learning rate to improve stability in the early stages of training.

[0107] The above network modules use a unified loss function Joint optimization is performed under constraints, and the formula is as follows:

[0108]

[0109] in, The classification loss is preferred (preferably cross-entropy loss). The bounding box intersection-union ratio loss is used (preferably the full intersection-union ratio loss). For the distribution focus loss, This represents the neighborhood-scale consistency loss. , , as well as These are the weighting coefficients for each loss term. In this embodiment, The typical value range is from 1.0 to 10.0 (e.g., preferably 7.5). and The typical value ranges are 0.1 to 2.0 (for example, preferably 0.5 and 0.375 respectively). The typical value range is set to 0.01 to 1.0 (e.g., preferably 0.1). , , All three factors work together to optimize the category probability distribution and spatial boundary positioning of the final target detection. This is a penalty term for controlling the neighborhood scale consistency constraint in step (4-5), which prompts the network to learn spatial isomorphic priors and forces adjacent fuzzy targets to maintain smooth consistency in scale selection.

[0110] It should be noted that the scale policy network in step (4) (except for) Without local smoothing constraints, the cross-frame feature compensation network in step (5) does not have independent explicit loss terms. The two are calculated through the fused feature... , , The errors from all three factors participate in the optimization, and backpropagation is used to implicitly adjust the weights in the network.

[0111] By constraining the unified loss function mentioned above, the scale-adaptive processing and cross-frame feature compensation mechanism can work synergistically with the backbone feature extraction process to improve the detection performance of fuzzy small targets.

[0112] To verify the effectiveness of the proposed method, the model was trained and validated on the Visdrone2019-VID dataset, and experimental results were obtained, demonstrating the model's detection performance and further verifying its detection capabilities in different scenarios. The model of this invention exhibits strong robustness in complex environments across scenes and lighting conditions. Specific detection results are shown in Table 1, where AP@0.5 represents the average accuracy when the Intersection over Union (IoU) threshold is 0.5, and AP@0.75 represents the average accuracy when the IoU threshold is 0.75.

[0113] In daytime scenarios, the method of this invention can not only detect large nearby targets such as cars, but also successfully detect blurry small-scale targets such as densely packed bicycles and crowds in the distance, effectively suppressing false alarms caused by over-magnification and false negatives caused by under-magnification. Nighttime scenarios demonstrate the anti-interference performance of the method in extremely challenging low-light and strong light-shadow interference environments. Despite poor lighting conditions and severe glare, the model still stably detects densely distributed crowds and vehicles thanks to its spatiotemporal feature compensation mechanism.

[0114] Table 1 Detection results for each category

[0115]

[0116] As shown in Table 1, this model demonstrates excellent overall detection performance in a UAV perspective detection task encompassing 10 complex target categories. Specifically, the model performs best for vehicles with obvious rigid body features, achieving an AP@0.5 of 63.3%. Notably, thanks to the scale-adaptive remapping and cross-frame compensation mechanisms designed in this invention, the model maintains a high detection rate even when facing small target categories prone to feature loss, such as pedestrians and bicycles, with AP@0.5 reaching 53.3% and 46.9% respectively, demonstrating the feature extraction capability of the method for blurry small targets.

[0117] The method of this invention can be deployed on an airborne computing platform for unmanned aerial vehicles (UAVs) or on a ground station server. In resource-constrained scenarios, the scale-adaptive processing can be further simplified, reducing the number of discrete scales or lowering the complexity of the scale policy network to meet strict single-frame processing latency requirements.

[0118] In another embodiment, the present invention provides a small target detection device for unmanned aerial vehicles (UAVs) based on fuzzy object enhancement, including a processor and a memory storing a plurality of computer instructions, wherein the computer instructions are executed by the processor to implement the steps of the small target detection method for UAVs based on fuzzy object enhancement.

[0119] The memory and processor are electrically connected directly or indirectly to enable data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines. The memory stores a computer program that can run on the processor, and the processor implements the method of the present invention by running the computer program stored in the memory.

[0120] The memory can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), and Electrically Erasable Programmable Read-Only Memory (EEPROM). The memory stores the program, and the processor executes the program after receiving the execution instruction.

[0121] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0122] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A method for detecting small targets on a UAV based on fuzzy object enhancement, characterized in that, Includes the following steps: The input UAV image frames are preprocessed and feature extracted to generate multi-scale feature maps, and preliminary target detection results of each scale feature map are output through preliminary target detection. Based on the multi-scale feature maps and preliminary target detection results, and using confidence level, target size, and target response intensity, the blurred object region is extracted, including: Condition 1: If the detection confidence of the candidate target is lower than the first threshold and the spatial size of the candidate target is smaller than the second threshold, then the candidate target is marked as a fuzzy object and the spatial location of the fuzzy object is taken as the fuzzy object region. Condition 2: Calculate the target response intensity map based on the highest resolution feature map in the multi-scale feature map, sort the spatial positions in the highest resolution feature map according to the target response intensity, and perform clustering on the spatial positions within the scale threshold to generate the minimum bounding rectangle that surrounds the connected components. Map the minimum bounding rectangle back to the UAV image frame coordinate space to obtain the blurred object region. The fuzzy object regions obtained from conditions one and two are deduplicated to obtain the final set of fuzzy object regions. Based on the extracted blurred object region, a region type label is assigned to the UAV image frame, which is either a blurred object region or a normal region. Multi-scale region features of each blurred object region are extracted, and the weights corresponding to each discrete scale are calculated. Based on the discrete scale, the multi-scale region features are scale-sampled to obtain preliminary enhanced region features. The calculation of the weights corresponding to each discrete scale includes: taking a predefined set of discrete scales, which contains several discrete scales; fusing the multi-scale region features and encoding them into a fixed-dimensional scale strategy feature vector; after the scale strategy feature vector is processed through a multilayer perceptron structure, the weights corresponding to each discrete scale are output by a probability normalization function. Filter historical secondary enhancement region features that match the blurred object region, and perform weighted fusion of the historical secondary enhancement region features and the initial enhancement region features to obtain the current secondary enhancement region features; wherein, the weighted fusion of the historical secondary enhancement region features and the initial enhancement region features to obtain the current secondary enhancement region features includes: Calculate the time decay weights corresponding to the features of each selected historical secondary enhancement region; Based on the time decay weight, the features of each selected historical secondary enhancement region are weighted and fused to obtain the historical fused features; By introducing a historical fusion coefficient, the historical fusion features and the current preliminary enhanced regional features are weighted and fused to obtain the secondary enhanced regional features corresponding to the current blurred object region. Based on the current secondary enhancement region features, fine target detection is performed. The fine target detection results are then merged with the candidate targets from the preliminary target detection results in the ordinary region to form a complete small target detection result.

2. The UAV small target detection method based on fuzzy object enhancement according to claim 1, characterized in that, The extraction of multi-scale region features for each blurred object region includes: Extract the bounding box of the blurred object region, and expand the bounding box by a preset multiple based on the center of the bounding box to generate a bounding box; If the intersection-union ratio of two bounding boxes exceeds the merging threshold, the two bounding boxes will be spatially merged to form a new bounding box. Based on the coordinates of the bounding box, for each blurred object region, feature blocks at corresponding locations are extracted from all corresponding resolution levels of the multi-scale feature map to form a multi-scale region feature set.

3. The UAV small target detection method based on fuzzy object enhancement according to claim 1, characterized in that, The process of scale sampling of multi-scale regional features based on discrete scales to obtain preliminary enhanced regional features includes: A soft selection strategy is adopted: for the regional features at each scale, scale sampling is performed at all discrete scales to obtain the scale sampling features corresponding to each discrete scale; using the weights corresponding to each discrete scale, the scale sampling features corresponding to each discrete scale are weighted and fused to obtain single-scale enhanced regional features; the single-scale enhanced regional features at all scales are combined to obtain multi-scale preliminary enhanced regional features. Alternatively, a hard selection strategy can be adopted: based on the weights corresponding to each discrete scale, the discrete scale with the largest weight is selected as the master scale, and the regional features of each scale are scale-sampled using the master scale. The scale-sampled features of all scales are combined to obtain the preliminary enhanced regional features of multiple scales.

4. The UAV small target detection method based on fuzzy object enhancement according to claim 1, characterized in that, The historical secondary enhancement region features are stored under the following conditions: After the current UAV image frame is processed, the secondary enhancement region features corresponding to targets with confidence scores greater than the storage confidence score threshold and target sizes greater than the storage size threshold are selected and stored as historical secondary enhancement region features.

5. The UAV small target detection method based on fuzzy object enhancement according to claim 1, characterized in that, The historical secondary enhanced region features for filtering and matching the blurred object region include: Historical secondary enhancement region features with timestamps within a range of several frames prior to the current frame are selected as candidate historical features; Calculate the spatial overlap between the current blurred object region and the target spatial coordinates corresponding to the candidate historical features, and use the historical secondary enhanced region features with a spatial overlap greater than the matching threshold as the final filtering result.

6. A small target detection device for unmanned aerial vehicles based on fuzzy object enhancement, comprising a processor and a memory storing a plurality of computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the steps of the UAV small target detection method based on fuzzy object enhancement as described in any one of claims 1 to 5.

Images