Method, device and equipment for detecting small target in image and storage medium

Abstract

This application provides a method, apparatus, device, and storage medium for detecting small targets in images. The method extracts image features from the image to be detected, then obtains frequency domain detail features representing the frequency details of the small target and spatial attention features representing the spatial structure based on the image features. The spatial structure information is used to enhance the frequency details, and the two are fused together to perform spatial recalibration and multi-scale aggregation, ultimately completing the classification and localization of small targets in the image. This method can improve the representation ability of edge details, structural morphology, and scale changes of small targets in complex backgrounds, thereby improving detection accuracy and robustness.

Description

Technical Field

[0001] This application relates to the field of image processing and target detection, and in particular to a method, apparatus, device and storage medium for detecting small targets in an image. Background Technology

[0002] Target detection based on UAV aerial imagery has been widely applied in key areas such as precision agriculture, geological surveying, traffic monitoring, and disaster emergency response. In high-altitude flight scenarios, UAVs typically acquire large-scale aerial images through rapid movement.

[0003] However, in aerial images captured by drones, the rapid movement of the drones can cause image blurring, especially at the edges of small targets, where the blurring effect is significant, making it difficult for existing image recognition algorithms to accurately extract features. Furthermore, due to the arbitrariness of the drone's perspective, the same type of target may exhibit highly diverse changes under different aerial angles, making it difficult for existing models to identify shape differences in small targets and affecting the positioning accuracy of small targets.

[0004] Therefore, how to improve the ability of small target detection to adapt to blurred edges, shape changes and complex backgrounds in drone aerial photography scenarios, so as to achieve higher accuracy and robustness, has become a technical problem that needs to be solved. Summary of the Invention

[0005] This application provides a method, apparatus, device, and storage medium for detecting small targets in images. It performs targeted enhancement and collaborative representation of image features, improves feature expression capabilities while taking into account both the detailed and structural information of small targets, and further enhances the adaptability of the detection process to complex backgrounds and multi-scale targets, thereby improving the accuracy and robustness of small target detection.

[0006] In a first aspect, embodiments of this application provide a method for detecting small targets in an image, comprising:

[0007] Extract image features from the image to be detected;

[0008] The image features are subjected to bi-branch feature extraction processing to obtain frequency domain detail features and spatial attention features, respectively. The frequency domain detail features are used to indicate the frequency detail information of small targets in the image to be detected, and the spatial attention features are used to indicate the spatial structure information of the small targets.

[0009] Based on the spatial attention features, the frequency domain detail features are reweighted to obtain frequency domain weighted features;

[0010] By fusing the frequency domain weighted features and the spatial attention features, a spatial frequency fusion feature is obtained;

[0011] Spatial recalibration is performed on the spatial frequency fusion features to obtain spatial calibration features;

[0012] The spatial calibration features are subjected to multi-scale feature aggregation processing to obtain aggregated features, which are used to enhance the feature representation of the small target.

[0013] The aggregated features are classified and regression predicted to obtain the small target detection results of the image to be detected.

[0014] In one possible implementation, the bi-branch feature extraction process on the image features to obtain frequency domain detail features and spatial attention features includes:

[0015] The frequency domain features corresponding to the small target are obtained by performing frequency domain transformation on the image features;

[0016] The image features are processed by convolution and global average pooling to capture the spatial structure information of small targets and obtain spatial attention features.

[0017] In one possible implementation, fusing the frequency domain weighted features and the spatial attention features to obtain spatial frequency fusion features includes:

[0018] The frequency domain weighted features are represented by multi-scale convolution to obtain spatial attention weights;

[0019] Based on the spatial attention weight, the frequency domain weighted features and the spatial attention features are weighted to obtain spatial frequency domain weighted features;

[0020] The spatial frequency domain weighted features are multiplied element-wise with the original spatial domain features to generate spatial frequency fusion features.

[0021] In one possible implementation, the step of spatially recalibrating the spatial frequency fusion features to obtain spatial calibration features includes:

[0022] Multi-branch spatial dimension perception feature extraction is performed on the spatial frequency fusion features to generate multiple sets of spatial dimension features, including channel height features and channel width features.

[0023] The multiple sets of spatial dimensional features are dynamically calibrated to generate spatial calibration features that adapt to changes in aspect ratio.

[0024] In one possible implementation, the multi-scale feature aggregation processing of the spatial calibration features to obtain aggregated features includes:

[0025] Extract multi-level features from the spatial calibration features;

[0026] For any hierarchical feature, a bidirectional sampling and fusion process is performed on the hierarchical feature to obtain a refined feature;

[0027] The refined features corresponding to multiple hierarchical features are aggregated to obtain aggregated features.

[0028] In one possible implementation, the bidirectional sampling and fusion processing of the hierarchical features to obtain refined features includes:

[0029] Obtain the upsampled features of the aforementioned hierarchical features;

[0030] The upsampled features are downsampled to obtain the downsampled features;

[0031] The upsampled features, the downsampled features, and the hierarchical features are concatenated and fused to obtain hybrid features;

[0032] The hybrid features are recombined and information is extracted to obtain the refined features corresponding to the hierarchical features.

[0033] In one possible implementation, classifying and regressing the aggregated features to obtain the small target detection result of the image to be detected includes:

[0034] The aggregated features are classified and regression branches are calculated to obtain multiple prediction results, including: predicted bounding boxes and the target category probability, bounding box offset and target existence probability corresponding to the predicted bounding boxes;

[0035] The bounding box offset is mapped to obtain the actual bounding box coordinates mapped back to the image to be detected.

[0036] Based on the target category probability and the target existence probability, the confidence level of each predicted bounding box is determined;

[0037] Based on the confidence level and the confidence threshold, a set of candidate bounding boxes is determined;

[0038] The small target detection result is determined based on the intersection-union ratio of the candidate bounding box set and the actual bounding box coordinates. The small target detection result is used to determine the location and category of the small target.

[0039] Secondly, embodiments of this application provide a device for detecting small targets in an image, comprising:

[0040] The acquisition module is used to extract image features from the image to be detected;

[0041] The processing module is used to perform bi-branch feature extraction processing on the image features to obtain frequency domain detail features and spatial attention features, respectively.

[0042] The processing module is used to reweight the frequency domain detail features based on the spatial attention features to obtain frequency domain weighted features;

[0043] The fusion module is used to fuse the frequency domain weighted features and the spatial attention features to obtain spatial frequency fusion features;

[0044] The calibration module is used to perform spatial recalibration on the spatial frequency fusion features to obtain spatial calibration features;

[0045] The processing module is used to perform multi-scale feature aggregation processing on the spatial calibration features to obtain aggregated features, which are used to enhance the feature representation of the small target.

[0046] The processing module is used to classify and regress the aggregated features to obtain the small target detection results of the image to be detected.

[0047] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor;

[0048] The memory stores computer-executed instructions;

[0049] The processor executes computer execution instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0050] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0051] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0052] The method, apparatus, device, and storage medium for detecting small targets in images provided in this application extract image features from the image to be detected and perform bi-branch feature extraction on the image features to obtain frequency domain detail features and spatial attention features respectively, which can simultaneously characterize the frequency detail information and spatial structure information of small targets. By reweighting the frequency domain detail features based on spatial attention features, fusing the frequency domain weighted features and spatial attention features, and performing spatial recalibration on the fused features, it can highlight the effective response related to small targets and suppress complex background clutter interference, thereby improving the ability to distinguish targets with blurred edges and changing shapes. By performing multi-scale feature aggregation on the spatial calibration features and performing classification and regression prediction based on the aggregated features, it can enhance the feature representation of small targets and the detection adaptability at different scales, thereby improving the accuracy and robustness of small target detection in UAV aerial photography scenarios. Attached Figure Description

[0053] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0054] Figure 1 A schematic diagram illustrating a scenario for the method of detecting small targets in images provided in this application;

[0055] Figure 2 Flowchart of the method for detecting small targets in images provided in this application Figure 1 ;

[0056] Figure 3 Flowchart of the method for detecting small targets in images provided in this application Figure 2 ;

[0057] Figure 4 Flowchart of the method for detecting small targets in images provided in this application Figure 3 ;

[0058] Figure 5 Flowchart of the method for detecting small targets in images provided in this application Figure 4 ;

[0059] Figure 6 Flowchart of the method for detecting small targets in images provided in this application Figure 5 ;

[0060] Figure 7 A schematic diagram of the structure of the device for detecting small targets in images provided in this application;

[0061] Figure 8 A schematic diagram of the structure of the electronic device provided in this application.

[0062] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0063] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application.

[0064] Image-based target recognition technology can automatically locate and identify specific objects in complex scenes, offering advantages such as non-contact operation, high efficiency, and batch processing capabilities. It has been widely applied in fields such as security monitoring, agricultural inspection, and traffic management. In recent years, with the rapid development of drone technology, drones, characterized by low cost, high maneuverability, and rapid deployment, can acquire large-scale, high-resolution image data from an aerial perspective, effectively overcoming the limitations of existing fixed cameras in terms of limited field of view and insufficient coverage. Therefore, target recognition methods based on drone aerial images have gradually become a research and application hotspot.

[0065] However, in actual deployment, the targets to be detected often occupy only a very small pixel area in the image. Furthermore, factors such as drone flight altitude, speed, viewpoint changes, lighting fluctuations, occlusion, vegetation texture, building shadows, and ground reflections can easily degrade the edge information, contour structure, and local texture of small targets in the image. Especially when drones rapidly fly over large fields of view to acquire images, the same target may exhibit blurring, stretching, tilting, and drastic scale changes in consecutive frames. This necessitates that the detection system not only possess sensitivity to subtle details but also maintain stable structural awareness and multi-scale representation capabilities in complex backgrounds. Therefore, such scenarios place high demands on the accuracy, robustness, and adaptability of the detection method.

[0066] In existing technologies, small target detection in UAV images often employs single-stage or two-stage target detection algorithms based on deep learning as the basic framework. These algorithms combine a backbone network to extract image features, and then use feature pyramids or path aggregation structures to achieve multi-scale information fusion, finally completing classification and bounding box regression prediction. While these methods can achieve good detection results for medium to large targets, their limitations become increasingly apparent under UAV aerial photography conditions.

[0067] First, small targets account for a very low percentage of pixels in high-altitude imaging, and the pose changes and rapid movements of UAV platforms can easily cause edge blurring. Most existing solutions focus on spatial domain texture enhancement or attention weighting, which do not adequately mine frequency information that can better reflect boundaries and details in small targets. This makes it difficult to effectively distinguish between target outlines and background textures, especially in complex backgrounds such as vegetation, roof textures, road markings, and rippled water surfaces, which can easily lead to false detections and missed detections.

[0068] Secondly, existing detection methods have limited adaptability to changes in target shape. The aspect ratio, local structure, and even contour distribution of the same type of target will change significantly under top, oblique, and side views. Existing spatial feature extraction methods often cannot simultaneously take into account the frequency details and spatial structure of the target, resulting in unstable feature representation and thus affecting the consistency of classification and localization.

[0069] Furthermore, while common multi-scale fusion paths can integrate features from different levels, high-resolution shallow information is easily weakened during the step-by-step transmission process. The edges, corners, and local structural details that small targets depend on are difficult to fully preserve in the subsequent prediction stage, resulting in the detection head lacking sufficient discrimination criteria when facing complex scenes.

[0070] It is evident that existing technologies for small target detection in UAV aerial photography still suffer from shortcomings, including insufficient perception of blurred edges, weak suppression of complex backgrounds, inadequate adaptation to shape changes, and limited ability to aggregate multi-scale details. These limitations make it difficult to meet the practical demands for high-precision and robust detection in complex application scenarios. Therefore, improving the adaptability of small target detection to blurred edges, shape changes, and complex backgrounds in UAV aerial photography scenarios has become an urgent technical challenge.

[0071] To address the aforementioned technical problems, this application provides a method for detecting small targets in an image. After acquiring the image to be detected, image features are first extracted from the image, and then bi-branch feature extraction processing is performed on the image features to obtain frequency domain detail features for representing frequency detail information of small targets and spatial attention features for representing spatial structure information of small targets. Subsequently, the frequency domain detail features are reweighted based on the spatial attention features to highlight the effective frequency response related to small targets, and the frequency domain weighted features and spatial attention features are further fused to obtain spatial frequency fusion features.

[0072] Based on this, the spatial frequency fusion features are spatially recalibrated to make the feature representation more consistent with the actual spatial distribution of small targets. Then, the spatially calibrated features are subjected to multi-scale feature aggregation processing to enhance the overall feature representation capability of small targets. Finally, classification and regression prediction are completed based on the aggregated features, and the small target detection results of the image to be detected are output.

[0073] This method organically combines frequency detail information, spatial structure information, spatial recalibration, and multi-scale aggregation, which can improve the perception of edge details and structural morphology of small targets in complex backgrounds, thereby improving the accuracy and robustness of small target detection in UAV images.

[0074] This method can be deployed in UAV aerial photography and detection systems, including UAV-borne imaging terminals, image transmission links, and ground analysis terminals, and is compatible with target detection architectures based on backbone networks, feature fusion networks, and detection heads.

[0075] The method for detecting small targets in images provided in this application belongs to the interdisciplinary field of computer vision, digital image processing and intelligent target recognition. It can be mainly used to detect and recognize small targets in drone aerial images. Drone aerial photography can be applied to scenarios such as precision agricultural inspection, road traffic supervision, park security patrol, ecological environment monitoring and disaster site search.

[0076] Figure 1 A scene illustration for the method of detecting small targets in images provided in this application, as shown below. Figure 1 As shown, in these scenarios, drones equipped with visible light imaging devices fly at high or low altitudes along preset routes, continuously collecting large-scale surface images. Then, a backend detection model identifies and locates small targets such as pedestrians, vehicles, pests, floating objects, waste, or equipment defects within the images. Specifically, as shown in the boxes in Figure 1, solid boxes represent vehicles, and dashed boxes represent pedestrians. Each box represents a small target, and the coordinates of the box and the category of the target within it are displayed nearby. Figure 1 (The coordinates and categories are not displayed in the image), or different colored boxes can be used to represent different categories of small targets; this application does not limit this.

[0077] The executing entity of this application can be a processor, image processing module, or target detection program deployed in an edge computing device, vehicle computing platform, cloud server, or integrated detection terminal. This application embodiment uses a processor as an example to specifically illustrate a method for detecting small targets in an image.

[0078] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0079] Figure 2 Flowchart of the method for detecting small targets in images provided in this application Figure 1 ,like Figure 2As shown, the method includes:

[0080] S201. Extract the image features of the image to be detected.

[0081] Understandably, the images to be detected can come from single-frame images captured in real time by UAV-borne visible light imaging equipment during flight, or from cached images sent to the ground analysis terminal via the image transmission link, or from offline images to be analyzed in historical aerial image datasets.

[0082] Image features are used to characterize the initial visual information of the image to be detected. They serve as the input basis for subsequent bi-branch feature extraction, reweighting, fusion, calibration, and aggregation. They are usually extracted by the backbone network through preliminary convolution of the image to be detected and carry the edge, texture, and semantic information of small targets.

[0083] In practice, the input image can first be normalized in size, standardized in pixel value, and aligned in color space to reduce the differences in data distribution caused by different drone equipment, different exposure conditions, and different flight times. Then, the preprocessed image is input into the backbone network. The backbone network can adopt a convolutional neural network structure or a structure combining convolution and lightweight attention modules. Shallow texture information and mid-level structure information are extracted through multi-layer convolution, non-linear activation, and stride downsampling.

[0084] S202. Perform bi-branch feature extraction on the image features to obtain frequency domain detail features and spatial attention features respectively.

[0085] Among them, frequency domain detail features are used to indicate the frequency detail information of small targets in the image to be detected, and spatial attention features are used to indicate the spatial structure information of small targets.

[0086] Understandably, the purpose of dual-branch feature extraction is to decouple different representation dimensions in the same image feature, so that one branch focuses more on small target edges, high-frequency textures and fine-grained changes, while the other branch focuses more on the spatial structure, location distribution and salience of the target area.

[0087] Frequency domain detail features, i.e., x fft It is used to indicate the frequency details of small targets in the image to be detected, mainly carrying edge, high-frequency and fine-grained texture components; spatial attention features, i.e., x att It is used to indicate the spatial structure information of small targets and to provide the importance distribution of different regions of the image.

[0088] In practice, the processor can input image features in parallel into the frequency domain branch and the spatial branch. For the frequency domain branch, a two-dimensional fast Fourier transform can be performed on the image features channel by channel to convert the spatial domain response to the frequency domain, obtaining a spectral representation containing amplitude and phase information, i.e., frequency domain detail features. Considering that high-frequency regions usually better reflect the edges and local contours of small targets, the processor can also perform frequency band reconstruction or high-frequency enhancement processing on the spectrum to obtain frequency domain detail features.

[0089] For spatial branches, channel compression can be performed on image features, and global average pooling, local convolution and context encoding can be performed in the spatial dimension to generate a two-dimensional weight map that can reflect the importance of different locations. This weight map can then be expanded or mapped to the corresponding channels to form spatial attention features.

[0090] Since small targets in UAV images are affected by viewpoint tilt, motion blur and scale abrupt changes, the dual-branch structure can express frequency details and spatial structure in a more explicit form, avoiding the problem of edge information and regional saliency being mutually masked by single-path features in complex backgrounds.

[0091] S203. Based on spatial attention features, the frequency domain detail features are reweighted to obtain frequency domain weighted features.

[0092] Understandably, the frequency domain weighted features are obtained by reweighting the frequency domain detail features using spatial attention features. This is used to enhance the frequency details of small targets and weaken irrelevant background components. The core of this step lies in using the regional importance information provided by the spatial branch to selectively enhance the high-frequency response in the frequency domain branch, so that frequency domain details consistent with the target location or related to the target structure are preserved, while random textures, repetitive textures, and noisy high-frequency components in the background are suppressed.

[0093] Specifically, to achieve reweighting of frequency domain detail features, the spatial attention feature xatt and the frequency domain detail feature xfft are multiplied element-wise in the frequency domain. Essentially, this operation uses the spatial attention feature as the modulation coefficient for frequency domain filtering, applying differentiated weight adjustments to different frequency components. That is, higher weights are given to frequency components corresponding to spatially important regions, while frequency components corresponding to unimportant regions are suppressed, thereby selectively enhancing and attenuating information in the frequency domain.

[0094] After the product operation is completed, the weighted frequency domain features are mapped back to the spatial domain using the inverse Fast Fourier Transform (iFFT) to obtain the final frequency domain weighted features xf, which is mathematically expressed as: xf = iFFT(xatt·FFT(x)), where x represents the input image features. This process realizes the reweighting of the frequency domain and the feature reconstruction of the spatial domain, so that the frequency domain weighted features retain the original frequency domain details while incorporating the structural priors guided by the spatial attention mechanism, thereby establishing an effective feature interaction between the frequency and spatial domains.

[0095] S204. By fusing the frequency domain weighted features and the spatial attention features, the spatial frequency fusion features are obtained.

[0096] Understandably, spatial-frequency fusion features are feature representations formed by fusing frequency-domain weighted features with spatial attention features, while preserving frequency details and spatial structural information, providing input for subsequent spatial recalibration. After processing in step S103, the frequency-domain weighted features already possess strong capabilities in representing small target edges and high-frequency details. However, without the synchronous participation of spatial structural semantics, unstable representation may still occur when there are significant changes in target morphology or strong background interference. Therefore, it is necessary to jointly encode the spatial attention features, which reflect the importance and structural distribution of the target region, with the reweighted frequency-domain information.

[0097] In practical implementation, frequency-domain weighted features and spatial attention features can be concatenated along the channel dimension to form joint features with expanded dimensions. Then, one or more convolutional layers are used to compress and blend the concatenated features to obtain spatial frequency fusion features with a unified number of channels. The convolutional kernel can be a 1×1 convolution to complete channel interaction, or a 3×3 convolution can be stacked to further model local neighborhood relationships. To improve fusion stability, a batch normalization layer and a non-linear activation function can be added after the convolution to enable scale alignment and non-linear renormalization of the two types of information within the same feature space.

[0098] Spatial attention features are used to indicate the spatial structure information of small targets, while frequency domain weighted features are used to enhance the frequency details of the targets. Therefore, the fused spatial-frequency fusion features simultaneously possess three types of expressive capabilities: edge sharpness, structural stability, and regional saliency.

[0099] S205. Perform spatial recalibration on the spatial frequency fusion characteristics to obtain spatial calibration characteristics.

[0100] Understandably, spatial calibration features are obtained by spatially recalibrating spatial frequency fusion features to further adapt to changes in the target under different viewpoints, attitudes, and aspect ratios. The purpose of spatial recalibration is to dynamically adjust the response intensity at each position and in each direction based on the structural differences in the spatial dimension, on the basis of the fused joint representation, so that the feature representation is more in line with the actual shape distribution of small targets.

[0101] In practical implementation, the spatial frequency fusion features can be encoded along the height and width directions respectively, forming channel-height description vectors and channel-width description vectors. Then, two parallel convolutional branches are used to learn vertical and horizontal structural variations, respectively. The outputs of these two branches can then be mapped back to the original spatial dimensions to obtain two directional calibration weight maps. Competitive normalization or gating mechanisms are used to suppress redundant directional responses and enhance dominant directional responses, ultimately generating a comprehensive spatial calibration weight. This weight is then applied to the spatial frequency fusion features to obtain the spatial calibration features.

[0102] S206. Perform multi-scale feature aggregation processing on the spatial calibration features to obtain aggregated features.

[0103] Among them, aggregated features are used to enhance the feature representation of small targets.

[0104] Understandably, aggregated features are obtained by multi-scale feature aggregation processing of spatial calibration features, used to enhance the feature representation of small targets and improve localization capabilities. Considering that the scale of small targets in UAV aerial images varies significantly, different flight altitudes, shooting angles, and target categories can lead to large differences in the size of targets in the image. Relying solely on spatial calibration features of a single scale is insufficient to simultaneously preserve the details of extremely small targets and semantically discriminate slightly larger targets. Therefore, multi-scale feature aggregation processing is required.

[0105] In practical implementation, feature maps from multiple resolution levels can be selected from the backbone network or feature fusion network. Spatial calibration features are used as the core input and laterally connected with features from other scales. Deeper low-resolution features can be upsampled to higher-resolution layers, while shallower high-resolution features can be mapped to lower-resolution layers through strided convolution or pooling, thus forming a bidirectional information flow. Subsequently, features from multiple scales can be concatenated, summed layer by layer, or weighted fused to obtain aggregated features that combine high-resolution texture and high-level semantics.

[0106] To avoid the loss of details in small targets due to multiple upsampling and downsampling, shallow features can be directly skipped to the aggregation module, so that edges, corners and fine-grained textures are preserved in the aggregation result.

[0107] S207. Classify and regress the aggregated features to obtain the small target detection results of the image to be detected.

[0108] Understandably, small target detection results are the final output obtained after classifying and regressing aggregated features, used to represent the location and category of small targets in the image to be detected. Classification and regression prediction can be implemented by a detection head, which can be placed after single-scale aggregated features or placed at multiple scale outputs to perform predictions separately.

[0109] In practice, aggregated features can be input into the classification and regression branches. The classification branch outputs the class probability and target presence probability corresponding to each candidate location or candidate box through several layers of convolution. The regression branch outputs bounding box parameters. After obtaining the initial prediction results, the classification confidence can be thresholded to remove low-confidence candidate boxes. Then, non-maximum suppression or its variants are used to remove redundant boxes with excessive overlap. Finally, the detection results of one or more small targets in the image to be detected are output.

[0110] Small target detection results typically include predicted bounding boxes, target category probabilities, bounding box offsets, and target presence probabilities. They can also be further mapped back to the original image coordinate system for use in UAV inspection interfaces, early warning systems, or subsequent task chains.

[0111] This step transforms the results of the aforementioned frequency detail enhancement, spatial structure guidance, spatial recalibration, and multi-scale aggregation into directly applicable detection output, thus completing the closed-loop processing from image input to target location and category determination. Because the aggregated features fully integrate the edge details, spatial distribution, and multi-scale context of small targets, the classification and regression prediction stages achieve higher localization accuracy and category recognition stability under complex backgrounds, blurred edges, and shape variations.

[0112] The method for detecting small targets in images provided in this application extracts image features from the image to be detected, performs bi-branch feature extraction processing on the image features to obtain frequency domain detail features and spatial attention features respectively, re-weights the frequency domain detail features based on the spatial attention features to obtain frequency domain weighted features, fuses the frequency domain weighted features and spatial attention features to obtain spatial frequency fusion features, and performs spatial recalibration on the spatial frequency fusion features to obtain spatial calibration features.

[0113] Multi-scale feature aggregation is performed on spatial calibration features to obtain aggregated features. Finally, classification and regression prediction are performed on the aggregated features to obtain the small target detection results of the image to be detected. This method decouples and extracts frequency detail information and spatial structure information related to small targets, and uses spatial attention to perform guided reweighting of frequency domain details. Then, through spatial frequency fusion, spatial recalibration, and multi-scale aggregation, high-discriminatory features for detection head prediction are formed. It can establish a continuous enhancement link for problems such as low pixel ratio of small targets, blurred edges, significant shape changes, and complex backgrounds in UAV aerial photography scenes.

[0114] Therefore, the detection network can more accurately distinguish target edges from background textures in complex backgrounds, more stably adapt to changes in the structure of small targets under different viewpoints and poses, and preserve and aggregate high-resolution details and deep semantic information at multiple scales, thereby improving the accuracy, robustness and scene adaptability of small target detection.

[0115] In some embodiments, the above-mentioned dual-branch feature extraction process specifically includes: performing frequency domain transformation on the image features to obtain frequency domain detail features corresponding to the small target; and performing convolution and global average pooling on the image features to capture the spatial structure information of the small target and obtain spatial attention features.

[0116] Understandably, frequency domain transformation is used to map image features from the spatial domain to the frequency domain in order to separate the high-frequency information corresponding to the edges, contours and textures of small targets from the low-frequency information corresponding to the background components, thereby forming frequency domain detail features that facilitate subsequent differentiation between targets and background.

[0117] Convolution is used to encode local neighborhood features of an image to enhance local responses related to the shape of small targets; global average pooling is used to perform global statistics on the features of each channel, compress the spatial dimension and retain the overall structural description, and then together with the convolutional features, they constitute spatial attention features to reflect the spatial distribution and structural relationship of small targets in the image.

[0118] Frequency domain transformation can be achieved using fast Fourier transform or wavelet transform. Fast Fourier transform is suitable for directly obtaining the spectrum representation, while wavelet transform is suitable for extracting detail components at different scales under multiple resolutions. In practical applications, other models of this component can also be selected, but this application does not limit the specific model.

[0119] In practical implementations, image features are typically composed of feature maps output by the backbone network, for example, they can be... The expression is represented as follows: C represents the channel dimension, H represents the height, and W represents the width. Dual-branch feature extraction can be performed in parallel on the same feature map. In the frequency domain branch, the input feature map is fed into the frequency domain transformation module by channel. After obtaining the spectral coefficients, the high-frequency response related to the boundary changes of small targets is retained, and frequency domain detail features, i.e., xfft, can be formed through amplitude mapping or frequency band filtering.

[0120] In the spatial branch, the input feature map is first processed by a convolutional layer to extract local spatial patterns, then global average pooling (GAP) is used to obtain channel-level global descriptions. Subsequently, an attention response is generated through convolutional mapping, which makes it more spatially indicative of the region where small targets are located. Finally, the spatial attention feature, i.e., xatt, is output. The outputs of the above two branches retain frequency details and spatial structure information, respectively, providing complementary representations for subsequent feature fusion.

[0121] The working principle of this dual-branch feature extraction method is that, on the one hand, frequency domain transformation is used to enhance the separability of small target edges and textures, and on the other hand, convolution and global average pooling are used to preserve the spatial structural consistency of the target, making the two types of features complementary in expression. This allows for simultaneous attention to detail and structural information under complex backgrounds and scale variations, providing a more stable feature foundation for subsequent reweighting, fusion, and detection.

[0122] By adopting this specific implementation method, the ability to perceive blurred edges and weak textures of small targets in UAV aerial images can be improved, the interference of vegetation, roof and road textures on detection results can be reduced, and the adaptability of the model to top-down, oblique-view and scale-changing scenes can be enhanced, thereby improving the accuracy and robustness of small target detection.

[0123] Figure 3 Flowchart of the method for detecting small targets in images provided in this application Figure 2 ,like Figure 3 As shown, in this embodiment... Figure 2 Based on the embodiments, the process of obtaining spatial frequency fusion features is described in detail. The method includes:

[0124] S301. Perform multi-scale convolution representation on the frequency domain weighted features to obtain spatial attention weights.

[0125] Understandably, domain-weighted features refer to the frequency response information retained after reweighting in the frequency domain. In UAV small target scenarios, they can highlight the effective components of edges, contours, and local textures. Spatial attention weights (LRM(X)) are weight maps obtained by representing frequency-domain weighted features based on different receptive fields. They are used to highlight target areas and suppress complex backgrounds in the spatial dimension.

[0126] In practical implementation, frequency-domain weighted features can be extracted in parallel by inputting the frequency-domain transformed feature map into multiple sets of convolutional units with different kernels. The kernel size can be set to 1x1, 3x3, and 5x5 to correspond to local texture, local neighborhood, and broader contextual information, respectively, thereby realizing a multi-level feature representation after frequency enhancement. Its mathematical expression is: xf=Conv 1×1 (xf) + Conv 3×3 (xf) + Conv 5×5 (xf).

[0127] Subsequently, a 1×1 convolutional layer is used to fuse the multi-scale convolutional outputs along the channel dimension. The core purpose of this operation is to extract the intrinsic correlation between adjacent channels, establish nonlinear interaction relationships between channels, and effectively reduce feature redundancy. Finally, the response values ​​of each channel are compressed to the (0, 1) interval using the Sigmoid activation function to obtain the normalized spatial attention weight LRM(X), which is essentially an adaptive recalibration of the importance of frequency domain weighted features in the spatial dimension.

[0128] S302. Based on spatial attention weights, the frequency domain weighted features and spatial attention features are weighted to obtain spatial frequency domain weighted features.

[0129] Understandably, the spatial frequency domain weighted feature (xF) is an intermediate feature formed by integrating the frequency domain weighted feature and the spatial attention feature under weight adjustment, which combines frequency details and spatial structure information.

[0130] Weighted processing can be achieved by multiplying the weighted graph with the frequency domain weighted features and the spatial attention features position by position and then summing them up, so that features from different sources complement each other in the spatial dimension.

[0131] Specifically, after obtaining the spatial attention weight LRM(X), the processor can perform element-wise multiplication with the spatial attention feature (xatt) to obtain the spatially recalibrated feature representation, and then perform weighted fusion with the frequency domain weighted feature xf. Its mathematical expression is: xF=LRM(xatt)·xf.

[0132] This process enables joint representation in the spatial and frequency domains: on the one hand, spatial attention weights selectively enhance frequency domain features, suppressing irrelevant spatial region responses; on the other hand, frequency-domain weighted features provide prior guidance on the frequency dimension for spatial attention, making attention allocation more precise. The element-wise multiplication of the two achieves complementary fusion of cross-domain features, resulting in a final spatial-frequency-domain weighted feature xF that simultaneously preserves spatial structure information and frequency distribution characteristics.

[0133] S303. Perform element-wise multiplication of the spatial frequency domain weighted features with the original spatial domain features to generate spatial frequency fusion features.

[0134] Understandably, the original spatial domain features are the basic spatial features directly extracted and retained from the input image, used as a fusion reference. The spatial frequency fusion feature (xout) is the fusion result obtained after element-wise multiplication coupling, which can be used as input for subsequent spatial recalibration and multi-scale aggregation.

[0135] Specifically, after the xF Fourier transform, different positions in its frequency domain representation correspond to different spatial frequency components. Low-frequency components carry global structural information, while high-frequency components reflect local details such as edges and textures. To adaptively refine the frequency information, a learnable resolver can be introduced. A learnable resolver is a trainable parameter matrix that modulates each frequency component element-wise in the frequency domain, and its function is equivalent to an adaptive frequency filter.

[0136] The learnable resolver operates on these frequency components element-wise, automatically learning which frequencies should be preserved or suppressed, thus achieving adaptive refinement of frequency information. In practical use, the learnable resolver can be initialized to 1 to maintain training stability. In the initial training phase, this resolver acts as an identity mapping, introducing no additional bias to the original frequency features and ensuring the smoothness of gradient propagation. As training progresses, the learnable resolver gradually deviates from 1, and the network dynamically adjusts the weights of each frequency component according to task requirements, ultimately learning the optimal frequency selection strategy.

[0137] Although the features modulated by the learnable resolver are more refined at the frequency level, they may lose some of the original spatial structure information. However, directly retaining the xF path can maintain the spatial integrity of the features. Therefore, after obtaining the frequency domain features refined by the resolver, they are first transformed back to the spatial domain by iFFT, and then multiplied by the original spatial-frequency domain weighted features by learnable coefficients α and β respectively, and then added to form a residual adaptive fusion mechanism.

[0138] α and β are learnable scalars that enable the network to adaptively balance the contribution ratio of the two paths according to the specific task. That is, when the benefit of frequency refinement is greater, α tends to increase; when the spatial structure of the original features is more critical, β tends to increase. This mechanism avoids the information loss that may be caused by a single path and ensures that the final output spatial-frequency fusion feature xout has both the discriminative power of frequency optimization and the structural fidelity of the original features.

[0139] The mathematical expression for the above process is: (xout=iFFT(FFT(xF)·p)×α+xF×β), where p represents the learnable parser.

[0140] This application first utilizes multi-scale convolution to extract spatial responses under different receptive fields from frequency-domain weighted features. Then, it adjusts the coupling strength between frequency and spatial information using spatial attention weights to achieve higher responses in the target region. Finally, it fuses the responses element-wise with the original spatial domain features, preserving the spatial layout of the target and enhancing detail discrimination capabilities. This specific implementation enhances the perception of small target edges and textures in complex backgrounds, reduces the interference of background noise on feature representation, and improves adaptability to changes in target scale, pose, and local blurring, thereby improving the accuracy and robustness of subsequent detection results.

[0141] Figure 4 Flowchart of the method for detecting small targets in images provided in this application Figure 3 ,like Figure 4 As shown, in this embodiment... Figure 2 Based on the embodiments, the space recalibration process is described in detail, and the method includes:

[0142] S401. Extract multi-branch spatial dimension perception features from the spatial frequency fusion features to generate multiple sets of spatial dimension features.

[0143] Among them, the spatial dimension features include channel height features and channel width features.

[0144] As is understandable, spatial frequency fusion features refer to the feature expression after fusing frequency detail information and spatial structure information, which includes the joint response of target edges, contours and local textures.

[0145] In practical implementation, multi-branch spatial dimension perceptual feature extraction can construct multiple perceptual branches in parallel based on the same spatial frequency fusion features. Each branch extracts contextual information along different spatial directions. Multiple perceptual branches perform perceptual feature extraction in different spatial directions, aiming to capture independent spatial semantic information from the orthogonal directions of the feature maps.

[0146] This process decomposes features into two independent perceptual paths: height and width. This allows the network to model spatial dependencies in the vertical and horizontal dimensions separately, thereby generating multiple sets of spatially sensitive features. For example, suppose the spatial frequency fusion feature is... Here, B, C, H, and W represent batch size, channel dimension, height, and width, respectively. Channel height and channel width features can be obtained by extracting them through the channel-height (CH) branch and the channel-width (CW) branch.

[0147] Specifically, the height-direction branch aggregates and encodes the vertical spatial information of the feature map along the channel dimension to extract the channel height feature. This feature reflects the target's scale distribution, morphological extension, and vertical texture arrangement in the vertical direction, and has a strong discriminative ability for identifying targets with significant vertical structures (such as columnar objects, human standing postures, etc.).

[0148] The width branch aggregates and encodes the lateral spatial information of the feature map along the channel dimension, extracting the channel width feature. This feature captures the spatial span and lateral structural distribution of the target in the horizontal direction, effectively characterizing the target's width variation, lateral proportional relationship, and horizontal contextual association.

[0149] The multi-branch design allows height and width features to be optimized independently in their respective dimensions, avoiding the coupling and interference of information from different spatial directions in a single branch. The resulting multiple sets of spatial dimensional features can carry spatial perception information in two orthogonal directions, height and width, at the channel level, providing a more refined and decoupled spatial representation for subsequent tasks such as feature fusion, target localization, or scale estimation.

[0150] S402. Perform dynamic calibration on multiple sets of spatial dimensional features to generate spatial calibration features that adapt to changes in aspect ratio.

[0151] Understandably, the main purpose of dynamically calibrating the extracted spatial dimensional features is to enable the features to adaptively respond to changes in the target's aspect ratio, thereby generating scale-robust spatial calibration features. Taking the CH branch as an example, specifically the height branch (CH branch), the calibration process first involves adjusting the input features... Perform the transpose operation to obtain The goal is to map the spatial information originally distributed along the height direction to the channel dimension, so that subsequent pooling operations can aggregate the contextual semantics along the height direction along the channel direction.

[0152] Subsequently, a parallel strategy of max pooling and average pooling is employed to process the transposed features. Max pooling focuses on capturing the most significant response values ​​along the height direction, preserving key structural information of the target; average pooling focuses on obtaining the overall statistical distribution along the height direction, reflecting global contextual features. The two are then concatenated along the channel dimension to obtain... The aim is to complement and integrate saliency information with global statistical information, thereby more comprehensively aggregating high-level context awareness.

[0153] After obtaining the concatenated contextual features, attention weights are generated through a concatenated activation function of convolutional layers. The aggregated contextual information is non-linearly mapped to importance weights for each channel, allowing the network to adaptively determine which channels in the height direction carry more discriminative information and which should be suppressed under the current input. The generated attention weights are then applied to the original features before transposition, recalibrating them channel by channel to strengthen feature responses semantically related to the target height while reducing the interference of redundant or noisy features. Finally, the refined features are restored to the original height × width spatial layout through dimension transposition. This completes the dynamic calibration of a single branch.

[0154] The processing principle of the CW branch is similar to that of the CH branch, but the attention mechanism mainly focuses on the CW dimension. This will not be elaborated further here.

[0155] After each branch completes its dynamic calibration, global average pooling is performed on the multi-branch features along the spatial dimension to aggregate the overall response intensity of each branch and generate the weights for each branch accordingly. This process can obtain the overall contribution of each spatial dimension branch to the current target, providing a basis for subsequent cross-branch fusion.

[0156] Building upon this, a Softmax-based competitive mechanism is employed to normalize the weights of each branch. The competitive nature of Softmax ensures that, under a given aspect ratio, branches that better match the target shape receive higher weights, while branches that do not match the current target shape are suppressed. This competitive adaptive weight allocation mechanism effectively addresses the issue of uneven response across different spatial dimensions for complex, small-aspect-ratio targets, achieving reliable fusion of multi-branch features and ultimately generating spatially calibrated features that adapt to changes in aspect ratio.

[0157] In this way, the processor can simultaneously perceive the target's height and width information during the spatial recalibration stage and perform dynamic weighting based on the competition between different branches. This makes the final output spatial calibration features more sensitive to aspect ratio changes, attitude deflection, and local stretching. As a result, subsequent multi-scale feature aggregation and classification regression prediction can obtain a more stable structural basis, thereby improving the detection accuracy, anti-interference ability, and robustness against complex backgrounds for small targets in UAV aerial photography scenarios.

[0158] Figure 5 Flowchart of the method for detecting small targets in images provided in this application Figure 4 ,like Figure 5 As shown, in this embodiment... Figure 2 Based on the examples, the process of obtaining aggregation features is described in detail, and the method includes:

[0159] S501, Extract multi-level features of spatial calibration features.

[0160] Understandably, multi-level features refer to feature layers of different depths or scales extracted from spatial calibration features, used to preserve shallow details and deep semantic information.

[0161] Extracting multi-level features means constructing a set of feature layers with different semantic granularities along the depth or scale dimension of the feature pyramid. Specifically, let the feature of the i-th layer be pi, where i represents the additional scale level of the feature. The smaller i is, the higher the spatial resolution and the shallower the semantic information; the larger i is, the lower the spatial resolution and the richer the semantic information.

[0162] This process can typically rely on the outputs of different stages of the backbone network, or be achieved through top-down and bottom-up cross-layer fusion using structures such as FPN (Feature Pyramid Network). The introduction of multi-level features allows subsequent processing to simultaneously capture high-frequency detail information of small targets and deep semantic information of large targets at different scales.

[0163] S502. For any hierarchical feature, obtain the upsampled feature of the hierarchical feature.

[0164] Understandably, hierarchical features refer to the intermediate layer representation extracted from spatial frequency fusion features during the multi-scale feature aggregation process. They typically include the target's local texture, edge contours, and contextual semantic information.

[0165] Upsampling features are feature maps obtained by upsampling the resolution of hierarchical features, used to supplement high-resolution spatial details. Specifically, they can be upsampled to a higher resolution using upsampling layers (such as nearest neighbor interpolation or transposed convolution) to obtain upsampling feature pi', which recovers the edge and corner information of small objects. The upsampling factor can be set to two or four times to establish a stronger correlation with adjacent high-resolution features. Upsampling can recover spatial details, enabling deep semantic features to have a spatial dimension that matches shallow features, thus creating conditions for subsequent feature aggregation.

[0166] S503. Perform downsampling on the upsampled features to obtain the downsampled features.

[0167] Understandably, downsampling features are feature maps obtained by scaling down upsampling features, used to achieve scale complementarity with the original layer features at the same representation level. Simultaneously with upsampling, a direct downsampling layer (such as a convolution or pooling operation with a stride of 2) can be introduced to downsampling the upsampling feature pi', resulting in the downsampling feature pi''. This scale reduction maintains the spatial alignment consistent with the original layer features, thus forming a bidirectional sampling scale loop.

[0168] S504. The upsampled features, downsampled features, and hierarchical features are concatenated and fused to obtain the hybrid features.

[0169] Understandably, hybrid features are intermediate representations that integrate features from multiple sources along the channel dimension, and can simultaneously preserve detailed information, structural information, and semantic information.

[0170] The upsampled feature Pi′, the downsampled feature Pi′′, and the original hierarchical feature Pi are concatenated along the channel dimension to generate a hybrid feature. This concatenation and fusion strategy enables the network to simultaneously utilize high-resolution details (from Pi′′), low-resolution semantic summaries (from Pi′′), and complete information at the original scale (from Pi) within the same feature space, thereby constructing a multi-source feature representation with complementary information.

[0171] S505. Perform feature recombination and information extraction on the mixed features to obtain the refined features corresponding to the hierarchical features.

[0172] Understandably, feature reorganization involves reorganizing the channel order and spatial response of hybrid features, while information extraction is used to filter out redundant responses and enhance effective features relevant to small targets. Refined features refer to the hierarchical output formed after bidirectional sampling fusion, which takes into account detailed texture, target contours, and neighborhood semantics.

[0173] Feature recombination and information extraction of the concatenated hybrid features can be achieved through the C2f-FRM mechanism. The C2f-FRM mechanism first performs two C2f transformations on the hybrid features. The first C2f takes the hybrid features as input and performs feature recombination through internal cross-stage partial connections (CSP) and bottleneck structures, extracting and enhancing the correlation between channels. Then, the output of the first C2f is concatenated with the original hierarchical features Pi and fed into the second C2f for secondary extraction. This cascaded structure of dual C2f allows the network to further integrate residual information that has not been fully utilized in the original hierarchical features, ultimately outputting refined features Pi. r .

[0174] Taking p3 as an example, its refining process can be represented as P3 r =C2f(C2f(P3′′‖P3′)‖P3), where ‖ represents the concatenation operation. The refined features combine high-resolution detail with strong semantic expressiveness, and, constrained by a bidirectional calibration path, possess superior small target representation capabilities.

[0175] Subsequently, with the refined P3 rBased on this, P2′′ is generated through an upsampling layer, and the final high-resolution output feature P2out=C2f(C2f(P2′′‖P2′)‖P2) is generated using a similar C2f-FRM mechanism. It is worth emphasizing that the generation of features at the remaining levels from P3 to P5 is based on the refined P2out. That is, the shallow refinement results will serve as semantic guidance for deep feature extraction, forming a bottom-up, layer-by-layer enhancement mechanism, thereby improving the consistency and detection accuracy of multi-scale features as a whole.

[0176] This processing method employs a bidirectional sampling path that first expands and then regresses, enabling features at the same level to simultaneously acquire high-resolution details and original-scale structural information. These are then stitched together to form a richer hybrid representation. Subsequently, feature recombination and information refinement are used to compress noise and enhance target-related responses. The resulting refined features retain the fine-grained texture of small targets while maintaining spatial semantics consistent with the current level, which helps improve the stability of subsequent multi-scale aggregation and detection prediction.

[0177] S506. Aggregate the refined features corresponding to multiple hierarchical features to obtain aggregated features.

[0178] Understandably, aggregated features refer to the final fused representation obtained by unifying and integrating refined features corresponding to multiple levels, which is used for subsequent classification and regression prediction.

[0179] Refined features from multiple levels can be concatenated based on channel dimension or weighted and fused according to preset weights, and the number of channels can be compressed through convolution to output aggregated features. This aggregation process enables features of different scales to work synergistically within a unified representation space.

[0180] This multi-scale feature aggregation method enhances the complementarity of high and low resolution information by introducing bidirectional sampling fusion within each level, and retains the discrimination cues at different scales through cross-layer aggregation, so that the spatially calibrated features have both detail sensitivity and strong contextual expression ability.

[0181] By adopting the above method, the aggregated features can more fully preserve the edge, corner and local texture information of small targets, and reduce the loss of shallow details in cross-layer transmission, thereby improving the ability to distinguish small targets in complex backgrounds, the stability of localization and the final detection accuracy.

[0182] Figure 6 Flowchart of the method for detecting small targets in images provided in this application Figure 5 ,like Figure 6 As shown, in this embodiment... Figure 2 Based on the examples, the process of obtaining small target detection results is described in detail. The method includes:

[0183] S601. Classify and calculate regression branches for aggregated features to obtain multiple prediction results.

[0184] The prediction results include: predicted bounding boxes and the corresponding object category probabilities, bounding box offsets, and object presence probabilities.

[0185] Understandably, the predicted bounding box is used to represent the candidate location of the target corresponding to the aggregated feature, the target category probability is used to characterize the likelihood that the candidate location belongs to each target category, the bounding box offset is used to describe the position correction information of the candidate box relative to the reference anchor box or feature grid, and the target existence probability is used to reflect the credibility of whether there is a real target in the candidate box.

[0186] In practice, after aggregating features and inputting them into the detection head, the detection head typically employs a multi-branch structure: the classification branch is responsible for predicting the probability distribution of the target's category, i.e., outputting the conditional probability of each category on a predefined set of categories. The regression branch is responsible for predicting the offset of the bounding box relative to predefined anchor boxes or grid cells, usually expressed as center coordinate offset and width / height scaling factors. Additionally, there is a target presence branch, which outputs the probability that a target is present at that location.

[0187] The three branches mentioned above share underlying features, but operate independently at the output layer. Since each spatial location on the feature map typically has multiple anchor boxes of different scales and aspect ratios, a single forward computation generates a large number of prediction results, forming a dense prediction set.

[0188] S602. Map the bounding box offset to obtain the actual bounding box coordinates mapped back to the image to be detected.

[0189] Understandably, the actual bounding box coordinates are the target position representation after restoring the network output to the coordinate system of the image to be detected.

[0190] The offsets output by the regression branch are not direct pixel coordinates, but rather relative transformation values ​​with respect to the reference boxes (anchor boxes or grid cells). To obtain the actual bounding boxes that can be used for subsequent processing, these relative transformation values ​​need to be inversely mapped back to the pixel coordinate system of the original image. Specifically, the normalized coordinates are multiplied by the width and height of the input image to be detected, thus completing the coordinate restoration from the feature map space to the image space. The essence of this process is to restore the relative geometric transformations learned by the network to absolute geometric positions, so that the predicted boxes can be compared with the ground truth labeled boxes in the same coordinate system.

[0191] S603. Based on the target category probability and the target existence probability, determine the confidence level of each predicted bounding box.

[0192] Understandably, confidence score is used to comprehensively measure the effectiveness of the predicted bounding box. It is typically calculated as the product of the target's presence probability and its class probability. The target presence probability reflects the likelihood that the location contains any target, while the class probability provides the probability of belonging to a specific class given the known presence of a target. The principle behind multiplying these two probabilities to determine the confidence score is that if a location does not contain a target, the confidence score will approach zero regardless of the class probability; if a target is present but the class is uncertain, the confidence score will also be suppressed. Only when the target exists and the class is clear will the confidence score take a high value.

[0193] The confidence level calculation can take into account both the existence of a target and the nature of the target, providing a unified ranking basis for subsequent screening.

[0194] S604. Determine the candidate bounding box set based on the confidence level and the confidence threshold.

[0195] Understandably, the candidate bounding box set is used to retain high-confidence boxes that meet the decision criteria.

[0196] After obtaining all predicted bounding boxes and their confidence scores, a large number of low-quality predictions need to be filtered out using a threshold. That is, a preset confidence threshold is set, and predicted boxes with confidence scores below this threshold are discarded, while those with confidence scores above the threshold are retained as a candidate set. The selection of the threshold directly affects the balance between recall and precision in detection; a threshold that is too high will lead to missed detections, while a threshold that is too low will introduce a large number of false positives. In practical engineering, this threshold can be dynamically adjusted according to the application scenario. After confidence filtering, the size of the candidate bounding box set is significantly reduced, retaining only predictions that the network considers relatively reliable, laying the foundation for subsequent post-processing operations such as non-maximum suppression.

[0197] S605. Determine the small target detection result based on the intersection-union ratio of the candidate bounding box set and the actual bounding box coordinates.

[0198] Understandably, small object detection results are used to determine the location and category of small objects. The intersection-union ratio (IU) is used to characterize the degree of overlap between candidate bounding boxes and their actual locations.

[0199] The candidate bounding box set often contains a large number of overlapping predicted boxes, which may correspond to the same ground truth target. To eliminate redundancy, it is necessary to calculate the Intersection over Union (IoU) ratio between the predicted boxes, which is the ratio of the intersection area to the union area of ​​two bounding boxes. A Non-Maximum Suppression (NMS) algorithm can be used to iterate through the candidate boxes from highest to lowest confidence: for the box with the highest confidence, it is retained as the final detection result, and then all other boxes with an IoU exceeding a set threshold are deleted, as they are highly likely to point to the same target.

[0200] Repeat this process until all candidate boxes have been processed. The final output bounding box is the small object detection result. The IoU threshold setting also needs to strike a balance between false negatives and false positives, and is a key hyperparameter in the post-processing stage of object detection.

[0201] This embodiment, while completing bounding box localization, introduces a joint determination mechanism of class probability and target existence probability, making candidate box selection more consistent with the actual target distribution. Furthermore, by performing precise matching through intersection-union comparison of candidate boxes, redundant boxes and false detections can be suppressed, improving the localization accuracy and classification stability of small targets. Since the aggregated features have already incorporated multi-scale information, combined with classification regression prediction and confidence filtering, it can enhance the robustness of small target detection in UAV aerial photography scenarios and improve the recognition accuracy in complex backgrounds.

[0202] Figure 7 A schematic diagram of the structure of the small target detection device in the image provided in this application is shown below. Figure 7 As shown, the image small target detection device 70 provided in this embodiment includes:

[0203] The acquisition module 701 is used to extract image features from the image to be detected;

[0204] Processing module 702 is used to perform bi-branch feature extraction processing on image features to obtain frequency domain detail features and spatial attention features respectively;

[0205] Processing module 702 is used to reweight the frequency domain detail features based on spatial attention features to obtain frequency domain weighted features;

[0206] The fusion module 703 is used to fuse frequency domain weighted features and spatial attention features to obtain spatial frequency fusion features;

[0207] The calibration module 704 is used to perform spatial recalibration on the spatial frequency fusion features to obtain spatial calibration features;

[0208] Processing module 702 is used to perform multi-scale feature aggregation processing on spatial calibration features to obtain aggregated features, which are used to enhance the feature representation of small targets.

[0209] The processing module 702 is used to classify and regress the aggregated features to obtain the small target detection results of the image to be detected.

[0210] In one possible implementation, the processing module 702 is specifically used to perform frequency domain transformation on the image features to obtain the frequency domain detail features corresponding to the small target.

[0211] The processing module 702 is specifically used to perform convolution and global average pooling on image features to capture spatial structure information of small targets and obtain spatial attention features.

[0212] In one possible implementation, the processing module 702 is specifically used to perform multi-scale convolution representation on the frequency domain weighted features to obtain spatial attention weights.

[0213] The processing module 702 is specifically used to perform weighted processing on the frequency domain weighted features and the spatial attention features based on the spatial attention weight to obtain the spatial frequency domain weighted features;

[0214] The fusion module 703 is specifically used to perform element-wise multiplication of the spatial frequency domain weighted features with the original spatial domain features to generate spatial frequency fusion features.

[0215] In one possible implementation, the processing module 702 is specifically used to perform multi-branch spatial dimension perception feature extraction on the spatial frequency fusion features, and generate multiple sets of spatial dimension features, including channel height features and channel width features.

[0216] The calibration module 704 is specifically used to dynamically calibrate multiple sets of spatial dimensional features to generate spatial calibration features that adapt to changes in aspect ratio.

[0217] In one possible implementation, the acquisition module is specifically used to extract multi-level features of the spatial calibration features;

[0218] Processing module 702 is specifically used to perform bidirectional sampling and fusion processing on any hierarchical feature to obtain refined features;

[0219] The processing module 702 is specifically used to aggregate the refined features corresponding to multiple hierarchical features to obtain aggregated features.

[0220] In one possible implementation, the acquisition module is specifically used to acquire upsampled features of the hierarchical features;

[0221] Processing module 702 is specifically used to perform downsampling processing on the upsampled features to obtain downsampled features;

[0222] Processing module 702 is specifically used to concatenate and fuse upsampled features, downsampled features, and hierarchical features to obtain hybrid features;

[0223] The processing module 702 is specifically used to perform feature recombination and information extraction on the mixed features to obtain the refined features corresponding to the hierarchical features.

[0224] In one possible implementation, the processing module 702 is specifically used to classify and calculate the regression branch of the aggregated features to obtain multiple prediction results, including: predicted bounding boxes and the target category probability, bounding box offset and target existence probability corresponding to the predicted bounding boxes;

[0225] Processing module 702 is specifically used to map the bounding box offset to obtain the actual bounding box coordinates mapped back to the image to be detected;

[0226] The processing module 702 is specifically used to determine the confidence level of each predicted bounding box based on the target category probability and the target existence probability;

[0227] Processing module 702 is specifically used to determine the candidate bounding box set based on confidence level and confidence threshold;

[0228] The processing module 702 is specifically used to determine the small target detection result based on the intersection-union ratio of the candidate bounding box set and the actual bounding box coordinates. The small target detection result is used to determine the location and category of the small target.

[0229] The small target detection device in the image provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0230] Figure 8 A schematic diagram of the structure of the electronic device provided in this application. Figure 8 As shown, the electronic device 80 provided in this embodiment includes at least one processor 801 and a memory 802. Optionally, the electronic device 80 further includes a communication component 803. The processor 801, memory 802, and communication component 803 are connected via a bus 804.

[0231] In the specific implementation process, at least one processor 801 executes computer execution instructions stored in memory 802, causing at least one processor 801 to execute the above-described method for detecting small targets in an image.

[0232] The specific implementation process of processor 801 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0233] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0234] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0235] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0236] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method for detecting small targets in an image.

[0237] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method for detecting small targets in an image.

[0238] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0239] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0240] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0241] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0242] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0243] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0244] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0245] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope.

Claims

1. A method for detecting small targets in an image, characterized in that, include: Extract image features from the image to be detected; The image features are subjected to bi-branch feature extraction processing to obtain frequency domain detail features and spatial attention features, respectively. The frequency domain detail features are used to indicate the frequency detail information of small targets in the image to be detected, and the spatial attention features are used to indicate the spatial structure information of the small targets. Based on the spatial attention features, the frequency domain detail features are reweighted to obtain frequency domain weighted features; By fusing the frequency domain weighted features and the spatial attention features, a spatial frequency fusion feature is obtained; Spatial recalibration is performed on the spatial frequency fusion features to obtain spatial calibration features; The spatial calibration features are subjected to multi-scale feature aggregation processing to obtain aggregated features, which are used to enhance the feature representation of the small target. The aggregated features are classified and regression predicted to obtain the small target detection results of the image to be detected.

2. The method according to claim 1, characterized in that, The bi-branch feature extraction process on the image features yields frequency domain detail features and spatial attention features, including: The frequency domain features corresponding to the small target are obtained by performing frequency domain transformation on the image features; The image features are processed by convolution and global average pooling to capture the spatial structure information of small targets and obtain spatial attention features.

3. The method according to claim 2, characterized in that, The fusion of the frequency domain weighted features and the spatial attention features to obtain spatial frequency fusion features includes: The frequency domain weighted features are represented by multi-scale convolution to obtain spatial attention weights; Based on the spatial attention weight, the frequency domain weighted features and the spatial attention features are weighted to obtain spatial frequency domain weighted features; The spatial frequency domain weighted features are multiplied element-wise with the original spatial domain features to generate spatial frequency fusion features.

4. The method according to claim 1, characterized in that, The spatial recalibration of the spatial frequency fusion features to obtain spatial calibration features includes: Multi-branch spatial dimension perception feature extraction is performed on the spatial frequency fusion features to generate multiple sets of spatial dimension features, including channel height features and channel width features. The multiple sets of spatial dimensional features are dynamically calibrated to generate spatial calibration features that adapt to changes in aspect ratio.

5. The method according to claim 4, characterized in that, The multi-scale feature aggregation process performed on the spatial calibration features to obtain aggregated features includes: Extract multi-level features from the spatial calibration features; For any hierarchical feature, a bidirectional sampling and fusion process is performed on the hierarchical feature to obtain a refined feature; The refined features corresponding to multiple hierarchical features are aggregated to obtain aggregated features.

6. The method according to claim 5, characterized in that, The bidirectional sampling and fusion processing of the hierarchical features to obtain refined features includes: Obtain the upsampled features of the aforementioned hierarchical features; The upsampled features are downsampled to obtain the downsampled features; The upsampled features, the downsampled features, and the hierarchical features are concatenated and fused to obtain hybrid features; The hybrid features are recombined and information is extracted to obtain the refined features corresponding to the hierarchical features.

7. The method according to claim 1, characterized in that, The process of classifying and regressing the aggregated features to obtain the small target detection results of the image to be detected includes: The aggregated features are classified and regression branches are calculated to obtain multiple prediction results, including: predicted bounding boxes and the target category probability, bounding box offset and target existence probability corresponding to the predicted bounding boxes; The bounding box offset is mapped to obtain the actual bounding box coordinates mapped back to the image to be detected. Based on the target category probability and the target existence probability, the confidence level of each predicted bounding box is determined; Based on the confidence level and the confidence threshold, a set of candidate bounding boxes is determined; The small target detection result is determined based on the intersection-union ratio of the candidate bounding box set and the actual bounding box coordinates. The small target detection result is used to determine the location and category of the small target.

8. A device for detecting small targets in an image, characterized in that, include: The acquisition module is used to extract image features from the image to be detected; The processing module is used to perform bi-branch feature extraction processing on the image features to obtain frequency domain detail features and spatial attention features, respectively. The processing module is used to reweight the frequency domain detail features based on the spatial attention features to obtain frequency domain weighted features; The fusion module is used to fuse the frequency domain weighted features and the spatial attention features to obtain spatial frequency fusion features; The calibration module is used to perform spatial recalibration on the spatial frequency fusion features to obtain spatial calibration features; The processing module is used to perform multi-scale feature aggregation processing on the spatial calibration features to obtain aggregated features, which are used to enhance the feature representation of the small target. The processing module is used to classify and regress the aggregated features to obtain the small target detection results of the image to be detected.

9. An electronic device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-7.

Images