Small target detection method based on hierarchical feature fusion and calibration

Abstract

The invention discloses a small target detection method based on hierarchical feature fusion and calibration, and the method comprises the steps: 1), reserving the multi-scale features from S2 to S5 through a ResNet50 backbone network, so as to give consideration to the details and deep semantic information of a small target; 2) constructing a feature fusion and calibration network, introducing inverse wavelet pooling operation to relieve edge blur and artifacts of a small target, transmitting low-layer features to high-layer features, performing local-global learning at the same time, and then performing information complementation and semantic alignment redundancy suppression on the fused features to improve feature discrimination; 3) in a decoder stage, coding features are interacted by means of learnable target query and a cross attention mechanism, and prediction is optimized step by step; and 4) performing end-to-end output through a Hungary algorithm, and inhibiting prediction frame expansion by adopting an InerPIoU loss function to improve positioning precision. The method effectively solves the problems of small target edge blur, feature loss, fusion redundancy and the like, and realizes high-precision detection.

Description

Technical Field

[0001] This invention relates to the field of small target detection, and in particular to a small target detection method that requires higher precision in detecting even smaller objects. Background Technology

[0002] Against the backdrop of a complex and volatile global situation and the urgent need for agricultural modernization, target detection technology has demonstrated significant application value in fields such as national defense and security monitoring and precision agriculture management. The development of this technology can be broadly divided into two main stages: the early stage of traditional methods and the current stage dominated by deep learning. Traditional methods typically employ detection frameworks based on handcrafted features. A typical process includes extracting candidate regions using a sliding window, representing them with manually designed features (such as SIFT and HOG), and using traditional machine learning classifiers (such as SVM) to complete target recognition. However, these methods have three inherent limitations: high computational complexity and high hardware resource consumption; heavy reliance on expert prior knowledge, resulting in high labor costs; and limited generalization ability of manually designed features. In recent years, with breakthroughs in deep learning technology, target detection methods based on convolutional neural networks (such as Faster R-CNN and YOLO) have gradually become the mainstream research approach. These methods can automatically learn feature representations and support end-to-end optimization, significantly improving the performance and efficiency of small target detection. Current object detection methods based on convolutional neural networks are mainly divided into two categories: single-stage and two-stage detection paradigms. Two-stage detectors (represented by Faster R-CNN) first use a Region Proposal Network (RPN) to generate candidate regions, and then perform fine-grained classification and location regression on these regions. They generally perform well in terms of detection accuracy, but have high computational complexity and relatively slow inference speed. In contrast, single-stage detectors (such as the YOLO series) do not require a region proposal stage and directly perform object classification and bounding box regression in parallel at pre-defined anchor boxes on the feature map, thus achieving higher inference efficiency. However, the performance of such methods is often constrained by post-processing strategies (such as Non-Maximum Suppression, NMS) and manually designed factors such as anchor box hyperparameters. In small object detection tasks, due to the small size of the target, complex background, and the fact that they are usually located in large-scale scenes, their visual features often lack saliency and discriminativeness. Furthermore, in detection models based on deep convolutional networks, as the network depth increases, the feature information of small targets is easily lost during multiple downsampling processes, which seriously affects detection performance. To address the aforementioned issues, this method improves the DETR model by introducing a feature pyramid structure to fuse feature maps at different levels. This aims to enhance the representation of small target features, alleviate the semantic misalignment problem in multi-scale feature fusion, and thus improve the accuracy of small target detection. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings and deficiencies of existing technologies and provide a small target detection method based on hierarchical feature fusion and calibration. This method solves the problems of jagged object edge artifacts, loss of small target feature information in deep convolutional networks, semantic misalignment after feature fusion, and decreased accuracy caused by unreasonable loss functions due to traditional upsampling methods. By replacing the traditional upsampling method with inverse wavelet pooling, extracting small target features using spatially separable deformable convolution, and constructing a three-branch structure of global branch, large-scale branch, and local branch, multi-level feature representation learning from global to local is achieved. A hierarchical complementary calibration module is designed to eliminate semantic misalignment during feature fusion, thereby improving the accuracy of small target detection.

[0004] To achieve the above objectives, the present invention provides a small target detection method based on hierarchical feature fusion and calibration, comprising the following steps:

[0005] 1) Using ResNet50 as the backbone network, the image is input into the backbone network to obtain feature maps at various levels. The S2, S3, S4, and S5 feature maps are retained. S2 contains rich small object feature information, which is crucial for improving the performance of small object detection. Subsequently, the highest-level feature map is processed by a self-attention mechanism to effectively capture long-distance dependencies and generate an enhanced F5 feature map.

[0006] 2) Constructing a Feature Pyramid Network: In the feature pyramid network, inverse wavelet pooling is used to replace the traditional upsampling method to solve the jagged edges of small targets caused by upsampling. A feature completion module is used to pass the small target feature information from the low-level feature maps to the high-level feature maps to supplement the missing small target features. The features after feature fusion are further optimized by a hierarchical complementary calibration module to optimize the feature representation between different levels of feature maps and solve the semantic misalignment problem after feature map fusion.

[0007] 3) Entering the decoder stage, the learnable target query interacts with the encoded features through a cross-attention mechanism to gradually optimize the prediction results;

[0008] 4) The Hungarian algorithm is used to directly output bounding boxes and class predictions, completing the end-to-end detection process. The improved loss function can solve the problem of the predicted box increasing in size during training, thereby improving detection accuracy.

[0009] Furthermore, step 1) comprises four stages:

[0010] a) The image is convolved using a large 7x7 kernel with a stride of 2, which quickly reduces the spatial dimensions (height and width) to half the original input. Immediately following, a 2x2 max-pooling layer with a stride of 2 further compresses the feature map size. The core task of this step is to rapidly reduce the computational load and initially capture some low-level, basic visual patterns, such as edges, corners, and color patches.

[0011] b. Image information formally enters the core of ResNet50—four sequentially connected residual stages (Stage 1 to Stage 4). Each stage consists of a series of stacked residual blocks. Within these blocks, information propagates along two paths: the main path performs a series of complex convolutions and nonlinear transformations; the other is a shortcut path, which may be an identity mapping (if the input and output dimensions are the same). Finally, the outputs of the main path and the shortcut path are element-wise added together and then passed through a ReLU activation function. This "addition" operation is the essence of residual learning, allowing the network to learn only the "residuals" or "small perturbations" based on existing features, greatly reducing the learning difficulty. Specifically, these four stages progressively deepen feature extraction: Stage 1 uses three residual blocks, focusing on combining more complex patterns, such as textures and simple shapes, from low-level features; Stage 2's four residual blocks begin to capture larger components, such as wheels or an animal's eyes; Stage 3's six residual blocks further integrate information to form components that approximate the object; and the final Stage 4 utilizes three residual blocks to extract highly abstract and semantic global features closely related to high-level concepts of the image, such as the entire object or scene. As the network deepens, the spatial size of the feature maps gradually decreases through stride or pooling, while the number of channels doubles with each stage. This means that spatial information is compressed, while the information capacity and richness of the features continuously increase.

[0012] c. The network typically outputs a two-dimensional feature map extracted from the final stage. This feature map is the rich, multi-layered feature representation that ResNet50 has learned for the input image. It perfectly balances spatial information (not yet completely destroyed by global pooling) and semantic depth, and can serve as a powerful, general visual feature that can be directly used in subsequent tasks such as image classification (by adding global average pooling layers and fully connected layers), laying a solid foundation for subsequent downstream tasks.

[0013] d. After obtaining the multi-level feature maps, retain feature maps S2, S3, S4, and S5, while discarding feature map S1. This is mainly because the spatial size of feature map S1 is large; retaining it would significantly increase computational complexity, hindering efficient model training and inference. Then, feature map S5 is processed using a self-attention mechanism. This mechanism effectively models the global dependencies within features, thereby improving the model's perceptual performance in complex scenes. The calculation formula for the self-attention mechanism is as follows:

[0014] ,

[0015] Where Q, K, and V are the Query, Key, and Value extracted from the input feature map, respectively, and d k This represents the dimension of the input key vector, and the Softmax operation is used for normalization. After performing self-attention computation on the S5 feature map, the enhanced feature map F5 is obtained.

[0016] Furthermore, in step 2), the specific implementation process of constructing the feature pyramid network is as follows:

[0017] a. A Feature Pyramid Network (FPN) is adopted as the core architecture to construct a top-down feature fusion path. This path starts with the high-level feature F5 output by the backbone network. This feature has strong semantic expressive power, but its spatial resolution is relatively low. To improve the upsampling quality, inverse wavelet pooling is used to replace traditional interpolation methods (such as bilinear interpolation or nearest neighbor interpolation). Traditional methods are prone to introducing jagged edge artifacts when processing small targets, thus affecting the ability to distinguish between targets and backgrounds; while inverse wavelet pooling, with its good frequency domain preservation properties, can effectively suppress edge distortion and improve the integrity of feature representation. In the specific implementation, the inverse wavelet transform uses the Haar wavelet basis to complete the two-dimensional discrete wavelet transform. The mathematical definition of the Haar wavelet basis is as follows:

[0018]

[0019] In the wavelet transform process, the feature map undergoes a one-dimensional discrete wavelet transform (DWT) sequentially along both row and column dimensions. This alternating row and column decomposition divides the input signal into four sub-bands with distinct physical meanings: the LL sub-band (containing low-frequency approximation information of the image), the LH sub-band (representing high-frequency details in the horizontal direction), the HL sub-band (representing high-frequency details in the vertical direction), and the HH sub-band (representing high-frequency details in the diagonal direction). Inverse wavelet pooling, as the inverse process of the wavelet transform, first performs a first-order wavelet decomposition on the input feature map, obtaining low-frequency approximation coefficients and three high-frequency detail sub-bands. Then, each high-frequency sub-band is upsampled by a factor of 2 to generate new first-order decomposition coefficients, while the original first-order decomposition results are considered second-order decomposition coefficients. Finally, these coefficients are reconstructed using the inverse discrete wavelet transform (IDWT) to generate a differentiable feature map suitable for gradient backpropagation. In this study, inverse wavelet pooling is applied to upsample the F5 feature map by a factor of 2, aligning its spatial dimensions with the F4 feature map. This method can effectively preserve image structural information and enhance the separability between the target and the background;

[0020] b. The upsampled F5 features are concatenated and fused with the features extracted in stage S4 using a Concat operation along the channel dimension. This concatenation method effectively combines high-level semantic information and mid-level feature representation. Continuing along the top-down fusion path, the optimized S4 features are amplified to a spatial size matching the S3 features using the same inverse wavelet pooling upsampling operation. Then, S2, S3, and S4 are input into the FCM module, where the rich small target feature information in the S2 feature map is passed to S3, and the missing small target feature information is supplemented in S4. Finally, the representation of small target feature information is enhanced through local-to-global feature learning.

[0021] c. The fused features are then fed into the HCCM module in a top-down manner to further optimize the interaction between features at different levels, effectively alleviating semantic misalignment and feature redundancy problems, and finally generating more discriminative feature representations.

[0022] Furthermore, in step 3), the specific implementation process for entering the decoder stage is as follows:

[0023] a. In the decoder stage, a fixed set of learnable parameters is first initialized as initial query vectors, typically set to the model's default maximum number of predicted targets. These query vectors are essentially trainable embedding vectors, each representing the location and category information of a potential target in the image. The initial query vectors are generated through random initialization and progressively optimized during training to capture the distribution characteristics of different targets;

[0024] b. In each decoder layer, the initial query vector first enters the self-attention module. This module models the correlation between different queries by calculating the similarity between query vectors. The self-attention mechanism captures their co-occurrence relationships in space. This process is implemented through a multi-head attention mechanism, which divides the query vector into multiple subspaces, calculates attention weights for each subspace, and finally aggregates them into a global relation representation. The output of the self-attention mechanism is the updated query vector, which has the same dimension as the input but contains richer contextual information;

[0025] c. The updated query vector enters the cross-attention module, where it interacts with the global image features output by the encoder. The encoder has already converted the input image into a feature map containing spatial relationships using a self-attention mechanism. The cross-attention module uses the query vector as a "guide" to extract information related to the current query from the feature map. Specifically, the query vector serves as the "query" (Q), and the image features serve as the "key" (K) and "value" (V). Attention weights are used to calculate aggregated features, generating a target feature representation corresponding to the query.

[0026] d. After multiple layers of decoder iteration, the final query vector is input into the prediction head to generate the object detection result. The prediction head consists of two branches: Class prediction branch: Through fully connected layers and the Softmax function, it outputs the probability distribution of the target class (including the background class) corresponding to each query. Bounding box prediction branch: Through a multilayer perceptron (MLP), it regresses the center coordinates, width and height (or corner coordinates) of the target box.

[0027] Furthermore, in step 4), the specific implementation process of performing Hungarian matching to output the prediction result is as follows:

[0028] A Hungarian algorithm is employed to perform binary matching between predicted results and ground truth labels. During this matching process, an improved InnerPIoU loss function is introduced as a measure of matching cost, replacing the traditional regression loss function. This method effectively alleviates the anchor box inflation problem caused by regression bias, ensuring that each ground truth target is assigned only a unique predicted box, thereby avoiding duplicate detection and improving the stability and accuracy of target assignment.

[0029] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0030] 1. This invention introduces an inverse wavelet pooling module (IWP) to replace the traditional upsampling method, which significantly improves the problem of reduced edge sharpness caused by bilinear interpolation, making the edges of small targets clear and easy to distinguish from complex backgrounds.

[0031] 2. This invention proposes a Feature Completion Module (FCM), which effectively alleviates the problem of missing small targets in high-level features by fusing small target information in low-level features with high-level features, and achieves multi-level feature learning by combining global, large-scale and local branches.

[0032] 3. This invention proposes a Hierarchical Complementary Calibration Module (HCCM) to address the complementarity problem between multi-level features. High-level features are semantically rich but lack information on small targets, while low-level features are detailed but semantically weak; both have limitations when used alone. HCCM achieves cross-layer interaction through a Bidirectional Feature Recalibration Unit (BFRU), enhancing the ability to extract key details from low-level features while strengthening the semantic focus of high-level features, thereby generating feature representations that are balanced in detail and semantics. This module effectively alleviates the problems of feature redundancy and semantic misalignment, significantly improving the performance of multi-scale target detection. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of the logic flow of the present invention.

[0034] Figure 2 This is a framework diagram of the method of the present invention.

[0035] Figure 3 This is a structural diagram of the FCM of the present invention.

[0036] Figure 4 This is a structural diagram of the HCCM of the present invention. Detailed Implementation

[0037] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0038] This embodiment discloses a small target detection method based on hierarchical feature fusion and calibration, such as Figure 1 As shown; the method is logically implemented as follows: Figure 2As shown, the original image is first input into the ResNet50 backbone network, generating multi-level feature maps through feature extraction. Compared to the baseline model, this embodiment specifically retains the S2-level feature map, as it contains rich small target feature information, which is crucial for improving the performance of small object detection. Subsequently, the highest-level feature map is processed by a self-attention mechanism to effectively capture long-distance dependencies, generating an enhanced F5 feature map. In the multi-scale feature fusion stage, a feature pyramid-like structure is adopted, innovatively introducing inverse wavelet pooling (IWP) to replace the traditional bilinear upsampling, significantly alleviating the edge blurring problem. Through the feature completion module (FCM), the small target information in the S2 feature map is systematically fused into the high-level features, and multi-level feature learning is achieved through global branches, large-scale branches, and local branches. The hierarchical complementary calibration module (HCCM) further optimizes the interaction between features at different levels, effectively alleviating semantic misalignment and feature redundancy problems, and ultimately generating a more discriminative feature representation. In the decoder stage, the learnable target query interacts with the encoded features through a cross-attention mechanism, gradually optimizing the prediction results. Finally, the Hungarian algorithm is used to directly output bounding boxes and class predictions, completing the end-to-end detection process. This architecture, through systematic improvements, significantly enhances the model's performance in small object detection tasks. Specifically, it includes the following steps:

[0039] 1) In Figure 2 Given an image, feed the image into the backbone;

[0040] a. The image undergoes an initial coarse preprocessing layer. Given an RGB image, it first passes through a relatively large 7x7 convolutional kernel with a stride of 2. The purpose of this step is to quickly reduce the spatial resolution of the feature map (halving its size) while initially extracting some very low-level features such as edges, color, and texture. Following this, a 2x2 max pooling layer with a stride of 2 further compresses the feature map size. This initial stage lays the foundation for the size and basic features of subsequent depth processing.

[0041] b. Image information enters the core of ResNet50—four convolutional layers consisting of residual blocks (typically called conv2_x to conv5_x). Each layer group consists of multiple stacked residual blocks (specifically [3, 4, 6, 3] blocks), and each layer group begins with a convolution operation with a stride of 2, halving the spatial size of the feature map while doubling the number of channels to achieve a spatial-to-depth transformation. The residual block is the soul of ResNet, and its core idea is "Shortcut Connection" or "Skip Connection." Within each block, the input propagates along two paths: one path goes through two or more convolutional layers (in ResNet50, this is a "bottleneck structure" of 1x1 dimensionality reduction, 3x3 convolution, and 1x1 dimensionality increase), performing a non-linear transformation; the other path is directly passed through the shortcut connection. Finally, the outputs of the two paths are summed element-wise. This design cleverly solves the gradient vanishing and degradation problems that occur in extremely deep networks, allowing the network to be effectively trained to 50 layers or even deeper, thus enabling it to learn very complex feature hierarchies. During this process, the features undergo profound changes: in earlier layers (such as conv2_x), the network primarily captures mid-level features, such as more complex textures and patterns; in intermediate layers (such as conv3_x, conv4_x), the features become more abstract, beginning to correspond to parts of objects, such as wheels, animal eyes, etc.; and in the final layers (conv5_x), the features are highly semantic, capable of representing complete objects or their salient components.

[0042] c. Finally, after processing by all convolutional layers, a set of feature maps with a small size (e.g., 7x7) but a high channel count (2048 dimensions) is obtained. At this point, the feature extraction work for the backbone network is complete. This set of feature maps is no longer the original pixels, but a highly condensed, information-rich tensor that represents the most essential semantic content of the input image. Depending on the downstream task, this set of features will be fed into different head networks.

[0043] 2) The specific implementation process of constructing a feature pyramid network similar to the extracted multi-scale features in the encoder stage is as follows:

[0044] a. After extracting multi-level feature maps, feature maps from stages S2, S3, S4, and S5 are typically retained, while features from stage S1 are discarded. This is mainly because the S1 feature map has a high spatial resolution, and retaining it would significantly increase computational and memory overhead, hindering efficient model training and inference. Subsequently, a self-attention mechanism is applied to the S5 feature map to obtain the enhanced feature map F5. This mechanism can effectively model the global dependencies within features, capture long-range contextual information, thereby enhancing the model's ability to understand complex scenes and improving overall perceptual performance.

[0045] b. After preliminary processing of the multi-level feature maps, multi-scale features are fused following a top-down path. During the fusion process, inverse wavelet pooling is used for upsampling. Inverse wavelet pooling, with its excellent frequency domain preservation properties, can effectively suppress edge distortion and improve the integrity of the feature representation. For example... Figure 3 As shown, the S2 feature map enters the FCM module, which passes rich small target feature information to the high-level feature map, and at the same time obtains richer scale feature information through feature learning from local to global. Figure 3 As shown, the specific implementation process can be described as follows: Shallow feature maps (such as layer S2) have higher spatial resolution and can better preserve the fine structural features of small targets. The FCM module adopts a three-level input architecture, receiving shallow features (F... L ), mid-level features (F M ) and deep features (F H This method innovatively employs Spatially Separated and Deformable Convolution (SPDC) on F... L Feature extraction is performed, and this convolutional operation effectively alleviates the problem of lost feature details by avoiding convolution and pooling operations with strides greater than 1 in traditional methods. Subsequently, the extracted small target features are combined with F... M F H Cross-layer feature concatenation is performed to achieve complementary fusion of multi-scale features. Simultaneously, feature representations of targets at different scales are learned through three parallel branches, ultimately achieving collaborative optimization of global features. This design not only enhances the feature preservation ability of small targets but also maintains the network's detection performance across different scales. The entire process can be described as follows: First, SPDC is used to combine the extracted small target features with F... M ,F H The process of complementary feature fusion can be represented as:

[0046] ,

[0047] Subsequently, the fused feature map is input into the Split-OmniKernel FusionBlock (SOFB) for multi-scale feature enhancement. The processing flow of this module can be divided into three key steps: First, the input feature F is enhanced through a 1×1 convolution. i (F i ∈R C×H×W First, a dimensionality transformation is performed to map the feature channel to the target feature space. Second, a channel segmentation strategy is adopted to divide the feature channel into two subspaces, eC and (1-e)C (where e∈[0,1] is the learnable channel segmentation ratio). This design significantly reduces computational complexity. This process can be formally represented as:

[0048]

[0049] Where F1∈Re C×H×W F2∈R (1-e)C×H×W These represent the segmented feature subspaces. To effectively address the scale diversity of targets in the image, multi-branch parallel processing is implemented on the F2 features:

[0050] Global Branch: A cascaded dual-module feature enhancement architecture was designed, which includes a spectral channel enhancement module (SCEM) and a frequency-aware spatial refinement module (FASR). This architecture achieves three key objectives through the synergistic effect of frequency domain transformation and attention mechanism: (1) replacing the traditional large convolution kernel operation with frequency domain analysis method, significantly reducing computational complexity while maintaining full receptive field coverage; (2) achieving adaptive enhancement of key features through dual optimization of spectral attention in the channel dimension and frequency-aware refinement in the spatial dimension; (3) ensuring that the model maintains low computational overhead while improving feature expressiveness based on the design of Fast Fourier Transform (FFT) and lightweight attention mechanism. This design effectively solves the trade-off between computational efficiency and feature expressiveness in the traditional large convolution kernel method. The specific process can be summarized as follows: In the feature processing process, the input features first pass through the spectral channel enhancement module (SCEM), which enhances the features through a dual-domain channel attention mechanism. Specifically, in the frequency domain stage, the input features F global After performing a Fast Fourier Transform (FFT), a 1×1 convolution (C) is performed. 1×1 Channel attention weights are calculated using global average pooling (GAP), multiplied by the original features, and then subjected to inverse IFFT to obtain the frequency domain enhanced features F. f The entire process can be represented as:

[0051]

[0052] Where F f and F globalThese represent the output after frequency domain channel attention processing and the input of the Global branch, respectively. In the spatial domain stage, the final output F is obtained through channel attention modulation. SCEM The SCEM module effectively enhances the response of key feature channels through dual-channel attention in both the frequency and spatial domains. This process can be represented as:

[0053]

[0054] Among them, F SCEM This represents the final output of the SCEM module. To compensate for the spatial limitations of SCEM, a Frequency Aware Spatial Refinement (FASR) module is subsequently introduced. The FASR module calculates spatial attention weights in the frequency domain, enabling the model to adaptively focus on key spectral components (such as high-frequency edges or low-frequency structural features). The final output F FASR It is obtained by multiplying the frequency domain spatial attention with the original features. The entire process can be described as follows:

[0055]

[0056] The cascaded design of SCEM and FASR achieves collaborative optimization of channel and spatial dimensions: SCEM is responsible for feature selection in the channel dimension (frequency domain + spatial domain), while FASR focuses on spectral component optimization in the spatial dimension, together completing the modeling of global features.

[0057] Large branches: such as Figure 3 As shown, an efficient large receptive field design strategy is employed: multi-scale contextual information is captured by combining depthwise separable convolutions of different shapes in parallel. Specifically, three convolutional operations are deployed simultaneously at the bottleneck: a regular K×K depthwise convolution, a 1×K horizontal strip convolution, and a K×1 vertical strip convolution. This design borrows from the idea of ​​strip-shaped self-attention, capturing both the global information of the square receptive field and the strip-shaped contextual features in the horizontal and vertical directions. Experiments have confirmed that in drone images where small objects are predominant, a K value of 31 yields the best results.

[0058] Local Branch: To further enhance the multi-level feature representation capability, a local branch is introduced on top of the global branch and the large branch. This branch employs a 1×1 depthwise separable convolutional structure, specifically designed for extracting fine-grained local features, thus effectively compensating for the shortcomings of the global branch and the large branch in capturing microscopic detail features. Through this hierarchical architecture design, the system achieves multi-scale feature learning from global context to local details, significantly improving the model's representation capability.

[0059] c. The fused feature map is then fused again with the original multi-level feature map into the HCCM module. This step replenishes the information lost during feature fusion. Simultaneously, the HCCM module also allows for feature interaction between modules and resolves semantic misalignment after feature fusion. The specific implementation process can be described as follows: Figure 4 As shown, the HCCM module adopts a dual-branch structure design, with each branch containing a BFRU unit to process feature inputs at different levels: deep features F d (Low resolution, rich in semantic information) and shallow features F s (High resolution, rich in edge details). Two BFRU units employ opposite processing orders to achieve bidirectional feature interaction: on the one hand, they utilize deep semantic information to calibrate shallow detail features; on the other hand, they enhance deep semantic representations through shallow detail features. This design not only effectively solves the semantic misalignment problem in feature fusion but also eliminates redundant information through the feature refinement process. The specific implementation of the BFRU unit includes the following key steps: first, feature dimensions are adjusted through 1×1 convolution, followed by activation functions to generate feature weights F1' and F2' to highlight important regions. This process can be represented as:

[0060]

[0061] Next, using the generated weights F1 ' The key information of the original feature F1 is enhanced, and the enhanced features are used to supplement the missing information in F2, while suppressing redundant and poorly aligned feature regions. To maintain information integrity, the original feature F1 is ultimately retained for concatenation. This process can be formally represented as:

[0062]

[0063] Where ⊙ denotes element-wise multiplication, (1-F1) ' This is used to filter redundant features. This dynamic weight adjustment mechanism enables intelligent feature complementarity, highlighting the strengths of each feature while accurately compensating for their weaknesses. Finally, the module concatenates the bidirectionally calibrated features and further extracts refined feature representations through 3×3 convolution. This design effectively improves the accuracy of small target detection while maintaining feature integrity.

[0064] 3) The specific implementation process of the fused features entering the decoder is as follows:

[0065] a) First, initialize a fixed number of learnable parameters as initial query vectors, typically set to the model's default maximum number of predictable targets. These query vectors are essentially trainable embedding vectors, each encoding the location and category information of a potential target in the image. The initial query vectors are generated randomly and continuously optimized during training using gradient descent, thereby learning the distribution characteristics of different targets;

[0066] b. In each decoder layer, the initial query vector is first input to the self-attention module. This module models the relationships between query vectors by calculating their similarity, paying particular attention to the co-occurrence patterns of different targets in space. The self-attention mechanism employs a multi-head design, splitting the vector into multiple subspaces to calculate attention weights separately, and finally fusing them into a representation with global dependencies. The query vector output by this module maintains the same dimensionality as the input but incorporates richer contextual information.

[0067] c. The updated query vector enters the cross-attention module, where it interacts with the image features output by the encoder. The encoder features have already constructed a global spatial context through self-attention, while the cross-attention module, guided by the current query, extracts relevant information from the image features. Specifically, this module uses the query vector as the query (Q), and the image features as the key (K) and value (V). It aggregates key features through the calculation of attention weights, ultimately generating a target representation corresponding to each query.

[0068] d. After iterative optimization by the multi-layer decoder, the final query vector will be fed into the prediction head to generate the target detection result. The prediction head includes two branches: the class prediction branch outputs the class probability distribution (including the background class) of the target corresponding to each query through a fully connected layer and a Softmax function; the bounding box prediction branch regresses the geometric information of the target bounding box, such as center coordinates, width and height, or corner coordinates, through a multi-layer perceptron (MLP).

[0069] 4) The specific implementation process of end-to-end training and Hungarian matching of the completed network architecture is as follows:

[0070] a. During the end-to-end training phase, the complete detection network is trained using a UAV small target detection dataset. The training process employs the InnerPIoU loss function. InnerPIoU consists of InnerIoU and Powerful-IoU. InnerIoU only calculates the overlap between the predicted bounding box and the interior region of the ground truth bounding box, rather than the entire bounding box, thus eliminating the influence of background noise. InnerIoU is defined as:

[0071]

[0072] in This represents the internal shrinkage region of the predicted bounding box. This represents the contracted region inside the true bounding box. Powerful-IoU addresses the anchor box bulging problem by introducing an adaptive penalty factor based on target size and a gradient adjustment function based on anchor box quality. The adaptive penalty factor can be adjusted according to different object sizes, which addresses the issue of large object size variations in UAV images. Both are defined as follows:

[0073]

[0074]

[0075] Where dw and dh represent the absolute values ​​of the corresponding edges of the predicted bounding box and the ground truth bounding box, respectively. gt and h gt These represent the width and height of the true bounding box. Therefore, the Powerful-IoU loss function is defined as:

[0076]

[0077] The InnerPIoU loss function is defined as follows:

[0078]

[0079] b. During training, the optimal prediction result is found through Hungarian matching. The main implementation process can be described as follows: The model predicts N object boxes, but there may only be M real objects in the image (M << N). It is necessary to know which of these N predicted boxes corresponds to which real object in order to calculate the classification loss and regression loss. The Hungarian algorithm is used to find an allocation method that minimizes the total cost of all "predicted-real" pairings (defined by the classification error and the box position error). Through this one-to-one matching, each real object is assigned a "positive sample" predicted box, and the remaining predicted boxes are labeled as "negative samples" (background). The model then calculates the loss based on this allocation result and backpropagates to update the weights.

[0080] In summary, by adopting the above scheme, this invention provides a new solution to some problems existing in small target detection. To alleviate the problems of blurred edges and artifacts of small targets caused by traditional upsampling methods, this framework uses an inverse wavelet pooling module to replace the conventional upsampling operation. Simultaneously, a Feature Completion Module (FCM) is constructed to effectively supplement the rich small target feature information in the lower-level feature maps to the higher-level feature maps, and enhances the discriminative ability of feature representation through a feature learning mechanism from local to global. To address the semantic misalignment problem between feature maps of different levels during feature fusion, this method further proposes a Hierarchical Complementary Calibration Module (HCCM). This module adopts a dual-branch structure, and based on mutual learning between features at different levels, it suppresses and calibrates the misaligned parts between features. These measures improve the accuracy of small target detection while maintaining good practicality in remote sensing target detection, possessing practical promotion and application value.

[0081] The above-described embodiments are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Therefore, any changes made in accordance with the shape and principle of the present invention should be covered within the protection scope of the present invention.

Claims

1. A small target detection method based on hierarchical feature fusion and calibration, characterized in that, This includes the backbone network, building a feature fusion calibration network, building a decoder, performing end-to-end training, and optimizing the loss function. Step A1: After inputting the image into the ResNet50 backbone network, S2, S3, S4 and S5 feature maps are obtained. The S5 feature map is processed by the self-attention mechanism to capture long-distance dependencies and generate the enhanced F5 feature map. Step A2: Construct a feature fusion calibration network: Design an inverse wavelet pooling module. Based on the feature pyramid network, feature maps at different levels are restored to a uniform target size through inverse wavelet pooling to fuse feature maps and solve the problem of jagged edges on small targets. The design feature completion module transmits small target feature information from the low-level feature map to the high-level feature map to supplement the missing small target features; A hierarchical complementary calibration module is designed to optimize the feature representation between feature maps of different levels and solve the semantic misalignment problem after feature map fusion. Step A3: Construct a decoder. The learnable target query interacts with the encoded features through a cross-attention mechanism to gradually optimize the prediction results. Step A4: Perform end-to-end training and loss function optimization. Use the Hungarian algorithm to directly output bounding boxes and class predictions to complete the end-to-end detection process. Improve the IOU loss function to solve the problem of the predicted box increasing in size during training, thereby improving the detection accuracy.

2. The small target detection method based on hierarchical feature fusion and calibration according to claim 1, characterized in that, In step A2, the inverse wavelet pooling module includes the following steps: Step A211: Using a feature pyramid network as the core architecture, a top-down feature fusion path is constructed. To improve upsampling quality, inverse wavelet pooling is used instead of the traditional interpolation method. Specifically, the inverse wavelet transform uses the Haar wavelet basis to complete the two-dimensional discrete wavelet transform. The mathematical definition of is as follows: ， Where x represents a continuous variable in the spatial domain; inverse wavelet pooling is applied to upsample the F5 feature map by a factor of 2 to align its spatial dimensions with those of the F4 feature map; Step A212: The upsampled F5 features and the features extracted in the S4 stage are concatenated and fused along the channel dimension using the Concat operation. Continuing along the top-down fusion path, the optimized S4 features are amplified to a spatial size that matches the S3 features by using the same inverse wavelet pooling upsampling operation.

3. The small target detection method based on hierarchical feature fusion and calibration according to claim 1, characterized in that, In step A2, the feature completion module includes the following steps: Step A221, Receive shallow features (F L ), mid-level features (F M ) and deep features (F H ), in which Space-to-Depth Convolution (SPDC) is used to apply F L Feature extraction alleviates the problem of lost feature details. The complementary fusion process of features can be represented as follows: ， in This indicates the result of the fusion; Step A222: Perform multi-scale feature enhancement using Split-OmniKernel Fusion Block (SOFB), which can be divided into three steps: First, apply a 1×1 convolution to the input feature F. i (F i ∈R C×H×W First, a dimensional transformation is performed to map the feature channel to the target feature space. Second, a channel segmentation strategy is used to divide the feature channel into two subspaces, eC and (1-e)C. This process can be formally represented as: F1,F2=Split(F i ), Where F1∈R eC×H×W F2∈R (1-e)C×H×W e∈[0,1] is the learnable channel segmentation ratio, C is the number of channels, H is the height of the feature map, and W is the width of the feature map; Step A223: Perform three-branch parallel processing on the F2 feature: global branch, large branch, and local branch; Step A224: After the results of the three-branch parallel processing are added element-wise, they are fed into a 1×1 convolution, then connected with the F1 features, and a 1×1 convolution is performed again to obtain complementary features.

4. The small target detection method based on hierarchical feature fusion and calibration according to claim 3, characterized in that, In step A223, the global branch is a cascaded dual-module feature enhancement architecture, which sequentially includes a Spectrum Channel Enhancement (SCEM) module and a Frequency Aware Spatial Refinement (FASR) module: The SCEM module enhances features through a dual-domain channel attention mechanism. Specifically, in the frequency domain stage, it enhances the input feature F... global After performing a Fast Fourier Transform (FFT), a 1×1 convolution (C) is performed. 1×1 Channel attention weights are calculated using global average pooling (GAP), multiplied by the original features, and then subjected to inverse IFFT to obtain the frequency domain enhanced features F. f The entire process can be represented as: ， Where F f and F global These represent the output after frequency domain channel attention processing and the input of the global branch, respectively. In the spatial domain stage, the final output F is obtained through channel attention modulation. SCEM The SCEM module effectively enhances the response of key feature channels through dual-channel attention in both the frequency and spatial domains. This process can be represented as: ， Among them, F SCEM This represents the final output of the SCEM module; The FASR module computes spatial attention weights in the frequency domain, enabling the model to adaptively focus on key spectral components. The output of the FASR module, F... FASR Obtained through the product of frequency domain spatial attention and the original features, the entire process can be described as follows: ， The cascaded design of SCEM and FASR achieves collaborative optimization of channel and spatial dimensions: SCEM performs feature selection in the channel dimension, while FASR optimizes the spectral components in the spatial dimension, together realizing the modeling of global features.

5. The small target detection method based on hierarchical feature fusion and calibration according to claim 3, characterized in that, In step A223, the large branch captures multi-scale contextual information by combining depthwise separable convolutions of different shapes in parallel: In terms of specific implementation, three convolutional operations are deployed simultaneously at the bottleneck: K×K depthwise convolution, 1×K horizontal strip convolution, and K×1 vertical strip convolution. This design draws on the idea of ​​strip self-attention, which can capture global information of the square receptive field and obtain strip contextual features in the horizontal and vertical directions.

6. The small target detection method based on hierarchical feature fusion and calibration according to claim 3, characterized in that, In step A223, the local branch adopts a 1×1 depthwise separable convolution structure to extract fine-grained local features, which supplements the shortcomings of the global branch and the large branch in capturing micro-detail features. Through this hierarchical architecture design, multi-scale feature learning from global context to local details is systematically realized, which significantly improves the model's representation ability.

7. The small target detection method based on hierarchical feature fusion and calibration according to claim 1, characterized in that, In step A2, the hierarchical complementary calibration module adopts a dual-branch structure design, with each branch containing a bidirectional feature recalibration unit (BFRU) to process feature inputs at different levels: deep features F d and shallow features F s The two BFRUs adopt opposite processing orders to achieve bidirectional feature interaction: on the one hand, they use deep semantic information to calibrate shallow detail features, and on the other hand, they enhance deep semantic representation through shallow detail features. This design can not only effectively solve the semantic misalignment problem in feature fusion, but also eliminate redundant information through the feature refinement process.

8. The small target detection method based on hierarchical feature fusion and calibration according to claim 7, characterized in that, The specific implementation of the bidirectional feature recalibration unit (BFRU) involves three key steps: Step B1: The feature dimensions are adjusted using a 1×1 convolution, followed by activation functions to generate feature weights F1' and F2' to highlight important regions. This process can be represented as follows: ， Step B2: The generated weights F1' are used to enhance the key information of the original feature F1, and the enhanced features are used to supplement the missing information in F2. Simultaneously, redundant and poorly aligned feature regions are suppressed. To maintain information integrity, the feature is further connected to the original feature F1. This process can be formally represented as: ， Where ⊙ represents element-wise multiplication, and (1-F1') is used to filter redundant features; this dynamic weight adjustment mechanism realizes intelligent complementarity of features, which can highlight the advantages of each feature and accurately make up for the shortcomings of the other. Step B3: Concatenate the bidirectionally calibrated features and further extract refined feature representations through 3×3 convolution.

9. The small target detection method based on feature fusion and calibration according to claim 1, characterized in that, Building a decoder involves the following steps: Step C1: Generate an initial query vector through random initialization and optimize it step by step during training to capture the distribution characteristics of different targets; Step C2: In each decoder layer, the initial query vector first enters the self-attention module. The self-attention mechanism captures their co-occurrence relationship in space. This process is achieved through a multi-head attention mechanism, which divides the query vector into multiple subspaces, calculates attention weights for each, and finally aggregates them into a global relation representation. The output of the self-attention mechanism is the updated query vector, which has the same dimension as the input but contains richer contextual information. Step C3: The updated query vector enters the cross-attention module and interacts with the global image features output by the encoder. The cross-attention module uses the query vector as a "guide" to extract information related to the current query from the feature map and generate the target feature representation corresponding to the query. Step C4: After multiple layers of decoder iteration, the final query vector is input into the prediction head to generate the target detection result.

10. The method for detecting surface defects in industrial products based on feature coupling according to claim 1, characterized in that, End-to-end training and loss function optimization include the following steps: Step D1: Train the detection network end-to-end using the UAV small target detection dataset, and use the InnerPIoU loss function during the training process; Step D2: During training, the optimal detection result is found through Hungarian matching. The main implementation process is as follows: The model predicts N target boxes, but there are only M real targets in the image (M << N). The Hungarian algorithm is used to find an allocation method that minimizes the total cost of all "predicted-real" pairings. Through this one-to-one matching, each real target is assigned a "positive sample" predicted box, and the remaining predicted boxes are marked as "negative samples". The model calculates the loss based on this matching result and backpropagates to update the weights.

Images