A small target detection system and method based on context awareness and multi-scale feature fusion

Abstract

This invention discloses a small target detection system and method based on context awareness and multi-scale feature fusion. This invention effectively improves the detection accuracy and robustness of small targets in complex scenes, significantly mitigating the false negatives and missed detections caused by weak features, background interference, and scale variations in traditional methods. By enhancing the global semantic consistency of features and suppressing noise, the boundaries and texture details of small targets are clearly preserved. Simultaneously, it overcomes the geometric misalignment and feature dilution phenomena in multi-scale fusion, achieving precise alignment and adaptive fusion of cross-layer features. Ultimately, while maintaining detection sensitivity, it reduces the computational burden and improves the system's practicality and stability in real-world scenarios.

Description

Technical Field

[0001] This invention relates to the field of image processing and target detection technology, specifically to a small target detection system and method based on context awareness and multi-scale feature fusion. Background Technology

[0002] In the field of computer vision, small object detection is an extremely challenging task. Its core difficulty lies in the small pixel proportion of the target, the scarcity of usable visual features, and the weak semantic information. In practical applications such as remote sensing image analysis, autonomous driving scene understanding, industrial defect detection, and video surveillance, small objects are often highly mixed with complex backgrounds and severely affected by factors such as changes in lighting, varying scales, and partial occlusion. Traditional detection methods often rely on hand-designed features, which have limited expressive power and are difficult to adapt to complex and ever-changing real-world scenes, resulting in low recall and high false positive rates when dealing with small objects.

[0003] With the rapid development of deep learning technology, object detection methods based on convolutional neural networks have made significant progress. These methods typically employ deep backbone networks to extract multi-level features and fuse information at different scales through structures such as feature pyramids. However, for small targets, their key features are easily lost in deep networks due to operations such as pooling, leading to the dilution of semantic information. Although introducing attention mechanisms or context modeling modules can aggregate global information to some extent to enhance feature representation, these methods often incur huge computational overhead and memory consumption, making efficient deployment on resource-constrained edge devices difficult. Furthermore, most context modules process all region information equally, failing to effectively distinguish key target regions from interfering backgrounds, which may introduce unnecessary noise while enhancing semantics.

[0004] On the other hand, multi-scale feature fusion is a key approach to improving the performance of small target detection. However, existing fusion strategies, such as simple addition or concatenation, often ignore the significant semantic gaps and geometric misalignments between feature maps at different levels. This direct fusion leads to feature conflicts and dilution, resulting in decreased localization accuracy, especially for small targets with blurred boundaries. Although some studies have attempted to align features using deformable convolutions or adaptive weights, these methods still struggle to accurately predict pixel-level offsets when dealing with extremely small targets or drastic scale changes. Furthermore, the fusion process is susceptible to aliasing artifacts and high-frequency noise generated during upsampling, further compromising the integrity of detailed information. Therefore, designing a detection framework that can effectively enhance the semantics of small targets, achieve accurate geometric alignment, and suppress noise interference while controlling computational complexity remains a pressing technical challenge in this field. Summary of the Invention

[0005] To address the technical problems mentioned above, the present invention provides the following technical solution: A small target detection system based on context awareness and multi-scale feature fusion includes: The feature extraction module is used to perform multi-level encoding on the input image to obtain feature sets at different scales; The context-aware module, connected to the feature extraction module, is used to obtain globally consistent enhanced features based on the feature maps at each scale output by the feature extraction module. The alignment and fusion module connects to the context-aware module. It is used to obtain aligned multi-scale fused features based on the enhanced features, and to obtain target classification and regression prediction results based on the aligned multi-scale fused features.

[0006] Preferably, the workflow of the context-aware module includes: Based on the feature map at each scale of the input, the response level at different spatial locations is determined by location-related gating branches to obtain weighted features; Based on the weighted features, long-distance dependency information is obtained through multi-path state recursive scanning. Based on the results of multi-path state recursive scanning, enhanced features are obtained through weighted aggregation of channel and spatial dimensions.

[0007] Preferably, the workflow of the alignment and fusion module includes: A difference map is calculated based on low-level features and upsampled high-level features. Based on the difference map, the pixel-level displacement field is predicted, and the displacement field is obtained by truncating the amplitude of the displacement field. Based on the displacement field, high-level features are aligned with low-level features through grid sampling operations to obtain the aligned high-level features.

[0008] Preferably, before calculating the difference map, an anti-aliasing upsampling step is included, comprising: Based on the upsampled high-level features, smoothed high-level features are obtained through low-pass convolutional filtering. Based on the smoothed high-level features, the channel dimensions of the high-level features are adjusted by channel-corrected convolution; Based on the high-level features after channel correction, high-level features for calculating the difference map are obtained by upsampling to the scale of the low-level features through bilinear interpolation.

[0009] Preferably, the workflow of the alignment and fusion module further includes a difference-guided dynamic feature fusion step: Based on the low-level features and the aligned high-level features, a difference map is calculated. Based on the difference map, generate channel-independent dynamic convolutional kernels; Based on dynamic convolution kernels, adaptive weighted fusion is performed on aligned high-level features and low-level features to obtain fused features.

[0010] Preferably, the process for obtaining target classification and regression prediction results based on aligned multi-scale fusion features includes: Based on the aligned multi-scale fusion features, edge map, detail map and signal-to-noise ratio map are obtained; Pixel-level fusion weights are generated based on the edge map, detail map, signal-to-noise ratio map, and the difference map output by the alignment and fusion module. Based on pixel-level fusion weights, the features are refined to obtain the final features used for prediction.

[0011] Preferably, the edge map is obtained using the Sobel operator; the detail map is obtained using separable convolution; and the signal-to-noise ratio map is obtained by calculating the variance of the local region.

[0012] This invention also provides a small target detection method based on context awareness and multi-scale feature fusion, the method being applied to the aforementioned system, comprising: Multi-level encoding is performed on the input image to obtain feature sets at different scales; Based on the feature map at each scale output by the feature extraction module, obtain the enhanced features with global consistency; Based on the enhanced features, obtain the aligned multi-scale fusion features; Based on the enhanced features, aligned multi-scale fusion features are obtained, and based on the aligned multi-scale fusion features, target classification and regression prediction results are obtained.

[0013] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention effectively improves the detection accuracy and robustness of small targets in complex scenes, significantly mitigating the false negatives and missed detections caused by weak features, background interference, and scale variations in traditional methods. By enhancing the global semantic consistency of features and suppressing noise, the boundaries and texture details of small targets are clearly preserved. Simultaneously, it overcomes the geometric misalignment and feature dilution phenomena in multi-scale fusion, achieving precise alignment and adaptive fusion of cross-layer features. Ultimately, while maintaining detection sensitivity, it reduces the computational burden and improves the system's practicality and stability in real-world scenarios. Attached Figure Description

[0014] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a schematic diagram of the model structure according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of the context-aware module according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the alignment and fusion module according to an embodiment of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0018] Example 1 This embodiment provides a small target detection system based on context awareness and multi-scale feature fusion, including: a feature extraction module, a context awareness module, an alignment and fusion module, and a fusion prediction module. The following will describe in detail how this invention solves technical problems in practical applications, using this embodiment as an example.

[0019] First, the feature extraction module uses mainstream pre-trained models to construct backbone network features, performs multi-level encoding on the input image, and obtains feature sets at different scales. ,like These correspond to shallow, medium, and deep features. Shallow features have high spatial resolution and rich detail information, while deep features contain stronger semantic expressive power and can be used to distinguish different categories of targets.

[0020] In the context-aware phase, this embodiment will output each scale feature map from the feature extraction module. As input to the context-aware module, the features at this scale are augmented with context to obtain enhanced features. This module selectively incorporates global contextual information beneficial to small targets through multi-path state-based scanning and weighted aggregation, while suppressing noise interference from complex backgrounds. This provides feature representations with global consistency and local detail for subsequent multi-scale fusion. In the multi-scale feature fusion stage, a difference-guided alignment and fusion module is introduced to enhance the boundaries and texture details of small targets while suppressing background noise. Finally, the multi-scale fused features are input into the detection head to complete classification and regression predictions at each scale.

[0021] The context-aware module employs a lightweight context-aware method based on a state-space model to enhance context-aware features. The context-aware module, for example... Figure 2 As shown.

[0022] In real-world scenarios, the detection accuracy of small targets is often limited by their weak features and severe background interference. Most existing methods compensate for the scarcity of features in small targets by introducing a context module. However, this method incurs significant computational costs. To address these issues, this embodiment proposes a context-aware module with linear complexity. This module borrows the selective scanning idea from the Mamba model, introduces a state-space model, and uses convolution operators for approximation. It models the long-distance dependence of single-scale features to obtain global semantics, thereby improving the problems of weak semantics and blurred boundaries of small targets.

[0023] During the selection phase, channel projection is performed on the input features, and a location-related gating branch is introduced to determine the response level at different spatial locations. For example, for regions with abrupt boundary changes or clustered targets, the gating branch will increase the attention to their context, and vice versa to avoid the spread of irrelevant information. This mechanism can increase the importance of key regions and reduce noise introduced by the context. The formula is as follows: in, For output features, For input features, For the gated weight matrix, This indicates element-wise multiplication. For position-dependent gating functions, It refers to spatial location.

[0024] During the scanning phase, single-scale features are rearranged into a spatial sequence according to a preset path. Long-distance dependencies are obtained using a state recursion approach. To mitigate the feature offset problem introduced by a single path, this module employs a multi-path scanning method. In addition to bidirectional scanning in the horizontal and vertical directions, a diagonal scanning path is further added to uniformly cover structural information in different directions. The scanning process can be represented by a state-space model as follows: in, Indicates time t state, A It is the state transition matrix. To control the input.

[0025] In the aggregation phase, the scanning results of each path are fused. Considering the different structural sensitivities of different paths, a weighted approach is used during fusion. Path weights are generated in the channel dimension, assigning higher weights to paths more relevant to the foreground semantic response. In the spatial dimension, the weights of regions rich in detail are increased, while responses from irrelevant regions are suppressed to avoid the accumulation of redundant information. Feature aggregation can be expressed by the following formula: in, These are weighting coefficients. The scan results are for different paths. This is the final fusion feature.

[0026] Finally, the features are integrated through normalization and convolution, and the original feature stream is connected in a residual manner to obtain more stable and globally consistent context-enhanced features.

[0027] The alignment and fusion module employs a difference-guided multi-scale feature alignment and fusion method to align and fuse the enhanced context features. The alignment and fusion module, as shown in the example... Figure 3 As shown.

[0028] To alleviate the problems of geometric misalignment and feature dilution in cross-scale fusion, this embodiment constructs a difference-guided feature fusion strategy. This process explicitly predicts the displacement field based on the difference between cross-layer features, aligns the upsampled high-level features with the low-level features, and introduces an anti-aliasing upsampling method during the alignment process to suppress aliasing and pseudo-high-frequency propagation caused by sampling. Then, dynamic fusion is performed guided by the difference map, adaptively weighting the aligned high-level features with the original low-level features to enhance boundary and fine-grained feature information, while suppressing false detections caused by background interference.

[0029] (1) Construction of difference diagram Let the low-level features be L The high-level features after upsampling are H First, calculate the difference plot. This serves as a guide for predicting pixel-level displacement. Then, this displacement is normalized and its amplitude truncation is used to obtain a stable offset value. The displacement normalization formula is: in, This represents the difference plot after normalization. This represents the maximum value in the original feature map.

[0030] To avoid overcorrection caused by excessive displacement, amplitude truncation is used to limit the displacement value within a certain range, ensuring the stability of feature alignment. Amplitude truncation can be achieved using the following formula: in, and These are the minimum and maximum values ​​of the amplitude truncation, respectively, and represent the displacement field after amplitude truncation.

[0031] Finally, the processed displacement field Used for mesh sampling operations, aligning high-level features to low-level features. Mesh sampling aligns high-level features to low-level features through pixel-level displacement. H Mapping to the space of low-level features yields aligned high-level features. This provides a geometrically consistent alignment representation for subsequent feature fusion, and its formula can be expressed as: in, It is a grid sampling operation that extracts high-level features. H According to the displacement field after amplitude truncation Sampling is performed to obtain the aligned feature map. .

[0032] (2) Anti-sampling In the process of multi-scale feature fusion, sampling operations may introduce artifacts and high-frequency noise. These disturbances will propagate layer by layer in the top-down feature fusion process, affecting the quality of the final fused features. This embodiment introduces an anti-aliasing upsampling method to reduce the propagation of high-frequency noise and ensure the stability of feature fusion.

[0033] First, a 3×3 convolution kernel is used to perform low-pass filtering on the upsampled features to approximate a smoothing operation. This step aims to effectively suppress high-frequency noise and prevent further propagation of noise. The convolution operation can be expressed as the following formula: in, This indicates that a 3×3 convolution kernel is used for smoothing. This is the smoothed feature map.

[0034] Then, a 1×1 convolution is used to correct the channels, adjust the number of channels, and achieve feature fusion across the channel dimensions to ensure consistency. The formula can be expressed as: in, This represents the intermediate feature map.

[0035] Finally, bilinear interpolation is used to upsample the features, adjusting the smoothed and corrected feature map to the scale of the low-level feature map. Bilinear interpolation smooths image edges by performing linear interpolation between pixels, reducing artifacts that may occur during the interpolation process. This step can be expressed as the following formula: in, For bilinear interpolation, This is the feature map obtained after upsampling.

[0036] (3) Difference-guided dynamic feature fusion To improve the accuracy of small target detection in complex scenes, this embodiment introduces a difference-guided dynamic feature fusion method, which can effectively preserve the edge contours and detailed information of small targets.

[0037] After cross-scale feature alignment, a difference-guided dynamic fusion method is introduced to fully preserve details such as the edge contours of small targets, thereby improving the quality of the fused features. First, low-level features are calculated... L Aligned high-level features Differences between To measure the degree of difference between features at two scales, a difference map is used. D This reflects the magnitude of structural changes at each pixel location. To adaptively adjust the convolution weights based on local differences, a channel-independent 3×3 dynamic convolution kernel is generated using the difference map. Specifically, the convolution weights for each pixel are controlled by the difference map; regions with larger differences are given greater convolution weights to highlight the edges and texture information of the target, while regions with smaller differences have reduced convolution weights to weaken irrelevant background information. The formula can be expressed as: in, For each pixel position Dynamic convolution kernel at the location, This is the convolution weighting function obtained from the difference map. The convolution calculation process can be represented as: in, This is the feature map obtained through dynamic convolution.

[0038] To further enhance boundary information and suppress background interference, the alignment and fusion module introduces a gating mechanism to assign different weights to each pixel, thereby controlling the contribution of different features and ensuring that boundary and detail information are enhanced while background and noise are suppressed. This module mainly consists of three parts: an edge map, a detail map, and a signal-to-noise ratio map.

[0039] The edge map is obtained through the Sobel operator, which can effectively capture the boundary and contour of the target: in, For the first p Edge map values ​​of each pixel.

[0040] Detail images are obtained through separable convolutions and are used to extract subtle texture information from the target: in, For detailed images, This is a separable convolution.

[0041] The signal-to-noise ratio (SNR) map is obtained by calculating the variance of a local region. It is mainly used to assess the noise level of that region. A higher variance within a region indicates that the region contains rich representations, while a lower variance indicates that it contains more irrelevant information. in, This is a signal-to-noise ratio diagram. This is a local variance calculation operation.

[0042] After obtaining the above auxiliary map, it is combined with the difference map and uniformly input into the gating module to generate pixel-level fusion weights and complete feature refinement.

[0043] The model framework constructed by the system in this embodiment is as follows: Figure 1 As shown.

[0044] Example 2 This embodiment also provides a small target detection method based on context awareness and multi-scale feature fusion. The steps include: performing multi-level encoding on the input image to obtain feature sets at different scales; obtaining enhanced features with global consistency based on the feature maps of each scale output by the feature extraction module; obtaining aligned multi-scale fusion features based on the enhanced features; obtaining aligned multi-scale fusion features based on the enhanced features; and obtaining target classification and regression prediction results based on the aligned multi-scale fusion features.

[0045] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A small target detection system based on context awareness and multi-scale feature fusion, characterized in that, include: The feature extraction module is used to perform multi-level encoding on the input image to obtain feature sets at different scales; The context-aware module, connected to the feature extraction module, is used to obtain globally consistent enhanced features based on the feature maps at each scale output by the feature extraction module. The alignment and fusion module connects to the context-aware module. It is used to obtain aligned multi-scale fused features based on the enhanced features, and to obtain target classification and regression prediction results based on the aligned multi-scale fused features.

2. The small target detection system based on context awareness and multi-scale feature fusion according to claim 1, characterized in that, The workflow of the context-aware module includes: Based on the feature map at each scale of the input, the response level at different spatial locations is determined by location-related gating branches to obtain weighted features; Based on the weighted features, long-distance dependency information is obtained through multi-path state recursive scanning. Based on the results of multi-path state recursive scanning, enhanced features are obtained through weighted aggregation of channel and spatial dimensions.

3. The small target detection system based on context awareness and multi-scale feature fusion according to claim 1, characterized in that, The workflow of the alignment and fusion module includes: A difference map is calculated based on low-level features and upsampled high-level features. Based on the difference map, the pixel-level displacement field is predicted, and the displacement field is obtained by truncating the amplitude of the displacement field. Based on the displacement field, high-level features are aligned with low-level features through grid sampling operations to obtain the aligned high-level features.

4. The small target detection system based on context awareness and multi-scale feature fusion according to claim 3, characterized in that, Before calculating the difference map, an anti-aliasing upsampling step is also included, including: Based on the upsampled high-level features, smoothed high-level features are obtained through low-pass convolutional filtering. Based on the smoothed high-level features, the channel dimensions of the high-level features are adjusted by channel-corrected convolution; Based on the high-level features after channel correction, high-level features for calculating the difference map are obtained by upsampling to the scale of the low-level features through bilinear interpolation.

5. The small target detection system based on context awareness and multi-scale feature fusion according to claim 3, characterized in that, The workflow of the alignment and fusion module also includes a difference-guided dynamic feature fusion step: Based on the low-level features and the aligned high-level features, a difference map is calculated. Based on the difference map, generate channel-independent dynamic convolutional kernels; Based on dynamic convolution kernels, adaptive weighted fusion is performed on aligned high-level features and low-level features to obtain fused features.

6. The small target detection system based on context awareness and multi-scale feature fusion according to claim 1, characterized in that, The process of obtaining target classification and regression prediction results based on aligned multi-scale fusion features includes: Based on the aligned multi-scale fusion features, edge map, detail map and signal-to-noise ratio map are obtained; Pixel-level fusion weights are generated based on the edge map, detail map, signal-to-noise ratio map, and the difference map output by the alignment and fusion module. Based on pixel-level fusion weights, the features are refined to obtain the final features used for prediction.

7. The small target detection system based on context awareness and multi-scale feature fusion according to claim 6, characterized in that, Edge maps are obtained using the Sobel operator; detail maps are obtained using separable convolution; and signal-to-noise ratio maps are obtained by calculating the variance of local regions.

8. A small target detection method based on context awareness and multi-scale feature fusion, the method being used to implement the system according to any one of claims 1-7, characterized in that, include: Multi-level encoding is performed on the input image to obtain feature sets at different scales; Based on the feature map at each scale output by the feature extraction module, obtain the enhanced features with global consistency; Based on the enhanced features, obtain the aligned multi-scale fusion features; Based on the enhanced features, aligned multi-scale fusion features are obtained, and based on the aligned multi-scale fusion features, target classification and regression prediction results are obtained.

Images