Small target detection method based on dynamic feature fusion

Abstract

The invention relates to the technical field of computer vision, in particular to a small target detection method based on separable large kernel convolution and dynamic feature fusion, and aims to solve the problem of insufficient detection precision caused by low pixel proportion, weak feature expression ability and serious background interference in a small target scene in the existing target detection technology. The core of the invention is to construct an efficient small target detection network through collaborative optimization of a separable large kernel convolution module, a multi-scale feature extraction module and a weighted feature fusion (CRC) module. The method comprises the following steps: by taking a YOLO series model as a basic framework, decomposing a large-size convolution kernel into point convolution of a channel dimension and deep convolution of a space dimension through a separable large kernel convolution module, so that the computational complexity is reduced while a receptive field is expanded, and the capability of capturing global context features of a small target is enhanced; a multi-scale feature extraction module is utilized to generate feature maps with different resolutions, a weighted feature fusion (CRC) module is combined to adaptively adjust the weight of each scale feature channel, key features are highlighted, and background noise is suppressed; and finally, the positioning loss, the classification loss and the attention loss are fused through a comprehensive loss optimization module, and high-precision small target detection is realized. According to the method, the detection recall rate and the positioning precision of the small target in the complex scene are effectively improved, and the practical value is high.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a small target detection method based on multi-scale dynamic feature fusion, which is particularly suitable for the accurate detection of small-sized targets in complex scenes. Background Technology

[0002] With the widespread application of deep learning technology in object detection, convolutional neural network-based object detection algorithms have made significant progress, such as Faster R-CNN, the YOLO series, SSD, and RetinaNet. These algorithms significantly improve the accuracy and efficiency of object detection by automatically extracting image features. However, existing technologies still face many challenges in small object detection tasks. Small objects typically have characteristics such as low pixel ratio, limited resolution, and susceptibility to interference from complex backgrounds, resulting in weak feature representation capabilities and difficulty in retaining sufficient semantic information in deep networks, thus affecting detection accuracy.

[0003] In existing technologies, object detection methods mainly include traditional methods based on handcrafted features and modern methods based on deep learning. Traditional methods typically employ handcrafted feature extraction strategies, such as sliding window methods based on edge detection or region proposal. However, these methods exhibit poor robustness in complex scenes, especially in small object detection, where the lack of adaptive feature extraction capabilities makes it difficult to achieve high-precision localization and classification.

[0004] Modern deep learning-based methods extract image features through multi-layer convolutional neural networks, significantly improving object detection performance. However, in small object detection scenarios, deep convolutional operations lead to the loss of feature information for small objects. Especially in high-level feature maps, the pixel regions of small objects are small and lack sufficient contextual information, resulting in decreased accuracy in classification and localization. Furthermore, noise, occlusion, and lighting variations in complex backgrounds further exacerbate the difficulty of small object detection. Existing multi-scale feature fusion methods, such as Feature Pyramid Networks (FPN), can alleviate the feature loss problem to some extent, but their fusion strategies are usually based on fixed weights or simple summation, making it difficult to dynamically adapt to the feature requirements of small objects, thus limiting detection performance.

[0005] In summary, existing technologies suffer from problems such as feature loss, background interference, and insufficient multi-scale feature fusion in small target detection. There is an urgent need for a method that can effectively improve the accuracy of small target detection in order to achieve high-precision target detection in complex scenes. Summary of the Invention

[0006] To address the technical problem of insufficient accuracy in small target detection in existing technologies, this invention provides a small target detection method based on multi-scale dynamic feature fusion. The aim is to significantly improve the feature representation capability and detection performance of small targets by optimizing the feature extraction network, designing a dynamic multi-scale feature fusion strategy, and introducing a context information enhancement module.

[0007] This invention proposes a small target detection method based on multi-scale dynamic feature fusion. The method optimizes small target detection performance by adding a module to the YOLO model, specifically including:

[0008] S1: Obtain a training dataset containing small targets, and use this training dataset to pre-train the YOLO model to generate a basic target detection model; extract small targets and their regional features through a separable large kernel convolution module and a context-guided attention module. The training dataset contains multiple small target categories (such as airplanes, ships, and vehicles in remote sensing images), with each category including several samples. Each sample is accompanied by a classification label and a location label. The classification label indicates the target category, and the location label indicates the coordinates of the target detection box.

[0009] S2: Input the preprocessed image data into the YOLO base model and extract the initial feature region through the separable large kernel convolution module; the separable large kernel convolution module adopts a combination of depthwise separable convolution and large kernel convolution, decomposes the large-size convolution kernel into point convolution in the channel dimension and large kernel convolution in the spatial dimension, and adaptively adjusts the convolution kernel size through a dynamic kernel selection mechanism to expand the receptive field, enhance the ability to capture the global context of small targets, and at the same time reduce computational complexity and reduce feature loss due to low pixel ratio;

[0010] S3: The initial features generated in step S2 are processed using the multi-scale feature extraction module. Feature maps of different resolutions are generated through multi-level convolution and upsampling operations. Multi-scale features of small target regions in each sample in the training set are extracted to ensure the integrity of the representation of small targets in high-resolution and low-resolution feature maps.

[0011] S4: The multi-scale feature maps generated in step S3 are fused using the Weighted Feature Fusion (CRC) module. The CRC module adaptively adjusts the weights of feature channels at each scale through channel reweighting and context modeling strategies, enhancing key features related to small targets, suppressing the influence of background noise, and calculating the regression loss of the detection box position based on the localization labels and predicted detection box coordinates of the training set.

[0012] S5: Optimize the target region features and regression loss of the training set samples generated in steps S4 and S5 using the context information enhancement module. This module adaptively adjusts the receptive field shape and extracts multi-level context information through dense connections and deformable convolution techniques, further mitigating feature attenuation caused by deep convolution.

[0013] S6: Through the comprehensive loss optimization module, based on the features generated in step S6, calculate the comprehensive loss including the target detection box position regression loss, classification loss and attention weight loss, and update the model parameters to improve detection accuracy.

[0014] In some embodiments of the present invention, the specific feature extraction process of the decomposable large kernel convolution module is as follows:

[0015] S2.1: Input the image feature map of the training set sample, and decompose the large kernel convolution into point convolution (1×1) in the channel dimension and large kernel convolution in the spatial dimension (such as 7×7 decomposed into 1×7 and 7×1) through depthwise separable convolution, thereby reducing the number of parameters and computational cost.

[0016] S2.2: Perform multi-branch parallel processing on the decomposed convolution operation, with each branch using convolution kernels of different sizes (such as 3×3, 7×7, 11×11) to capture multi-scale contextual information and enhance the diversity of small target features;

[0017] S2.3: Multi-branch features are fused with the input feature map through residual connections, preserving low-level detailed features and avoiding information loss caused by deep convolution;

[0018] S2.4: Batch normalization and activation function (such as ReLU6) are applied to the fused feature maps to generate initial feature regions for use by subsequent multi-scale feature extraction modules. This process, through decomposition and parallel processing, maintains computational efficiency while ensuring a large receptive field, making it particularly suitable for feature extraction of small targets in remote sensing images.

[0019] In some embodiments of the present invention, the working mechanism of the context-guided attention module includes: recalibrating the importance of feature channels through a channel attention mechanism, and focusing on the key region where the small target is located by combining a spatial attention mechanism, thereby effectively improving the feature representation capability.

[0020] In some embodiments of the present invention, the fusion strategy of the weighted feature fusion module includes: adopting an adaptive weight learning mechanism to dynamically adjust the fusion weights according to the importance of features at different scales, so as to achieve the optimal combination of features.

[0021] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0022] (1) This invention replaces traditional convolution with separable large kernel convolution, which effectively solves the problem of small targets losing contextual information due to insufficient receptive field. At the same time, the design of decomposing convolution significantly reduces the computational cost and achieves a balance between accuracy and speed.

[0023] (2) This invention introduces the CEM module and CRC module, and through multi-scale context aggregation and dynamic channel weighting, it greatly enhances the model's ability to extract features of small targets in complex backgrounds, effectively suppresses environmental noise interference, and improves the robustness of detection.

[0024] (3) The present invention uses the CIoU loss function to optimize the regression process of small target detection boxes, making the localization more accurate and significantly improving the recall rate and mean precision of small targets. Attached Figure Description

[0025] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0026] Figure 1 This is an overall flowchart of the small target detection method in a specific embodiment of the present invention.

[0027] Figure 2 This is a schematic diagram of the separable large kernel convolution module in this invention. Detailed Implementation

[0028] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention.

[0029] This invention provides a small target detection method based on multi-scale feature fusion, which is particularly suitable for small target detection tasks with low pixel ratio and severe background interference in scenarios such as remote sensing image analysis and UAV aerial photography. The specific method flow is described below:

[0030] Step 1: Obtain publicly available remote sensing image datasets and perform preprocessing.

[0031] Step 2: Select YOLOv8 as the base model, integrate innovative modules on it, and build a new network model to optimize small target feature extraction.

[0032] Step 3: Train a target detection network for remotely sensed images based on a new network model.

[0033] Specifically, the preprocessing of the dataset in step 1:

[0034] The publicly available VisDrone-2019 dataset was selected, containing approximately 10,209 high-resolution remote sensing images covering 10 small object categories (such as pedestrians, birds, and vehicles), with object sizes mostly between 10 and 50 pixels. The training set contains approximately 6,471 images, and the validation set contains approximately 1,613 images, which were converted to YOLO format.

[0035] Specifically, in step 2, the optimized YOLOv8 model is constructed:

[0036] YOLOv8 was chosen as the base model, with its backbone network employing the CSPDarknet architecture, which supports efficient feature extraction. Based on YOLOv8, innovative modules were integrated through the following steps to construct a system optimized for small target feature extraction.

[0037] Initialize YOLOv8: Load pre-trained weights and adjust the network input resolution to 640×640 to adapt to high-resolution remote sensing images.

[0038] When YOLOv8 uses fixed convolution kernels for feature extraction in its backbone network, it cannot extract all the feature information around small targets. It replaces this with separable large kernel convolutions to build a feature extraction network with a larger receptive field.

[0039] Decompose the large-kernel convolution into a series of depth-wise convolutions with continuously growing and expanding kernels:

[0040] k i-1 ≤k i ,d1=1,d i-1 <d i ≤RF i-1

[0041] RF1 = k1, RF i =d i (k i -1)+RF i-1

[0042] In the formula, the kernel size is k, the dilation rate is d, and the receptive field is RF.

[0043] The decomposed convolutional layers are then applied to the input X to obtain contextual information at different scales, as shown in the formula:

[0044] U0 = X,

[0045] in, It has a kernel k i and expansion rate d i The i-th depth convolutional layer, U i+1This represents the output features of the i-th deep convolutional layer. Assuming there are N decomposed kernels, each kernel is further processed by a 1x1 Conv layer, with the following formula:

[0046]

[0047] in It is the feature representation of the i-th dimension.

[0048] In the multi-scale feature fusion stage, to address the problem of feature information loss during the propagation process in the current feature fusion strategy, a path from the input node to the output node is added. In this path, the context extraction module (CEM) is used to supplement multi-scale information. At the same time, a multi-scale feature mapping fusion strategy module with weighted channels (CRC) is introduced to more accurately focus on key regions and features and reduce the impact of noise.

[0049] The Context Extraction Module (CEM) consists of multipath-expanded convolutional layers with different ratios, such as rate = 3, 6, 12, 18, 24. Using convolutional layers of different sizes allows for the acquisition of multiple feature maps within different receptive fields.

[0050] Furthermore, to enhance the modeling capability of geometric transformations, deformable convolutional layers are introduced in each path. By learning the offset and adjusting the sampling position of the convolutional kernel, the CEM can learn transformation-invariant features from the given data.

[0051] To refine the merging of multi-scale information, dense connections are used in CEM to fuse the features of each path layer by layer, thus mitigating the gradient vanishing problem.

[0052] The formula is:

[0053] Among them, F i Let be the input features of the t-th path, and let DCN be a deformable convolution, Δp i The corresponding offset is M=5, where M is the number of paths.

[0054] To address the issue of noise introduction during the fusion process, a weighted channel multi-scale feature mapping fusion strategy (CRC) is adopted. This strategy enables the feature fusion network to more accurately focus on key regions and features, while eliminating some noise, enhancing the representation of small targets by multi-scale features, and fully utilizing features from different channels. The channels of the feature mapping are weighted.

[0055] The feature maps are concatenated, and then the normalized trainable weights are multiplied by the total number of parameters in the channels, as shown in the formula:

[0056]

[0057] Where ωj ω represents the trainable weights in the j-th channel. m This represents the trainable weights for the m-th channel. m represents the total number of channels after concatenation. ε is set to 0.0001 to avoid numerical instability. Output represents the weights of the final output.

[0058] The fused feature map is calculated using the weighted feature fusion module, and the formula is as follows:

[0059] F fused =X·ω

[0060] Where F fused The output fused feature map

[0061] F fused Input the YOLOv8 detector head to generate bounding boxes and category labels.

[0062] CIoU is used as the bounding box regression loss function, and its calculation formula is as follows:

[0063]

[0064] Where IoU represents the ratio of the intersection area of ​​the predicted bounding box and the ground truth bounding box to the union area; ρ 2 (b,b gt ) represents the squared Euclidean distance between the center point of the predicted bounding box and the center point of the ground truth bounding box, where b is the coordinate of the center point of the predicted bounding box. gt is the coordinate of the center point of the ground truth bounding box; c is the diagonal length of the smallest closure region that simultaneously contains the predicted box and the ground truth box; α is the weight coefficient, and its formula is:

[0065] Where v is the aspect ratio consistency parameter, and its formula is:

[0066] Specifically, in step 3, a target detection network based on the new network model for remotely sensed images is trained:

[0067] The new network model was iteratively trained using the VisDrone-2019 training set, with the following training parameters set: initial learning rate set to 0.01, training epochs set to 200, and batch size set to 16.

[0068] Specific training process:

[0069] The preprocessed training set is input into the model, and initial features are extracted through a separable large kernel convolution module.

[0070] Multi-resolution feature maps are generated through a multi-scale feature extraction module;

[0071] The Contextual Information Enhancement (CEM) module enhances feature representation.

[0072] Optimized features are generated by fusing multi-scale features through the CRC module;

[0073] Calculate the loss based on the loss function and update the model parameters.

[0074] Specifically, in step 4, the model is tested on the validation set. The trained model is tested on the VisDrone-2019 validation set. The specific operation is as follows:

[0075] 1. Input validation set images (640×640 resolution), and generate predicted detection boxes and class labels using the trained model;

[0076] 2. Evaluate the model's localization and classification capabilities in small object detection tasks, focusing on the detection performance of small objects;

[0077] 3. Apply model pruning and quantization to optimize inference speed and adapt to the needs of real-time remote sensing image processing.

[0078] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A small target detection method based on dynamic feature fusion, characterized in that, Includes the following steps: A small target detection network model is constructed. The network model is based on the YOLO model and integrates a separable large kernel convolution module, a context information enhancement module (CEM) and a weighted feature fusion module (CRC). Image data is acquired and preprocessed. The preprocessed image data is then input into the network model, and the small target detection results are obtained through the following steps: S1: The image data is extracted using the separable large kernel convolution module. The receptive field is expanded by decomposing the large kernel convolution, and an initial feature map containing global context information is generated. S2: The initial feature map is processed using a multi-scale feature extraction module, and multi-scale feature maps of different resolutions are generated through multi-level convolution and upsampling operations. S3: Use the CEM module to perform context guidance and enhancement on the multi-scale feature map, and capture multi-level context information through multi-path extended convolution and deformable convolution; S4: The enhanced multi-scale feature map is fused by channel reweighting using the CRC module. The weights of the feature channels at each scale are adaptively adjusted to suppress the influence of background noise and obtain a fused feature map. S5: Through the comprehensive loss optimization module, calculate the position regression loss, classification loss, and attention weight loss of the target detection box based on the fused feature map; S6: Based on the comprehensive loss calculated in step S5, update the parameters of the network model until the model converges, and obtain the final small target detection model.

2. The method according to claim 1, characterized in that, In step S1, the specific feature extraction process of the separable large kernel convolution module includes: S1.1: The large kernel convolution is decomposed into a combination of point convolutions in the channel dimension and large kernel convolutions in the spatial dimension through depthwise separable convolution; S1.2: Perform multi-branch parallel processing on the decomposed convolution operation, with each branch using a different convolution kernel size or dilation rate to capture multi-scale contextual information; S1.3: The fusion result of the multi-branch features is fused with the input feature map through residual connection to retain low-level detailed features and avoid information loss.

3. The method according to claim 1, characterized in that, In step S3, the specific working mechanism of the context information enhancement module (CEM) includes: Construct a multipath structure consisting of multipath expanded convolutional layers with different ratios, for example, rate = 3, 6, 12, 18, 24; Deformable convolution technology is introduced into the multipath structure to adaptively adjust the sampling position of the convolution kernel by learning the offset, thereby enhancing the modeling capability of geometric transformations. A dense connection strategy is adopted to fuse the features of each path layer by layer, so as to finely merge multi-scale information and alleviate the gradient vanishing problem during training.

4. The method according to claim 1, characterized in that, In step S4, the fusion strategy adopted by the weighted feature fusion module (CRC) includes: S4.1: Connect the multi-scale feature maps, calculate the weight of each channel through the channel reweighting submodule, and adaptively adjust the weight according to the spatial and semantic information of the feature maps; S4.2: Employs a context modeling submodule, which captures global contextual information of feature maps through global pooling and convolution operations, generates context-guided weights, and focuses on small target regions; S4.3: The channel reweighting and the context-guided weights are fused, and the fused feature map is generated by element-wise weighting operations.

5. The method according to claim 1, characterized in that, In step S5, the target detection box position regression loss adopts the CIoU loss function, and the calculation formula of the CIoU loss function is as follows:

6. A small target detection system based on dynamic feature fusion, characterized in that, The system for performing the method according to any one of claims 1-5, the system comprising: The feature extraction module integrates the separable large kernel convolution module, which is used to perform deep feature extraction on the image and expand the effective receptive field; A multi-scale feature generation module is used to process the output of the separable large kernel convolution module to generate multi-scale feature maps of different resolutions. The context enhancement module integrates the context information enhancement module (CEM) and is used to perform context guidance and enhancement on the multi-scale feature map; The dynamic fusion module integrates the weighted feature fusion module (CRC), which is used to perform channel reweighting fusion on the enhanced multi-scale features; The detection output module is used to calculate the location and classification loss based on the fused features, update the network parameters, and output the small target detection results.

Images