Small target detection method for driving scene

Abstract

The invention provides an efficient and accurate small target detection method aiming at the problems of low detection precision, high false detection rate and poor real-time efficiency of small target objects in the field of automatic driving. According to the method, a YOLOv8 network is used as a baseline model, a dynamic enhancement convolution module is introduced into a backbone feature extraction network, and more detailed target detail information is extracted. And an adaptive weighted receptive field fusion module is adopted, so that image feature enhancement of diversified receptive fields is realized, and the richness and adaptability of feature representation are improved. And a large kernel channel space attention module is utilized to highlight a small target area and suppress background noise, so that the distinguishing capability and the detection performance of the whole model are remarkably improved. According to the technical scheme, the precision of small target detection is effectively improved, the omission ratio and the false detection ratio are reduced, and the perception capability and the safety of the automatic driving system in a complex traffic environment are enhanced.

Description

Technical Field

[0001] The present invention belongs to the field of target detection, and mainly relates to a small target object detection method for driving scenes. Background Art

[0002] In recent years, autonomous driving technology has become an important development direction for future transportation. Autonomous vehicles rely on efficient and accurate environmental perception systems to achieve a comprehensive understanding of the surrounding environment and real-time response. The environmental perception system mainly includes modules such as sensor data acquisition and processing, target detection and recognition, path planning and decision-making. Among them, target detection, as the core link of environmental perception, directly affects the safety and reliability of autonomous vehicles.

[0003] In an autonomous driving system, the task of target detection is to identify and locate various objects in the surrounding environment, including pedestrians, vehicles, traffic signs, obstacles, etc. The detection of these targets requires not only high precision and high recall, but also real-time response in a complex and changing traffic environment. However, traditional target detection methods often show low detection accuracy and high false detection rate when facing small targets, which is particularly prominent in autonomous driving applications.

[0004] The importance of small target detection in autonomous driving cannot be ignored. First, many potentially dangerous objects such as pedestrians, bicycles, road obstacles, etc. are usually small in size and far away. Timely and accurate detection of these small targets is crucial to avoiding traffic accidents. Second, autonomous vehicles need to quickly scan and respond to the surrounding environment under high-speed driving conditions. The detection of small targets requires algorithms with efficient computing performance and real-time processing capabilities.

[0005] At present, researchers have proposed a variety of improvement methods for small target detection. Common strategies include enhancing feature extraction capabilities, optimizing network structures, and introducing multi-scale detection mechanisms. For example, by adopting a deeper convolutional neural network (CNN), richer feature information can be extracted, thereby improving the detection accuracy of small targets; using a feature pyramid network (FPN), target detection can be performed at different scales to enhance the response to small targets; in addition, methods combining attention mechanisms are also widely used to increase the network's attention to small targets.

[0006] Although many studies have made some progress in small target detection, there are still many challenges in practical applications. First, small targets are easily confused with the background due to their small size, resulting in missed detection or false detection in the detection algorithm. Second, the real-time requirements in the autonomous driving environment are high. How to achieve efficient computing processing while ensuring detection accuracy is an urgent problem to be solved. Summary of the invention

[0007] The purpose of the present invention is to propose an efficient, accurate and robust small target detection method for driving scenes in view of the shortcomings of existing small target detection technologies in driving scenes in terms of feature utilization, error sensitivity, high-precision positioning and network structure.

[0008] The technical solution of the present invention is as follows:

[0009] A small target detection method for driving scenes, including:

[0010] Step 1) preprocessing the high-resolution complex road scene image to be detected to obtain a target image containing target position and category information;

[0011] Step 2) inputting the processed driving scene image into the dynamic feature enhancement backbone network to perform preliminary feature extraction and obtain a multi-scale feature map;

[0012] Step 3) Input the multi-scale feature map into the adaptive weighted receptive field fusion module to achieve cross-branch feature interaction and integration and generate an enhanced feature map;

[0013] Step 4) Send the enhanced feature map to the improved feature pyramid network and upsampling module to fuse and integrate multi-scale features;

[0014] Step 5) Input the fused feature map into the large-core spatial channel attention module to enhance key features and optimize the information interaction of the feature map in the spatial and channel dimensions;

[0015] Step 6) The enhanced information is sent to the detection layer, and the detection results of small target objects are output based on the multi-scale feature map, including target category, location and confidence score, to complete the entire small target detection task.

[0016] Further, the step 1) specifically includes the following steps:

[0017] Step 11) Using marking software to mark small targets in the road scene in the initial data set, ignoring large interfering targets in the image, generating corresponding annotation files, and using them together with the original image files as training data sets;

[0018] Further, the step 2) specifically includes the following steps:

[0019] Step 21) The processed image data is sent to the feature extraction backbone network for feature extraction, and the dynamic feature enhancement convolution module (DFEC) is used as the backbone network downsampling method, combined with C2f to obtain enhanced multi-scale features, from low to high, {S1, S2, S3};

[0020] Further, the step 3) specifically includes the following steps:

[0021] Step 31) The three different scale feature maps extracted by the backbone network are respectively input into the adaptive weighted receptive field fusion module (Adaptive Receptive Field Fusion Module, ARFFM) to obtain the high-level semantic features after feature enhancement {S1 ′ 、S2 ′ 、S3 ′};

[0022] Further, the step 4) specifically includes the following steps:

[0023] Step 41) The enhanced semantic features are sent to the improved feature pyramid network (FPN). High-level semantic features S3 ′ After upsampling and S2 ′ The feature maps are fused to obtain S32, which is then upsampled and combined with S1 ′ The feature maps are fused and passed through C2f to obtain S1 ″ Then, the DFEC module is used to downsample and concatenate with the S32 feature map to obtain S2 ″ , and then after DFEC sampling enhancement, it is combined with S3 ′ Fusion gets S3 ″ ;

[0024] Further, the step 5) specifically includes the following steps:

[0025] Step 51) The fused high-level semantic feature S1 ″ , S2 ″ , S3 ″ They are input into the Large Kernel Spatial-Channel Attention Module (LKSCAM) respectively, and the spatial attention mechanism of global average pooling and maximum pooling is introduced. The large kernel convolution and decomposed convolution are combined to extract spatial context information and channel attention information, and finally the global high-level semantic features containing channel attention and spatial attention are obtained. ″′、S2 ″′ 、S3 ″′};

[0026] Further, the step 6) specifically includes the following steps:

[0027] Step 61) The final global high-level semantic feature S1 ″′ 、S2 ″′ 、S3 ″′ The results are sent to the decoupled detection head to generate corresponding category predictions and bounding box regression predictions. For each predicted box, its confidence score is calculated, and non-maximum suppression (NMS) is used to eliminate redundant prediction boxes with high overlap. Finally, the confidence scores after NMS processing are summarized to generate the final detection results and complete the small target detection task. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings, in which:

[0029] Figure 1 This is the flow chart of the method;

[0030] Figure 2 This is the overall model framework diagram of this method;

[0031] Figure 3 This is the structural diagram of the dynamic feature enhancement backbone network proposed in this method;

[0032] Figure 4 This is the structure diagram of the adaptive weighted receptive field fusion module proposed in this method;

[0033] Figure 5 The structure diagram of the large-kernel spatial channel attention module proposed in this method. DETAILED DESCRIPTION

[0034] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0035] The present invention provides a small target object detection method for driving scenes, the method flow is as follows Figure 1 As shown in Figure 2, the overall model framework proposed by this method is as follows: Figure 2 As shown, the method comprises the following steps:

[0036] Step 1) Build a large-scale small target detection dataset for driving scenes;

[0037] Step 2) construct a dynamic feature enhancement backbone network;

[0038] Step 3) construct an adaptive weighted receptive field fusion module;

[0039] Step 4) construct a large-core spatial channel attention module;

[0040] Step 5) Build a small target detection network for driving scenes;

[0041] Step 6) training the target detection network;

[0042] Step 7) Test the object detection network.

[0043] Further, step 1) specifically includes the following steps:

[0044] Step 11) Use a public website to obtain a large-scale small target detection dataset for driving scenes and preprocess it: For high-resolution driving scene images, first check the image size, and use a sliding window to cut the image into multiple overlapping image patches for images that meet the size. Convert the annotation position in the original image to the relative position within the patch, and calculate the intersection over union (IoU) to decide whether to retain the annotation. For the image annotation marked as the ignored area, use a preset color block to fill it to prevent the area from misleading the model, and finally obtain the preprocessed image patch;

[0045] Step 12) The preprocessed image patches are divided into training set, test set and validation set in a ratio of 7:2:1. The allocated data sets are preprocessed by data enhancement methods such as random cropping, random flipping, color jittering, etc. to simulate various common situations in real driving scenes, improve model stability and generalization, and finally obtain a large-scale small target detection data set for driving scenes.

[0046] Further, step 2) specifically includes the following steps:

[0047] Step 21) Figure 3 As shown, the dynamic enhanced convolution module (DFEC) consists of a non-strided convolution extraction block, a spatial to depth transformation block, a dynamic channel weighting block, and a channel adjustment and normalization activation function. First, the non-strided convolution block formula is defined as:

[0048] X′=Conv k＝3,s＝1,p＝1 (X)

[0049] Among them, k represents the convolution kernel size of 3, s represents the stride of 1, does not change the spatial resolution, and p represents the padding of 1. This block is mainly used to fully extract local features of the input features. Then the local features are sent to the spatial depth conversion for downsampling. The input feature map is converted from space to depth with a scaling factor of s=2. First, the feature map X′ is divided into s 2sub-feature maps, each of which has a size of (H / s)×(W / s). These feature maps are then connected in series along the channel direction to obtain a new feature map. This ensures that the key details of small objects can be retained after downsampling. SPD Then perform SE operation to obtain the corresponding dynamic weight. First, X SPD Perform global average pooling:

[0050]

[0051] Then the channel weights are calculated through two fully connected layers and nonlinear activation. Defining the dimension reduction ratio r = 4, we can get:

[0052]

[0053] Where σ is the Sigmoid function. When restored to channel weights, the dimension becomes That is, a weight in the range [0, 1] is generated for each channel. By comparing these weights with the original X SPD By multiplying the features, we can get the corresponding channel weighting:

[0054]

[0055] Where ⊙ is the element-wise multiplication operation on the channel.

[0056] Finally, the weighted features are adjusted using a 1×1 convolution, and then normalized and activated.

[0057]

[0058] X norm =BN(X out )

[0059] Y=SiLU(X norm )

[0060] Among them, BN reduces the internal variable offset through normalization and accelerates the training convergence; SiLU improves the representation ability of the network through smoothing and nonlinearity, and the final output is

[0061] Step 22) Figure 3 As shown in the figure, the dynamic feature enhancement backbone network consists of the C2f block and the dynamic enhancement convolution module mentioned above. The dynamic enhancement convolution module completes the downsampling of the feature map to ensure that the fine-grained information of small objects will not be discarded during the downsampling process. Combined with C2f, efficient feature encoding is achieved, so that the entire backbone network can extract more small target features.

[0062] Further, step 3) specifically includes the following steps:

[0063] Step 31) Figure 4 As shown in Figure 1, the adaptive weighted receptive field fusion module (ARFFM) consists of two parts: a multi-branch feature extraction unit and an adaptive weighted fusion module. Where B is the batch size, C in is the number of input channels, H and W are the height and width of the feature map. It is input into four branches (Branch0, Branch1, Branch2, Branch3) in parallel. Before entering each branch, a 1×1 convolution is usually used to reduce the channel dimension of X to ensure that the number of intermediate channels output by the four branches is Facilitates subsequent integration.

[0064] Branch0 is mainly based on standard local feature extraction, including a 3×3 convolution (stride=1, no dilation convolution), whose receptive field is relatively small but can capture fine-grained information, defined as:

[0065] X0=f0(X)=Conv 3×3 (Conv 1×1 (X)

[0066] Branch1 and Branch2 first use (3×1) convolution to capture detailed features in the vertical direction ((1×3) convolution to obtain horizontal features), and then connect a 3×3 dilated convolution (dilation=3) to expand the receptive field to obtain richer context information. The process of this branch is defined as:

[0067] X 1，2 =f 1，2 (X) = Conv 3×3，d＝3 (Conv 3×1，1×3 (Conv 1×1 (X)))

[0068] The final Branch3 is more complex, and multiple convolutions are stacked to achieve a larger receptive field. Finally, a 3×3, d=5 dilated convolution is used to expand the receptive field, thereby capturing a wider range of contextual information.

[0069] X3=f3(X)=Conv 3×3，d＝5 (Conv 3×3 (Conv 3×3 (Conv 1×1 (X))))

[0070] Then use the learnable parameters W0, W1, w2, w3 to adaptively allocate the weights of the four branches. Normalized by Softmax:

[0071]

[0072] By dynamically adjusting the weight parameters, the best branch features are provided in images with different distributions and target sizes. These weights are then used to weight the outputs of each branch according to the channel dimension:

[0073] X fused =concat(w′0X0, w′1X1, w′2X2, w′3X3; dim=1)

[0074] In order to reduce the difficulty of training deep networks, a residual structure is introduced. The input X is projected to space, and Alignment:

[0075] X short =Conv 1×1 (X)

[0076] Output the main path With X short Add them together and use the ReLU activation function to improve the nonlinear expression ability. In order to control the influence of the main path in the early stage of training, add a scaling factor α:

[0077]

[0078] The multi-branch structure with different convolution kernel sizes and dilation rates is used to capture diverse features from local to global, from low scale to high scale. It achieves an organic combination of receptive field diversity and feature adaptive weighted fusion, thus showing stronger robustness and expressiveness when facing targets of varying sizes and shapes and complex backgrounds;

[0079] Further, step 4) specifically includes the following steps:

[0080] Step 41) Figure 5 As shown in the figure, the large kernel spatial channel attention module (LKSCAM) establishes a dual-level adaptive attention of space and channel through global statistics and large kernel attention convolution to improve the feature representation capability.

[0081] Given input features First, global average pooling (GAP) and global maximum pooling (GMP) are performed, and the results are normalized by Softmax along the channel dimension to obtain A avg and A max Two global channel distributions are described.

[0082] After adjusting the channel dimension of X through 1×1 convolution in another branch, decomposed large kernel convolution is used to obtain the context feature representation V under wide receptive field. Next, decomposed large kernel convolution is used, that is, two strip-shaped large kernel convolutions of 1×k and k×1 are used to model long-distance spatial dependencies, so that V can capture large-scale context information in the horizontal and vertical directions respectively. In addition, the input feature X is Softmax normalized along the spatial dimension through 1×1 convolution to generate the spatial attention weight K. The obtained spatial attention weight K is matrix multiplied with V, and the result is the context feature Y in the global space range, whose size is N×C×1×1.

[0083] Then use A avg With A max Multiply V to obtain the spatial mapping Y under two global statistical guidance avg and Y max , and concatenate them into Y cat Finally, use 1×1 convolution and sigmoid activation to transform Y cat Converted into a spatial attention map, Y is channel-adjusted and added to the input X residual:

[0084] X out =x+(M(Y)×σ(M2(Y cat )))

[0085] Where M represents the adjustment of the channel dimension.

[0086] Further, step 5) specifically includes the following steps:

[0087] Step 51) The dynamic feature enhancement backbone network, adaptive weighted receptive field fusion module, large core spatial channel attention module, two-level feature fusion CSP module and decoupled detection head constructed in the above steps are used to form the small target detection network for driving scenarios in this paper;

[0088] Further, step 6) specifically includes the following steps:

[0089] Step 61) input the data set into the target detection network model, configure the corresponding training environment, set the corresponding training parameters, and carry out the model training task;

[0090] Step 62) When configuring the training environment, the software parameters are: operating system version is Windows 11, deep learning framework is Pytorch 2.2.2, programming language is Python 3.10, hardware parameters are: GPU: NVIDIA RTX 4070 12GB, CPU: Intel Core i5-13400F, memory is 64GB;

[0091] Step 63) In the configuration of training parameters, the optimizer uses SGD, the learning rate is set to 0.001, the momentum is 0.937, the weight decay is 0.0005, the training rounds are 300, and the batch size is 32;

[0092] Further, step 7) specifically includes the following steps:

[0093] Step 71) Load the trained weight file into the target detection network, randomly select images from the test data set for detection, and generate annotated images containing target location and category information;

[0094] The above is only a preferred embodiment of the present invention. For those skilled in the art, various changes, modifications and combinations can be made to the above embodiments and descriptions under the premise of conforming to the concept and spirit of the present invention. These equivalent substitutions and improvements to the present invention should be deemed to belong to the protection scope of the present invention. The protection scope of the present invention shall be based on the scope defined by the attached claims.

Claims

1. A small target object detection method for driving scenes, characterized in that: The following steps are involved: Step 1, collect relevant datasets for small target detection in the field of autonomous driving and preprocess them: download large-scale small target detection datasets in driving scenes from public websites. For each high-resolution driving road scene image, first check whether the image size is less than the minimum preset size. If it is less than the minimum preset size, edge filling is used. For the high-resolution image after detection, the sliding window method is used to divide the image into multiple overlapping image patches to ensure that all areas are covered. According to its position in the original image, the annotation information is adjusted, the bounding box in the original image is converted to the relative coordinates within the patch, and the intersection over union (IoU) is calculated to decide whether to retain the annotation. For the image areas marked as ignored, the preset filling color blocks are used to fill them to ensure that these areas will not mislead the model. The processed image patches and the corresponding annotation information are divided into training set, test set and validation set in the ratio of 7:2:

1. There are 9 categories of detection targets, namely pedestrians, cyclists, bicycles, motorcycles, vehicles, traffic signs, traffic lights, traffic cameras and warning cones. The entire preprocessed dataset is used as the training and validation dataset; Step 2: Construct a dynamic feature enhancement backbone network: This network is mainly composed of C2f and dynamic feature enhancement convolution modules. The dynamic feature enhancement convolution module is used as the backbone network downsampling module, and the C2f module is the feature encoding and deep feature extraction module after downsampling. The dynamic feature enhancement convolution module uses the non-step convolution extraction block and the space-to-depth transformation block as the core modules to retain the fine-grained information of small targets and enhance the detailed features of small target detection; Step 3: Construct an adaptive weighted receptive field fusion module: It consists of an adaptive weighted fusion module and a multi-branch feature extraction unit. The four branches use convolution kernels of different sizes plus dilated convolution to obtain multiple receptive fields, respectively extracting information of different directions, granularities, and contexts, and use adaptive weight adjustment to learn the importance of each branch, so that it can show strong representation capabilities for targets of varying sizes and shapes; Step 4, construct a large-kernel spatial channel attention module: By combining large convolution kernels with global statistical information, a feature enhancement module with dual attention of channels and space is realized. First, the global average and maximum statistical distribution of the feature map are extracted to provide a basis for channel screening; at the same time, the decomposable large-kernel convolution is used to obtain the feature representation of the wide receptive field, and the normalized weights of the spatial dimension are used to realize the adaptive weighting of the pixel position of the feature map. Finally, LKSCAM combines the global channel information with the spatial attention and adds it to the input feature residual, so as to effectively highlight the target area, suppress irrelevant background, and improve the discrimination ability and robustness of the overall feature; Step 5: Construct a cross-stage partial stacking module: By decomposing the input features into several sub-feature groups and performing cross-level fusion between different groups, its representation ability is enhanced to achieve effective integration of features from shallow to deep layers; Step 6, construct the Detect decoupled detection head: After passing through the large-core spatial channel attention module, the decoupled detection head performs the final target detection prediction on the fused features. This detection head adopts a decoupling strategy to process the classification and regression tasks separately, so that the network can achieve performance improvements in both positioning and classification dimensions. By independently optimizing the classification and positioning branches, the Detect module can better focus on the category discrimination and bounding box regression of the object, and then cooperate with post-processing steps such as NMS to achieve efficient and accurate detection result output; Step 7: Build a small target detection network for driving scenarios: Use the dynamic feature enhancement backbone network, adaptive weighted receptive field fusion module, large core spatial channel attention module, cross-stage partial stacking module and Detect decoupled detection head built in the above steps to form a small target detection network for driving scenarios; Step 8: Train the target detection network model: Use the training set to comprehensively train the small target detection network dedicated to driving scenarios, which consists of a dynamic feature enhancement backbone network, an adaptive weighted receptive field fusion module, a large core spatial channel attention module, a cross-stage partial stacking module, and a Detect decoupled detection head, and finally obtain the trained network model weights; Step 9, test the target detection network model: load the trained weight file into the target detection network, randomly select images from the test set for detection, generate annotated images containing the location and category information of the target to be detected, and calculate the detection accuracy and detection speed of each category in the driving scene image dataset, as well as the overall average detection accuracy.

2. The small target detection method applicable to driving scenes according to claim 1, characterized in that: In step 2, a dynamic feature enhancement convolution module is constructed. This module integrates non-strided convolution, space-to-depth conversion, and dynamic channel weight adjustment functions. Specifically, the space-to-depth conversion module can retain fine-grained detail information, thereby enhancing the feature expression of small targets; while the dynamic channel weight adjustment can screen out key features and improve the detection capability of small targets.

3. The small target detection method applicable to driving scenes according to claim 1, characterized in that: In step 3, an adaptive weighted receptive field fusion module is constructed. This module consists of an adaptive weighted fusion module and a multi-branch feature extraction unit. The multi-branch feature extraction unit extracts detail information and semantic information of different scales and depths, and uses an adaptive weighted fusion method to enable the model to learn more important feature extraction branches, thereby achieving adaptive adjustment and enhancing the image features of small target objects.

4. The small target detection method for driving scenes according to claim 1, characterized in that: In step 4, a large-core spatial channel attention module is constructed. The module first performs global average pooling and global maximum pooling on the input feature map to extract the global average distribution and maximum distribution of the feature map in the channel dimension, thereby providing a basis for channel screening. The module uses a decomposable large convolution kernel to perform convolution operations on the feature map to obtain a spatial feature representation with a wide receptive field and enhance the perception of distant targets. By normalizing the spatial features obtained by the large-core convolution, the weight coefficients in the spatial dimension are generated to achieve adaptive weighting of each pixel position in the feature map and improve the attention to key areas. Finally, the global channel information is combined with the spatial attention information to generate a comprehensive attention weight, which is added to the input feature map through a residual connection to complete the feature enhancement process. This module can effectively highlight the target area, suppress irrelevant background, and improve the discrimination ability and robustness of the overall feature, so as to more effectively detect small targets in driving scenarios and improve the accuracy and stability of detection.

Images