Detection method suitable for small target in road image

Abstract

The invention relates to the technical field of computer vision, in particular to a detection method suitable for a small target in a road image, which comprises the following steps of: collecting road image data, and marking and normalizing a road image; constructing a detection model based on a specific backbone network and module optimization; a camera image is input into a detection model for target identification and positioning, and detection information is transmitted to a monitoring center; according to the invention, the feature complementary mapping module, the multi-core perception convolution module and the optimized loss function are introduced, so that the recognition capability of the model on weak texture and small-scale targets is enhanced, the small target detection stability and accuracy under various road scene conditions can be effectively improved, and the method is suitable for large-scale detection. Therefore, the problem of insufficient detection precision of the existing target detection algorithm is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, in particular to a detection method suitable for small targets in road images. BACKGROUND

[0002] In the intelligent transportation system, the detection of small targets in road images is a highly challenging task. Small targets have low pixel occupancy in images, insufficient feature resolution, and interference from background noise, making it difficult to ensure detection accuracy and efficiency.

[0003] Existing target detection algorithms have varying degrees of limitations when dealing with these issues. For example, YOLOv11 uses AFPN (adaptive feature pyramid network) to simply fuse multi-scale features and complete target framing through fixed detection heads, which performs well in normal scale target detection. However, for small targets in road scenes, such algorithms have significant shortcomings: small targets have low pixel occupancy in images, insufficient feature resolution, and traditional feature fusion methods cannot effectively capture cross-layer spatial semantic associations, resulting in a significant decline in detection accuracy. RT-DETR introduces a Transformer architecture to better capture long-range dependencies and improve detection performance. Cascade-RCNN has a multi-stage detection process, and Cascade-RCNN performs more refined classification and regression on candidate boxes at each stage. RetinexNet is a deep learning-based low-light enhancement method that simulates the human eye's visual processing mechanism to separate the reflection component and illumination component of the image to enhance low-light images, effectively improving image brightness and contrast, and thus improving detection performance. FCOS algorithm detects targets by predicting target center offset and scale information pixel by pixel, which is simple and efficient, and can quickly detect targets.

[0004] The above algorithms perform well in normal scale target detection, but when dealing with small targets in road scenes, they have low pixel occupancy in images, insufficient feature resolution, and high model computation complexity, and traditional feature fusion methods cannot effectively capture cross-layer spatial semantic associations, resulting in a significant decline in detection accuracy. SUMMARY

[0005] The present application aims to provide a detection method suitable for small targets in road images, aiming to solve the problem of insufficient detection accuracy of existing target detection algorithms.

[0006] To achieve the above-mentioned purpose, the present application provides a detection method suitable for small targets in road images, comprising the following steps:

[0007] Collect road image data, label and normalize the road images;

[0008] Based on a specific backbone network and a module optimization, a detection model is constructed;

[0009] The camera image is input into the detection model for target recognition and positioning, and the detection information is transmitted to the monitoring center.

[0010] In the "collecting road image data, labeling and normalizing road images", the following steps are included:

[0011] Collect road image data, and label the collected images;

[0012] The labeled data set is divided into a training set, a validation set and a test set;

[0013] The image is normalized to scale the pixel value to a predetermined range.

[0014] In the "constructing a detection model based on a specific backbone network and a module optimization", the following steps are included:

[0015] Determine the backbone network;

[0016] Construct a feature complementary mapping module;

[0017] Introduce a multi-core perception convolution module;

[0018] Replace the traditional IoU with NWD;

[0019] Set the momentum, weight decay, batch size parameters, use the linear decay strategy to train the model, and obtain the detection model.

[0020] In the "constructing a feature complementary mapping module", the following steps are included:

[0021] Divide the input feature channel into two branches according to the proportion α;

[0022] Process the two branches with standard 3x3 convolution and point convolution respectively to separate semantic and position information;

[0023] Complementary mapping is performed through channel and spatial interaction to generate channel information weight and spatial information weight;

[0024] Map the weights to the corresponding features respectively, and perform element-wise multiplication and addition operation to aggregate and generate a feature map containing spatial and semantic double mapping features.

[0025] In the "introducing a multi-core perception convolution module", the following steps are included:

[0026] Connect different size convolution kernels in sequence, and add point-by-point convolution between the kernels;

[0027] The input features are extracted into multi-scale local features through different size deep convolution, and after each convolution, 1*1 point convolution is used to realize cross-scale feature transformation and integration.

[0028] The multi-scale features are fused through residual connection in series to generate a feature representation containing global multi-scale context.

[0029] The small target detection project in the road image scene is input into the detection model, and the monitoring model identifies and locates the objects in the image.

[0030] The target position and category information recognized are transmitted to the monitoring center server through the MQTT protocol.

[0031] The detection method for small targets in road images provided by the application effectively solves the problems of multi-noise and few details in small target detection under complex scene conditions, significantly improves the detection accuracy, and achieves higher precision under the map@0.5 index in visual comparison with mainstream algorithms. This improvement not only improves the accuracy of detection, but also enhances the robustness and reliability of the model in practical application, making it better adapt to different environmental conditions, and provides more efficient and accurate solutions for road safety monitoring, traffic management and other related fields. BRIEF DESCRIPTION OF DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0033] Figure 1is the structural schematic diagram of the LLRT small target detection model. Downsample represents a downsampling operation; Upsample represents an upsampling operation; Detect-P3 / P4 / P5 represents a prediction head on different scales (P3: small, P4: medium, P5: large) of the pyramid; Concat represents channel splicing; Upsample represents an upsampling operation; Conv represents a standard convolution layer; MaxPool2d represents a two-dimensional maximum pooling operation; channel represents a'semantic branch' in FCM; spatial represents a'spatial branch' in FCM; C3k2 represents an improved C3 module, and 'k' represents the convolution kernel size in the bottleneck, and '2' represents that two bottleneck units are stacked in the backbone branch; Bottleneck represents a residual bottleneck structure; FCM represents a feature complementary mapping module; MKP represents a multi-kernel perception convolution module; and PSABlock represents a position-sensitive attention block.

[0034] Figure 2 is an effect visualization comparison chart of a road image scene small target detection algorithm (LLRT-YOLO) and mainstream target detection algorithms (YOLO series models, RT-DETR models, Retinenet models, Cascade-RCNN models, FCOS models and LLRT-YOLO models).

[0035] Figure 3 is a comparison chart of the present application and mainstream detection algorithms on the UA-DETRAC-G2 dataset, based on MAP@0.5. The table compares the precision of different mainstream target detection models (YOLO series models, RT-DETR models, Retinenet models, Cascade-RCNN models, FCOS models and LLRT-YOLO models) in Car (small car class), Bus (bus class), Vans (van class) and Other (other class) category detection.

[0036] Figure 4 is a model comparison experiment table when the input image resolution is 640x640. The table compares the performance of different target detection models (YOLO series models, RT-DETR models, Retinenet models, Cascade-RCNN models, FCOS models and LLRT-YOLO models) in different evaluation indexes, including mAP (average precision mean), recall rate (Recall), GFLOPs (floating point operations per second) and model parameter number.

[0037] Figure 5 is a flowchart of a detection method for small targets in road images provided by the present application.

[0038] Figure 6 is a flow chart of collecting road image data, labeling road images and normalizing the road images.

[0039] Figure 7 is a flow chart of constructing a detection model based on a specific backbone network and a module optimization.

[0040] Figure 8 is a flow chart of inputting a camera image into a detection model for target recognition and positioning, and transmitting the detection information to a monitoring center.

[0041] Figure 9 is a flow chart of constructing a feature complementary mapping module.

[0042] Figure 10 is a flow chart of introducing a multi-core perception convolution module. DETAILED DESCRIPTION

[0043] The embodiments of the present application will be described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by reference to the accompanying drawings are exemplary and are intended to explain the present application, and cannot be understood as a limitation of the present application.

[0044] Please refer to Figures 1 to 10 The present application provides a detection method for small targets in road images, comprising the following steps:

[0045] S1 Collecting road image data, labeling road images and normalizing the road images;

[0046] S11 Collecting road image data and labeling the collected images;

[0047] Specifically, representative road image data is collected and labeled, and the labeled images are preprocessed, covering various targets under different weather, lighting and shooting angles.

[0048] S12 Dividing the labeled data set into a training set, a validation set and a test set;

[0049] Specifically, the data set is divided into a training set, a validation set and a test set to ensure data independence and generalizability.

[0050] S13 Normalizing the images to scale the pixel values to a predetermined range.

[0051] Specifically, the images are normalized to scale the pixel values to an appropriate range (640x640px) to improve the distinguishability of small targets.

[0052] S2 constructs a detection model based on the specific backbone network and the module optimization;

[0053] S21 determines the backbone network;

[0054] Specifically, based on the demand for improving the recognition accuracy of small target in enhanced road images, we modified the YOLOv11 model and introduced two modules, FCM and MKP, into the original CSPDarknet53 structure of the backbone network. After the input image is extracted by the convolutional backbone network, it enters the FCM and MKP modules for feature enhancement in turn. The selection of the backbone network lays the foundation for the performance of the model, and its structure and parameters affect the quality and efficiency of feature extraction.

[0055] S22 constructs a feature complementary mapping module;

[0056] S221 divides the input feature channel into two branches according to the proportion α;

[0057] Specifically, the input feature channel is divided into two branches according to the proportion, focusing on semantic and spatial position information respectively. The feature complementary mapping module (FCM): In small target detection tasks, especially in different scenes, deep neural networks have the key problems of loss of small target spatial position information due to downsampling operations and difficulty in precise alignment of semantic features and spatial features. The FCM module addresses the problem of easy loss of small target information in deep networks by relying on channel segmentation, feature transformation and complementary mapping mechanism, and integrates shallow spatial position information into deep semantic features to alleviate information imbalance. For the characteristics of small detection targets, the performance of small target detection is optimized through differentiated segmentation strategies. The core mathematical expression of this module is as follows:

[0058] First, the input feature map

[0059]

[0060] is divided into two branches according to the proportion α:

[0061]

[0062] wherein,

[0063]

[0064] contains more channel semantic information,

[0065]

[0066] retains more spatial position information.

[0067] wherein represents the input feature map tensor, denotes the number of channels of the input feature map, denotes the spatial height of the input feature map, denotes the spatial width of the input feature map, denotes the channel division ratio hyperparameter, denotes the channel split operation, denotes the first branch feature. denotes the second branch feature.

[0068] For challenging small target detection scenes such as low light, the segmentation ratio parameter of the original module is designed. In the backbone of YOLO11, different stages adopt differentiated segmentation strategies: in the shallow feature layer (such as P3), set α=0.75 to retain more spatial information to meet the target positioning demand under low light; in the deep feature layer (such as P5), set α=0.25 to emphasize the extraction of semantic information to meet the classification challenge of small targets in different scenes.

[0069] S222 processes the two branches with standard 3x3 convolution and point convolution respectively to separate semantic and position information;

[0070] Specifically, the two branches are processed with standard 3x3 convolution and point convolution respectively to separate semantic and position information.

[0071] S223 generates channel information weights and spatial information weights through channel and spatial interaction for complementary mapping;

[0072] Specifically, channel information weights and spatial information weights are generated through channel interaction and spatial interaction for complementary mapping. .

[0073] S224 maps the weights to the corresponding features respectively, and aggregates to generate a feature map containing spatial and semantic double mapping features through element-wise multiplication and addition operations.

[0074] Specifically, the weights are mapped to the corresponding features respectively, and the element-wise multiplication and addition operations are performed to aggregate to generate a feature map containing spatial and semantic double mapping features. , which alleviates the loss of target spatial position information in the down-sampling process of the backbone network and enhances small target feature matching.

[0075] S23 introduces a multi-core perception convolution module;

[0076] S231 connects different size convolution kernels in sequence, and adds point-by-point convolution between the kernels.

[0077] Specifically, different size convolution kernels (such as 3x3, 5x5, 7x7) are connected in sequence, and point-by-point convolution (1x1 convolution) is added between the kernels.

[0078] ​Multi-kernel perception convolution module (MKP): The core of the MKP module is to enhance the relationship between targets of different scales and improve the perception ability of targets at different scales by using different sizes of convolution kernels. By replacing the final downsampling layer, the network structure is simplified, and then the small target information is enhanced. The operation process is as follows: the input feature first enters the feature extraction link composed of 3x3, 5x5, 7x7 different size deep convolution (DWConv). These deep convolutions of different specifications focus on mining local features in multi-scale dimensions; after each level of deep convolution operation, a 1x1 point convolution (PWConv) is connected. The point convolution is responsible for cross-scale feature transformation and information integration. It can adjust the channel dimension and feature interaction of the features extracted by different deep convolutions without increasing too much calculation, and promote the mutual penetration and supplement of feature information from different scales. Finally, the multi-scale features processed by multiple rounds of deep convolution and point convolution are connected and fused through a residual link to gather the feature clues scattered in each scale, generating a feature representation containing global multi-scale context information Let the model master the details of small targets and the global correlation of large targets at the same time.

[0079] The mathematical description is as follows:

[0080] Wherein, is the input local feature, which carries the initial image details; is the global mapping feature output after multi-scale fusion, which integrates rich scale correlation information; represents a deep convolution operation, and the convolution kernel size is , responsible for the preliminary extraction of local features of different scales; is a point-by-point convolution (i.e. 1x1 point convolution) transformation, which realizes the interaction and integration of cross-scale features.

[0081] S232 The input feature extracts multi-scale local features through different size deep convolutions, and a 1x1 point convolution is used after each convolution to realize cross-scale feature transformation and integration.

[0082] Specifically, the input feature extracts multi-scale local features through different size deep convolutions, and a 1x1 point convolution is used after each convolution to realize cross-scale feature transformation and integration.

[0083] S233 The multi-scale features are connected and fused through a residual link to generate a feature representation containing global multi-scale context.

[0084] Specifically, the multi-scale features are finally connected and fused through a residual link to generate a feature representation containing global multi-scale context, enhance the network's perception ability of targets of different scales, replace the network's final downsampling layer, and simplify the network structure.

[0085] S24 replaces the traditional IoU by NWD;

[0086] Specifically, NWD is a new bounding box similarity measure used to replace the traditional IoU in label assignment, loss calculation and NMS. In LLRT, NWD is used to adjust the positive and negative sample assignment, and when the predicted box matches the real box on the small target, NWD provides more stable similarity evaluation; at the same time, NWD is used to replace the traditional IoU (or GIoU / CIoU) in the loss function to calculate the regression error, avoiding the problem of gradient sparsity caused by low IoU value of small target. Experiments show that after using NWD, the AP of small target detection in road image is significantly improved, thereby improving the detection performance of the system on distant or small size targets.

[0087] Normalized Wasserstein distance loss function (NWD): To solve the problem of insufficient regression accuracy of small target bounding box, NDWLoss adopts Gaussian distribution modeling-Wasserstein distance measurement mechanism, which regards the predicted box and the real box as Gaussian distribution, and measures the distribution difference by calculating the second-order Wasserstein distance. This method overcomes the gradient disappearance problem of traditional IoU loss when the target overlap is low, and is especially suitable for fine positioning of small targets. Assuming that there are two bounding boxes , respectively representing the center coordinates and the width and height of the box, we regard each bounding box as a Gaussian distribution, and the mean value is the center of the box, and the covariance matrix is generated according to the size. First, define the 2-order Wasserstein distance of two bounding boxes

[0088]

[0089] where respectively represent the multivariate Gaussian distribution of box a and box b.

[0090] In order to make the distance have better interpretability and numerical stability, we normalize it to get:

[0091]

[0092] where represents the normalized Wasserstein distance similarity, and C represents the normalization constant.

[0093] The final loss function is defined as:

[0094]

[0095] ​wherein and respectively represent the Gaussian distribution corresponding to the prediction frame and the real frame.

[0096] S25 sets the momentum, weight decay, batch size parameters, adopts a linear decay strategy to train the model, and obtains a detection model.

[0097] Specifically, a stochastic gradient descent (SGD) optimizer is used for training, the momentum (momentum) is set to 0.937, and the weight decay (weightdecay) is set to 0.0005. The batch size is set to 32, the initial learning rate is 0.01, and the total number of training iterations is about 300. A linear decay (LinearLR) strategy is adopted, which is essentially a phased decay: lr0 is used in the early stage of training, and gradually decreases linearly to lrf in the later stage. Such a strategy is conducive to the rapid convergence of the model and the full learning of features. In different model comparison experiments, the parameter differences such as the parameter amount and the calculation amount (such as GFLOPs) of different models are reflected.

[0098] S3 inputs the camera image into the detection model for target recognition and positioning, and transmits the detection information to the monitoring center.

[0099] S31 inputs the small target detection project in the road image scene into the detection model, and the monitoring model identifies and positions the objects in the image.

[0100] Specifically, the present application is applied in the small target detection project in the road scene in various complex scenes. We deploy our algorithm (algorithm structure as shown in Figure 1 ) using an edge processing device JetsonTX2NX. We place the JetsonTX2NX in the device box of the road monitoring system, install the external camera at the top of the monitoring pole, and align it with the road to ensure that it can cover the panorama of the road. The resolution of the camera is configured according to actual needs. The camera transmits the collected images to the edge processing device JetsonTX2NX deployed in the device box in real time through a wired manner.

[0101] The collected images are first input into the detection model. The first part of the detection model is the detection backbone network of YOLOv11 (CSPDarknet53 structure). The backbone network gradually extracts multi-level features of the image through a series of convolutional layers. These features include not only low-level detail features such as edges and textures, but also high-level semantic features such as object shapes and categories. In order to enhance the detection ability of small targets, the algorithm embeds a feature complementary mapping module (FCM) at each stage of the backbone network. The main function of FCM is to deeply fuse the low-level spatial location information with the high-level semantic information to improve the positioning and detection performance of small targets. Specifically, the input features are first divided into two parts, one part is used to extract rich channel information, and the other part is used to retain more original spatial location information. Then, different types of convolution operations are used to process these two parts of features, one part is processed by standard 3x3 convolution to extract rich channel information, and the other part is processed by pointwise convolution to retain spatial location information. Next, the two parts of features are fused through complementary mapping to generate features containing spatial and semantic relationships. This process not only enhances the performance of the features, but also effectively solves the problem of loss of target spatial location information caused by down-sampling in traditional networks.

[0102] At the last stage of the backbone network, the algorithm introduces a multi-kernel perception convolution module (MKP). MKP enhances the perception ability of different scale targets by using different size convolution kernels, so as to better handle the feature extraction problem of small targets. MKP connects different size convolution kernels in sequence and adds pointwise convolution between different scale convolution kernels to highlight multi-scale feature representation and further simplify the network structure. This design not only captures the feature information of the target at different scales, but also improves the detection performance of the network for small targets through feature fusion.

[0103] The multi-scale features processed by the FCM and the MKP are sent to a detection head part for final target detection. The detection head further enhances the performance of the features by fusing feature maps of different scales. Specifically, first, the feature maps of different scales are up-sampled or down-sampled to make their sizes consistent. Then, the feature maps of different scales are fused by weighted summation to enhance the performance of the feature maps. The fused feature maps are sent to the detection head, which includes a bounding box prediction branch and a class prediction branch. The bounding box prediction branch predicts the bounding boxes of the target objects by a regression algorithm, and outputs the center coordinates (x, y), width (w) and height (h) of each bounding box. The class prediction branch outputs the class probability of each bounding box by a softmax function. Specifically, the network outputs the center coordinates (x, y), width (w) and height (h) of each bounding box. In order to optimize the prediction accuracy of the bounding boxes, the application introduces an optimization loss function, which calculates the normalized Wasserstein distance between the predicted boxes and the true boxes, and more robustly handles the positioning deviation caused by the small number of pixels and the large amount of noise of small targets. The class prediction branch outputs the class probability of each bounding box by a softmax function. The network uses a cross-entropy loss function to optimize the accuracy of class prediction during the training process.

[0104] S32 transmits the recognized target position and category information to the monitoring center server through the MQTT protocol.

[0105] Specifically, the recognized target position, category and other information are transmitted through the Mqtt protocol. The Mqtt protocol is a lightweight publish / subscribe message transmission protocol suitable for use in low-bandwidth and unstable network environments. The Mqtt message is transmitted to the monitoring center server through wireless communication technology. After receiving the target information, the monitoring center server further analyzes and processes the information according to the position, category and speed of the target. For example, in a traffic monitoring scenario, the server can predict the driving trajectory of a vehicle according to its driving speed and position, and issue a traffic warning signal in advance; in a security monitoring scenario, the server can trigger the corresponding alarm mechanism according to the detected target type and behavior. In this way, the application not only can detect small targets on the road in real time, but also can provide intelligent decision support according to specific application scenarios, improving the overall efficiency and security of the monitoring system.

[0106] Advantages:

[0107] I. In the detection of small targets on roads in various complex scenarios, the application effectively solves the problems of multiple noises and few details in small target detection under complex scene conditions, significantly improving the detection accuracy.

[0108] II. In the visual comparison with mainstream algorithms, the present application can achieve higher accuracy under the index of map@0.5. This improvement not only enhances the accuracy of detection, but also enhances the robustness and reliability of the model in practical application, making it better adapt to different environmental conditions, providing more efficient and accurate solutions for road safety monitoring, traffic management and other related fields.

[0109] The above disclosure is only a preferred embodiment of the application for detecting small targets in road images. Of course, it cannot limit the scope of the rights of the application. Those skilled in the art can understand that all or part of the above-mentioned embodiments can be implemented, and equivalent changes made in accordance with the claims of the present application still fall within the scope of the application.

Claims

1. A method for detecting small targets in a road image, characterized in that, It comprises the following steps: Collecting road image data, labeling and normalizing road images; Building a detection model based on a specific backbone network and module optimization; Inputting camera images into the detection model for target recognition and positioning, and transmitting the detection information to the monitoring center.

2. The method for detecting small objects in a road image according to claim 1, wherein In "collecting road image data, labeling and normalizing road images", it comprises the following steps: Collecting road image data and labeling the collected images; Divide the labeled data set into training set, validation set and test set; Normalize the image and scale the pixel value to the preset range.

3. The method for detecting small objects in a road image according to claim 2, wherein In "building a detection model based on a specific backbone network and module optimization", it comprises the following steps: Determine the backbone network; Build a feature complementary mapping module; Introduce a multi-core perception convolution module; Replace the traditional IoU with NWD; Set the momentum, weight decay, and batch size parameters, use linear decay strategy to train the model, and get the detection model.

4. The method for detecting small targets in a road image according to claim 3, wherein In "building a feature complementary mapping module", it comprises the following steps: Divide the input feature channel into two branches according to the proportion α; Separate semantic and location information by using standard 3x3 convolution and point convolution on the two branches respectively; Complementary mapping through channel and spatial interaction to generate channel information weight and spatial information weight; Map the weights to the corresponding features respectively, and aggregate them through element-wise multiplication and addition operation to generate a feature map containing spatial and semantic double mapping features. In "introducing a multi-core perception convolution module", it comprises the following steps:

5. The method for detecting small objects in a road image according to claim 3, wherein Connect different size convolution kernels in turn, and add point convolution between kernels; Input features extract multi-scale local features through different size deep convolution, and use 1x1 point convolution after each convolution to realize cross-scale feature transformation and integration; Fuse multi-scale features through residual link to generate a feature representation containing global multi-scale context. In "inputting camera images into the detection model for target recognition and positioning, and transmitting the detection information to the monitoring center", it comprises the following steps:

6. The method for detecting small objects in a road image according to claim 1, wherein Input the small target detection project in the road image scene into the detection model, and the monitoring model recognizes and locates the objects in the image; Transmit the recognized target position and category information to the monitoring center server through the MQTT protocol. ​

Images