A height-limiting pole detection algorithm with high precision and light weight

Abstract

The present invention belongs to the field of computer vision object detection and relates to a high-precision and lightweight height-limiting pole detection algorithm. The present invention improves the YOLOv5s detector by using a constructed deformable module, achieving real-time detection of height-limiting poles in the scene. The present invention conducts evaluation experiments using a real height-limiting pole dataset and finally obtains a detection accuracy superior to the baseline YOLOv5s.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision object detection, and relates to a height-limiting pole detection algorithm with high precision and light weight. Background Art

[0002] A height-limiting pole is a common traffic device on traffic roads used to limit the height of passing vehicles. During driving, many traffic accidents occur because drivers misestimate the height of the height-limiting pole or even fail to notice the height-limiting pole ahead due to fatigue driving. In addition, many height-limiting pole height measurement algorithms are not yet perfect. The first step of these algorithms often requires detecting and locating the height-limiting pole in the scene. Therefore, it is highly necessary to design an efficient height-limiting pole detection algorithm that can automatically determine the position of the height-limiting pole and alert the driver when a height-limiting pole is found in the scene, which can avoid many traffic accidents to a certain extent.

[0003] The object detection task is one of the fundamental and important research directions in the field of computer vision, aiming to detect the position and category of objects in an image. This task is widely applied in unmanned driving systems and robot interaction systems. However, the current research on object detection focuses more on pedestrian detection, sign detection, and vehicle detection, and height-limiting pole detection has not received much attention. Therefore, the present invention has conducted some research on this gap in height-limiting pole detection. In recent years, the emergence of convolutional neural networks has promoted the further development of the object detection task. Object detectors based on convolutional neural networks can be divided into two-stage detectors and single-stage detectors according to whether they require a Region Proposal Network. Two-stage detectors often have a very complex network structure and a relatively slow inference speed because they need to generate region proposals, and generally it is difficult to meet the real-time requirement. While the network structure of single-stage detectors is generally relatively simple and the inference speed is faster. The YOLO (You Only Look Once) series, as an outstanding single-stage detection algorithm, perfectly balances the relationship between detection accuracy and speed. Its latest version, YOLOv5, has an even faster inference speed, and it is user-friendly using the Pytorch framework and is easier to integrate into production. Further, the YOLOv5s model in YOLOv5 has the best light weight and the fastest inference speed.

[0004] As far as we know, there is currently no learning-based height limit pole detection algorithm that can achieve real-time performance. Therefore, in view of the problem of detecting height limit poles on traffic roads, the present invention proposes a deformable module based on deformable convolution and a YOLOv5s detection optimization algorithm based on this deformable module. The constructed deformable module can be embedded into the Backbone of the YOLOv5s detector to improve the YOLOv5s detector so that it can serve the detection task of height limit poles in intelligent transportation and autonomous driving systems. Summary of the Invention

[0005] The present invention first constructs a deformable module based on deformable convolution. This module can replace ordinary convolution and complete a more flexible and more object geometry-adaptive convolution operation by adjusting the position of the convolution kernel. Then, the deformable module is embedded into the Backbone of the YOLOv5s detector to improve the YOLOv5s detector so that it can serve the detection task of height limit poles in intelligent transportation and autonomous driving systems. Experiments show that the improved YOLOv5s can achieve good detection results in the problem of height limit pole detection.

[0006] The technical solution of the present invention is as follows:

[0007] A height limit pole detection algorithm with high precision and light weight is proposed. This algorithm is designed and improved based on YOLOv5s and deformable convolution. The specific design features are as follows:

[0008] Different from the traditional YOLOv5s detection network, the Backbone part of the present invention uses a deformable module based on deformable convolution to extract basic features. The deformable module flexibly adjusts the sampling position of the convolution kernel and fuses it with low-level features, thereby obtaining a more object geometry-adaptive feature representation and improving the quality of the basic features extracted by the Backbone.

[0009] A height limit pole detection algorithm with high precision and light weight, the steps are as follows:

[0010] Step S1: At the input end, perform Mosaic data augmentation on the input image and scale the processed image to a unified size, such as 640×640.

[0011] Step S2: Input the result of Step S1 into the Backbone, and sequentially pass through each module to extract basic features. The Backbone includes a deformable module constructed based on deformable convolution.

[0012] Step S3: Input the feature map obtained after passing through the Backbone into the Neck for further processing to obtain three different scales of features: large, medium, and small.

[0013] Step S4: Input the three types of features into the multi-head detector for prediction respectively, and then obtain the final target positions and categories through non-maximum suppression operation.

[0014] Advantages of the present invention:

[0015] The present invention improves the YOLOv5s detector by using the constructed deformable module, and realizes the real-time detection of height-limiting poles in the scene. The present invention uses a real height-limiting pole dataset for evaluation experiments, and finally obtains a detection accuracy better than that of the baseline YOLOv5s. Description of the drawings

[0016] Figure 1 It is a traffic accident diagram caused by a height-limiting pole;

[0017] Figure 2 It is the detailed information of the deformable module constructed by the present invention, and two deformable convolutional layers are stacked in the experiment;

[0018] Figure 3 It is the flow chart of the YOLOv5s optimization detection algorithm proposed by the present invention;

[0019] Figure 4 It is a road height-limiting pole diagram;

[0020] Figure 5 It is a road height-limiting pole detection diagram. Specific implementation manners

[0021] The following further describes the present invention in detail with reference to the drawings and specific implementation examples.

[0022] The first part, the construction of the deformable module based on deformable convolution, the steps are as follows:

[0023] Step S1: Construction of the deformable convolution module (DBH)

[0024] The common convolution module (CBH) in the detection network is usually composed of a convolution operation, a Batchnorm normalization operation, and an activation function in sequence. The present invention replaces the convolution operation in the convolutional layer with a deformable convolution with a modulation mechanism having a convolution kernel of 3×3, a stride of 1, and a padding of 0, keeps the normalization operation, and selects Hardwish as the activation function to obtain the deformable convolution module. The deformable convolution module can more flexibly adjust the position of the convolution kernel through deformable convolution, so as to obtain a better feature representation without changing the size of the feature map.

[0025] Step S2: Feature fusion

[0026] Repeatedly and continuously stacking N deformable convolution modules can obtain a more flexible feature representation. At the same time, in order to maintain the quality of the features, the feature map initially input before the N deformable convolution modules is added element-wise to the result obtained after passing through the N deformable convolution modules for feature fusion. In the experiment, N = 2 is taken, that is, two deformable convolution modules are stacked, as Figure 2 shown.

[0027] Step S3: Further extract semantic information.

[0028] The result after feature fusion is input into a convolution module with a convolution kernel of 3×3 and a stride of 2 for downsampling to further refine the semantic context information.

[0029] Through the above three steps, a deformable module based on deformable convolution is constructed.

[0030] The second part, a high-precision and lightweight height-limiting pole detection algorithm, the steps are as follows:

[0031] Step S1: Data augmentation at the input end;

[0032] To solve the problem of difficult detection of small targets with a small number, at the input end, 4 images are randomly scaled and randomly cropped, and then stitched in a randomly arranged manner. Then the stitched image is scaled to a unified size, such as 640×640, and input into the subsequent detection network.

[0033] Step S2: Use the Backbone to extract basic features;

[0034] Furthermore, the detailed process of step S2 is as follows:

[0035] Step S21: The input image passes through the Focus module, a convolution module (convolution with 3×3 followed by batch normalization using the Batchnorm layer and the Hardwish activation function), and the CSP1_1 module to extract image features.

[0036] Step S22: The result of step S21 is used as the input of the deformable module to obtain a feature representation containing richer information.

[0037] Step S23: The result of the deformable module is continuously input into other modules in the Backbone part of YOLOv5s, which are the CSP1_3 module, a convolution module, the CSP1_3 module, a convolution module, the SPP (Spatial Pyramid Pooling) module, and the CSP2 module in sequence to complete the extraction of basic features.

[0038] Step S3: The basic features are fused through the Neck part of YOLOv5s to obtain three different-scale features;

[0039] It mainly includes the Feature Pyramid Networks that convey strong semantic features from top to bottom, and the Path Aggregation Network that conveys strong localization features from bottom to top. The combined use of the two further improves the feature extraction ability.

[0040] Step S4: Input the features of three different scales into the detection head, and perform non-maximum suppression operation to identify and locate the target;

[0041] Furthermore, the specific process of step S4 is as follows:

[0042] Step S41: Input the features of three different scales obtained in the previous stage into the detection head, and finally obtain feature maps of three sizes of 20×20, 40×40, and 80×80 respectively, which are responsible for predicting targets of large, medium, and small sizes respectively.

[0043] Step S42: Filter out the redundant prediction boxes obtained in the previous step. The specific steps are as follows: First, select the prediction box with the largest confidence, and then calculate the intersection-over-union ratio of this box and the remaining prediction boxes. If the intersection-over-union ratio of a certain prediction box and this box is greater than the pre-set threshold, then delete this prediction box. Repeat the above two steps for the remaining prediction boxes until all prediction boxes are processed.

[0044] The above four steps construct a YOLOv5s optimized detection algorithm that can perform height limit bar detection. As Figure 3 shown, the deformable module constructed based on deformable convolution is surrounded by a dashed rectangular box. The example of the road height limit bar is as Figure 4 shown. After the YOLOv5s optimized detection algorithm proposed by the present invention, the road height limit bar can be framed by a prediction box with the "Height limit bar" label, and the specific effect is as Figure 5 shown.

Claims

1. A height-limiting pole detection algorithm with high precision and light weight, characterized in that, The steps are as follows: Step S1: At the input end, perform Mosaic data augmentation on the input image and scale the processed image to a unified size; Step S2: Input the result of Step S1 into the Backbone, and extract basic features through each module in sequence; The Backbone includes a deformable module constructed based on deformable convolution; Step S3: Input the feature map obtained through the Backbone into the Neck for further processing to obtain features of three different scales, large, medium, and small; Step S4: Input the three types of features into the multi-head detector for prediction respectively, and then obtain the final target positions and categories through non-maximum suppression operations; The construction of the deformable module constructed by deformable convolution is as follows: Step S01: Construction of the deformable convolution module In the detection network, the convolution module CBH consists of a convolution operation, a Batchnorm normalization operation, and an activation function in sequence; replace the convolution operation in the convolution module CBH with a deformable convolution with a modulation mechanism, a convolution kernel of 3×3, a stride of 1, and a padding of 0, keep the normalization operation, and select Hardwish as the activation function to obtain the deformable convolution module; Step S02: Feature fusion Stack N deformable convolution modules repeatedly and continuously to obtain a feature representation; at the same time, to maintain the quality of the features, add the feature map input at the beginning before the N deformable convolution modules element-wise to the result obtained through the N deformable convolution modules for feature fusion; Step S03: Further extract semantic information; Input the result after feature fusion into a convolution module CBH with a convolution kernel of 3×3 and a stride of 2 for downsampling to further refine the semantic context information.

2. The height-limiting pole detection algorithm with high precision and light weight as described in claim 1, wherein The detailed process of Step S2 is as follows: Step S21: The input image passes through the Focus module, the convolution module CBH, and the CSP1_1 module to extract image features. The convolution module CBH performs a 3×3 convolution and then is followed by a Batchnorm layer for normalization operation and a Hardwish activation function; Step S22: Use the result of Step S21 as the input to the deformable module; Step S23: Continue to input the result of the deformable module into other modules in the Backbone part of YOLOv5s, which are the CSP1_3 module, the convolution module CBH, the CSP1_3 module, the convolution module CBH, the SpatialPyramid Pooling module, and the CSP2 module to complete the extraction of basic features.

3. The high-precision and lightweight height-limiting pole detection algorithm according to claim 1 or 2, characterized in that, The detailed process of Step S4 is as follows: Step S41: Input the three different scales of features obtained in the previous stage into the detection head, and finally obtain feature maps of three sizes, 20×20, 40×40, and 80×80, which are responsible for predicting targets of large, medium, and small sizes respectively; Step S42: Filter out the redundant prediction boxes obtained in the previous step.

4. The height limit pole detection algorithm with high precision and light weight as described in claim 3, characterized in that, The detailed process of Step S42 is as follows: First, select the prediction box with the highest confidence, and then calculate the intersection over union (IoU) between this prediction box with the highest confidence and the remaining prediction boxes; if the IoU of a certain prediction box with the prediction box with the highest confidence is greater than a pre-set threshold, then delete this certain prediction box; repeat the above two steps for the remaining prediction boxes until all prediction boxes are processed.

Images