Expressway cone barrel target detection method

Abstract

The invention discloses an expressway cone barrel target detection method. The method comprises the following steps: 1, obtaining a road picture through an unmanned aerial vehicle; step 2, performing feature extraction on the road image through a cone barrel detection model, and performing feature extraction on the irregular shape by the cone barrel detection model by using a deformable convolution (DCNv3) module; and 3, introducing an Inner-IOU loss function based on an auxiliary frame into the cone-barrel detection model, and calculating IOU loss through the auxiliary frame so as to improve bounding box regression loss. According to the invention, a deformable convolution DCNv3 module is fused to adapt to an irregular state, obtain important features of a target and reduce missing detection and error detection, so that the algorithm has robustness for traffic signs under different scales, angles and illumination conditions; inner-IOU is introduced, the bounding box regression loss is improved, and the method is particularly suitable for detection of non-uniformly distributed targets or targets of different scales, which is beneficial for improving the detection efficiency.

Description

Technical Field

[0001] The invention belongs to the field of highway cone target detection, in particular to a highway cone target detection method. Background Art

[0002] With the continuous advancement of urban construction and infrastructure construction, the scale and complexity of construction site operations are increasing. However, the safety of construction sites has always been the focus of the industry, and various potential hazards have brought safety hazards to construction workers and pedestrians. Among these potential hazards, the lack of obvious warning signs is one of the main causes of accidents. Detecting cones in construction areas is a key link in ensuring road traffic safety, especially in construction areas, which often have road surface changes, limited vision and potential dangers due to road construction, maintenance work or other activities. In such an environment, strict management of traffic flow and advance warning become particularly important. Cones, with their eye-catching orange appearance and reflective properties, serve as important visual cues at construction sites. They can not only attract the attention of drivers and prompt them to slow down when approaching the construction area, but also effectively convey changes in road conditions ahead and remind drivers to make corresponding driving adjustments. By arranging cones in an orderly manner around the construction area, a clear boundary can be formed, which not only helps to isolate the construction area from the driving area, prevent vehicles from mistakenly entering the construction area, and reduce the risk of accidents in the construction area, but also provides a safe working environment for construction workers. The detection of cones also provides a real-time safety monitoring method for the construction area. By ensuring that the cones are arranged as required and have not been lost or moved due to wind, vehicles or other factors, it ensures that the safety warnings in the construction area are always effective.

[0003] With the rapid development of deep learning-based target detection technology, highway cone detection is combined with drone aerial photography. Drones provide a high-altitude perspective in the detection task, but the image from the high-altitude perspective will bring about the problem of large changes in target scale and mutual occlusion between targets, which will lead to false detection and missed detection problems when using conventional target detection algorithms for detection tasks.

[0004] In recent years, with the rapid development of deep learning technology, researchers at home and abroad have gradually applied deep learning technology to target detection. Target detection algorithms based on deep learning can be divided into one-stage detection algorithms and two-stage detection algorithms. The two-stage detection algorithm performs target positioning and classification in stages, mainly including R-CNNFaster-R CNN and other algorithms. These algorithms have high detection accuracy, but have problems such as large computational complexity, slow running speed, and low real-time performance, making them difficult to deploy on mobile devices. The one-stage detection algorithm directly identifies the category and position of the target in the image, and the computational complexity is small. With the iterative update of the one-stage algorithm, this type of algorithm can simultaneously meet the requirements of the unmanned driving system for accuracy and real-time performance. The mainstream one-stage algorithm is the You Only Look Onc (YOLO) series. The YOLO algorithm has the characteristics of small computational complexity, high real-time performance, and a wide range of application scenarios, such as YOLOv1, YOLOv2, YOLOv3, YOLOv4, YOLOv5, YOLOv8, etc.

[0005] The cone barrel occupies few pixels in the picture, has low resolution and weak expression ability, and the complex background information in the picture is prone to false detection and missed detection, and the detection accuracy is low. In the research of target detection algorithms in recent years, Tan Liang et al. added deep hyperparameter convolution Slim Neck paradigm and SPD-Conv module to YOLOv5, replaced the Loss in YOLOv5, solved the problems of false detection and missed detection in target detection, and effectively improved the accuracy, but the parameters and calculation amount of the improved model increased; Cheng Huanxin et al. added EMA multi-scale attention to YOLOv8, improved the C2f module, and proposed a lightweight Bi-YOLOv8 feature pyramid network, and used the W Iou Loss loss function, which effectively reduced the number of network parameters, but the detection accuracy was not significantly improved. Yu Junyu et al. added concentrated feature pyramid CFP and mixed attention ACmix, as well as WIOU loss function to YOLOv7, mainly to solve the influence of target scale difference and complex detection background in remote sensing target detection, but there is still a certain degree of false detection and missed detection. These methods have some shortcomings to varying degrees. For example, in order to improve accuracy, the number of model parameters and the amount of calculation are increased. In the face of complex scenes, small targets are often missed, and the real-time detection efficiency needs to be improved. Summary of the invention

[0006] The invention provides a highway cone target detection method, which is used to solve the defects in the prior art.

[0007] The present invention is achieved through the following technical solutions:

[0008] A method for detecting cone targets on a highway comprises the following steps:

[0009] Step 1: Obtain road images through drones;

[0010] Step 2: Extract features from road images through the cone detection model. The cone detection model uses the deformable convolution DCNv3 module (deformable convolution v3) to extract features from irregular shapes.

[0011] Step 3: The cone bucket detection model introduces the Inner-IOU loss function based on the auxiliary border. The IOU loss is calculated by the auxiliary border, which can improve the bounding box regression loss.

[0012] In the above-mentioned highway cone target detection method, the enhancement means of the cone detection model include mosaic enhancement (Mosaic), mixed enhancement (Mixup), spatial perturbation (random perspective) and color perturbation (HSV augment).

[0013] In the highway cone target detection method as described above, the backbone network structure of the cone detection model is a C2f module.

[0014] As described above, in the highway cone target detection method, the cone detection model adopts a decoupling head structure, and two parallel branches extract category features and position features respectively, and then each uses 1×1 convolution to complete the classification and positioning tasks.

[0015] In the above-mentioned highway cone target detection method, the operation process of the deformable convolution DCNv3 module is as follows: Among them, {ΔP n |n=1,2,…,N},N=|R|, corresponding to each position of offsets in the figure.

[0016] In the above-mentioned highway cone target detection method, the deformable convolution DCNv3 module adds a two-dimensional offset ΔP on the basis of the original offset R. n (offset on the x and y axes), this ΔP n The value of corresponds to the value of the position corresponding to offsets in the figure. Since offsets is a floating point value, it does not correspond to a real position on the feature map. Therefore, the value of this position is obtained by calculating the bilinear difference of the four surrounding real values. The bilinear difference is obtained by the following formula

[0017] G(q,p)=g(q x ,p x )·g(q y ,p y ),in,

[0018] g(a,b)=max(0,1-|ab|).

[0019] In the above-mentioned highway cone target detection method, a 3×3 convolution with padding=3 is used on the original feature map in the operation of the deformable convolution DCNv3 module to obtain a feature map with the same length and width as the original feature map and 2N channels; each feature point on the map has 2N values: 2 corresponds to the offset of the x and y axes, and N corresponds to N ΔP n For each P 0 , P n There are N values, corresponding to the size of the convolution kernel, ΔP n There are also N values, corresponding to the N channels of the offset field feature map. For each point on the output feature map, the spatial position of the 3×3 feature point sampled on the original image can be determined individually.

[0020] In the above-mentioned highway cone target detection method, the calculation method of applying the Inner-IOU loss function to the CIOU loss function is as follows:

[0021]

[0022]

[0023]

[0024]

[0025]

[0026] union=(w gt ×h gt )×(ratio) 2 +(w×h)×(ratio) 2 -inter

[0027]

[0028] L Inner-CIOU =L CIOU +IOU-IOU inner .

[0029] In the above-mentioned highway cone target detection method, the Inner-IOU loss function controls the size of the auxiliary bounding box through the scale factor ratio, realizes dynamic adjustment for different detection tasks and detection targets, and makes the cone detection model have better generalization ability.

[0030] The advantages of the present invention are: the present invention integrates the deformable convolution DCNv3 module to adapt to irregular states, obtain important features of the target, reduce missed detections and false detections, so that the algorithm has the robustness of traffic signs under different scales, angles and lighting conditions; the introduction of Inner-IOU improves the bounding box regression loss, which is particularly suitable for detecting unevenly distributed targets or targets of different scales, which helps to improve detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0032] Figure 1 It is a schematic diagram of cone barrel target detection of the present invention;

[0033] Figure 2 The structural diagram of the present invention is the working principle of the deformable convolution DCNv3 module;

[0034] Figure 3 It is a schematic diagram of the Inner-IOU calculation method of the present invention under the smaller scale and larger scale conditions;

[0035] Figure 4 This is the mAP@0.5 curve obtained by cone bucket detection training of the algorithm of the present invention and the YOLOv8 algorithm. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] like Figure 1 As shown, a highway cone target detection method includes the following steps:

[0038] Step 1: Obtain road images through drones;

[0039] Step 2: Extract features from road images through the cone detection model. The cone detection model uses the deformable convolution DCNv3 module (the working principle of the deformable convolution DCNv3 module is to learn an offset based on a network, so that the convolution kernel is offset at the sampling point of the input feature map and focuses on the area or target of interest in this article, such as Figure 2 As shown), feature extraction is performed on irregular shapes;

[0040] Step 3: The cone bucket detection model introduces the Inner-IOU loss function based on the auxiliary border. The IOU loss is calculated by the auxiliary border, which can improve the bounding box regression loss.

[0041] Preferably, the enhancement means of the cone barrel detection model described in this embodiment include mosaic enhancement (Mosaic), mixed enhancement (Mixup), spatial perturbation (random perspective) and color perturbation (HSV augment).

[0042] Preferably, the backbone network structure of the cone barrel detection model described in this embodiment is a C2f module.

[0043] Preferably, the cone barrel detection model described in this embodiment adopts a decoupling head structure, and two parallel branches extract category features and position features respectively, and then each uses 1×1 convolution to complete the classification and positioning tasks.

[0044] Preferably, the operation process of the deformable convolution DCNv3 module described in this embodiment is: Among them, {ΔP n |n=1,2,…,N},N=|R|, corresponding to each position of offsets in the figure.

[0045] Preferably, in the operation of the deformable convolution DCNv3 module described in this embodiment, a two-dimensional offset ΔP is added on the basis of the original offset R. n (offset on the x and y axes), this ΔP n The value of corresponds to the value of the position corresponding to offsets in the figure. Since offsets is a floating point value, it does not correspond to a real position on the feature map. Therefore, the value of this position is obtained by calculating the bilinear difference of the four surrounding real values. The bilinear difference is obtained by the following formula G(q,p)=g(q x ,p x )·g(q y ,p y ), where g(a,b)=max(0,1-|ab|).

[0046] Preferably, in the operation of the deformable convolution DCNv3 module described in this embodiment, a 3×3 convolution with padding=3 is used on the original feature map to obtain a feature map with the same length and width as the original feature map and 2N channels; each feature point on the map has 2N values: 2 corresponds to the offset of the x and y axes, and N corresponds to N ΔP n For each P 0 , P n There are N values, corresponding to the size of the convolution kernel, ΔP n There are also N values, corresponding to the N channels of the offset field feature map. For each point on the output feature map, the spatial position of the 3×3 feature point sampled on the original image can be determined individually.

[0047] In current technology, the IOU loss function is widely used in computer vision tasks. The IOU-based loss function used by the original YOLOv8 is the CIOU Loss, which is consistent with YOLOv5 and is a loss function commonly used in classification problems. It measures the accuracy of the target detection model in detecting the position and size of the target, and can better handle the overlap and misalignment between target boxes, thus improving the performance evaluation of the model. The calculation method of CIoU Loss is as follows: Among them, v only contains the aspect ratio to be predicted. Since w and h of the detected target are relatively small in small target detection, CIOU cannot reflect the actual situation and may optimize the similarity in an unreasonable way.

[0048] Therefore, in actual target detection application scenarios, CIOU Loss has certain limitations because the scale of objects may change greatly, especially in small target detection, samples are greatly affected by environmental factors. On the one hand, its judgment ability for objects with a large aspect ratio and irregular objects is weak; on the other hand, it cannot be adaptively adjusted for different detection tasks and detection targets, and does not consider the balance between different samples. When the sample differences are large, it will also restrict the convergence speed of the model. Therefore, CIOU Loss does not have good judgment and generalization capabilities in small target detection. To solve the above limitations, this paper introduces the Inner-IOU loss function based on auxiliary borders, and calculates the IOU loss through auxiliary borders. Using smaller-scale auxiliary borders to calculate the IOU loss will help high-IOU sample regression and achieve the effect of accelerated convergence. On the contrary, using larger-scale auxiliary borders to calculate the IOU loss can accelerate the regression process of low-IOU samples. Based on this, for different data sets and detectors, Inner-IOU introduces a scale factor ratio to control the scale of the auxiliary border for calculating the loss, which can obtain regression results faster and more effectively, such as Figure 3As shown in the figure, they are schematic diagrams of the Inner-IOU calculation method in the case of smaller scale and larger scale.

[0049] Preferably, the calculation method of applying the Inner-IOU loss function described in this embodiment to the CIOU loss function is as follows:

[0050]

[0051]

[0052]

[0053]

[0054]

[0055] union=(w gt ×h gt )×(ratio) 2 +(w×h)×(ratio) 2 -inter

[0056]

[0057] L Inner-CIOU =L CIOU +IOU-IOU inner .

[0058] Preferably, the Inner-IOU loss function described in this embodiment controls the size of the auxiliary bounding box through the scale factor ratio, realizes dynamic adjustment for different detection tasks and detection targets, and makes the cone bucket detection model have better generalization ability. Compared with the traditional IOU calculation method that considers the overall overlapping area between the predicted bounding box and the overall bounding box, Inner-IOU provides a more accurate judgment of the overlapping area by focusing on the core part of the bounding box. Inner-IOU is a more detailed and more focused performance evaluation indicator on the target center. It improves the accuracy and efficiency of the target detection task by adjusting the scale of the auxiliary box. In small target detection, the characteristics of Inner-IOU are brought to the extreme, and the detection target can be accurately judged in a small range.

[0059] Experimental results and analysis

[0060] Experimental environment

[0061] The names and parameters of the server configuration and virtual environment configuration used in the experiment are shown in Table 1.

[0062]

[0063] Table 1 Experimental environment

[0064] In the experiment, uniform parameters were used for training: the input image resolution was 640×640, and the model was optimized with the stochastic gradient descent (SGD) optimizer, the total number of training rounds (epochs) was set to 200, the initial learning rate was set to 0.001, the weight decay was set to 0.005, the momentum parameter was set to 0.9, the batch size (batch) was set to 8, the number of working threads (works) was set to 8, and other parameters used the default values.

[0065] Experimental Dataset

[0066] The experimental data set in this paper comes from a video data set of a highway collected by a drone. After frame processing, available pictures are selected as data sets for assistance, including highway cone images and corresponding annotation files in scenes such as main lines, tunnels, steep slopes, and sharp bends; the original 1400 pictures are expanded to 8000 as the final PCB board defect data set; the training set, validation set, and test set are randomly divided in a ratio of 8:1:1, and 6400 of them are selected as training sets for training, 800 as validation sets, and 800 as test sets.

[0067] Evaluation indicators

[0068] In order to facilitate the evaluation of the performance of the improved model, accuracy (Precision, P), recall (Recall, R), mean Average Precision (mAP), Giga Floating-point Operations Per Second (GFLOPs), number of frames detected per second (Frames Per Second, FPS), weight file, and parameter quantity (Params / M) are used as evaluation indicators.

[0069] The accuracy (P) is the ratio of correct predictions among all the results predicted as positive samples. It is defined as follows:

[0070]

[0071] Recall (R) refers to the proportion of all actual positive examples that the model correctly predicts as positive examples. It is defined as follows:

[0072]

[0073] Among them, TP represents the number of samples predicted correctly in the detection results, FP represents the number of samples predicted incorrectly in the detection results, and FN represents the number of samples that have not been detected among all correct targets.

[0074] The average precision (mAP) comprehensively considers the precision (P) and recall (R) of m categories, and more comprehensively reflects the performance of the network. The definition is as follows:

[0075]

[0076] Among them, mAP is used to evaluate the effect of improving the detection accuracy of the model. The larger the value, the higher the model detection accuracy; FPS represents the number of images that the model can detect per second. The larger the value, the better the real-time performance of the model. It is generally believed that FPS greater than 30 means that the model meets the real-time detection function; GFLOPs, model size, and model parameter quantity reflect the degree of lightweight of the model. The smaller these values ​​are, the more lightweight the model is and the lower the hardware performance requirements are.

[0077] Experimental process and result analysis

[0078] 3.4.1 Ablation Experiment

[0079] In order to verify the effectiveness of adding DCNv3 module and Inner-IOU to YOLOv8 algorithm, ablation experiment was conducted to evaluate the influence of each module on the detection performance of YOLOv8. YOLOv8 model was used as the baseline model of this experiment. The dataset used the cone bucket dataset established by ourselves. Each algorithm was run for 200 rounds and the results were recorded in Table 2.

[0080]

[0081] Table 2 Ablation experiment

[0082] As can be seen from the data in Table 2, the experimental results show that when the deformable convolution DCNv3 module is used, mAP@0.5 increases by 0.5%, and the weight file increases by 0.2M. The computational cost of the model is balanced while the detection accuracy is maximized. This is because the deformable convolution strengthens the modeling ability of object deformation, which is conducive to the information fusion of local features and global features. The model can fit defect targets of different directions and sizes more accurately, improve the model's performance in detecting defects with large shape differences, and improve the overall parameters of the model, so that the network can perceive defect features faster. Effective measures need to be taken to compensate, and adding Inner-IOU will reduce the number of parameters. Experiments show that compared with the YOLOv8 algorithm, the mAP@0.5 index of the method of the present invention is increased from 90.5% to 93.6%, and the number of parameters is increased by 26% compared with the YOLOv8 algorithm, while the detection accuracy of the detection model for the target is improved.

[0083] Depend on Figure 4It can be seen that after more than 200 trainings, the mAP@0.5 curve of the optimization algorithm is always higher than that of the YOLOv8 algorithm, indicating that the algorithm proposed in this paper has better detection accuracy.

[0084] 3.4.2 Comparative Experimental Effectiveness Analysis

[0085] In order to evaluate the effectiveness of the method of the present invention, the method of the present invention is compared with YOLOv5, YOLOv6, YOLOv7, YOLOv7-tiny, Faster R-CNN, and CenterNet under the same data set, and the results are shown in Table 3.

[0086]

[0087] Table 3

[0088] From the data in Table 3, it can be seen that the improved algorithm of the present invention is superior to the current target detection algorithm in terms of parameter quantity, calculation speed and detection accuracy on the data set. The experimental comparison results show that the improved algorithm has greatly improved performance and lightness compared with the current mainstream detection algorithm, and is more suitable for the deployment and application of target detection models in small devices.

[0089] In summary, firstly, the deformable convolution DCNv3 module is used to have better feature extraction capabilities for irregular shapes in feature maps, so that the backbone network can better adapt to irregular spatial structures and focus on important targets more accurately, thereby improving the model's detection capabilities for occluded and overlapping targets. Finally, the Inner-IOU loss function based on auxiliary borders is introduced, and the IOU loss is calculated through auxiliary borders. Using smaller-scale auxiliary borders to calculate the IOU loss will help high IOU sample regression and achieve the effect of accelerated convergence. The experimental results show that compared with the original YOLOv8 algorithm, the improved YOLOv8 algorithm can maintain improved accuracy while maintaining lightweight and increasing detection speed, meeting the requirements of highway cone barrel detection environment deployment from the perspective of drones. The research method proposed in this paper can be applied to traffic application scenarios and can also be extended to other small target detection.

[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting cone targets on highways, characterized in that: The steps include: Step 1: Obtain road images through drones; Step 2: Extract features from road images through the cone detection model. The cone detection model uses the deformable convolution DCNv3 module to extract features from irregular shapes. Step 3: The cone bucket detection model introduces the Inner-IOU loss function based on the auxiliary border. The IOU loss is calculated by the auxiliary border, which can improve the bounding box regression loss.

2. The highway cone target detection method according to claim 1 is characterized by: The enhancement means of the cone barrel detection model include mosaic enhancement, mixed enhancement, space disturbance and color disturbance.

3. The highway cone target detection method according to claim 1 is characterized by: The backbone network structure of the cone barrel detection model is a C2f module.

4. The highway cone target detection method according to claim 1 is characterized in that: The cone barrel detection model adopts a decoupling head structure, where two parallel branches extract category features and position features respectively, and then each uses 1×1 convolution to complete the classification and positioning tasks.

5. The method for detecting cone targets on highways according to claim 1, characterized in that: The operation process of the deformable convolution DCNv3 module is as follows: Among them, {ΔP n |n=1,2,…,N},N=|R|, corresponding to each position of offsets in the figure.

6. A highway cone target detection method according to claim 5, characterized in that: In the operation of the deformable convolution DCNv3 module, a two-dimensional offset ΔP is added on the basis of the original offset R. n (offset on the x and y axes), this ΔP n The value of corresponds to the value of the position corresponding to offsets in the figure. Since offsets is a floating point value, it does not correspond to a real position on the feature map. Therefore, the value of this position is obtained by calculating the bilinear difference of the four surrounding real values. The bilinear difference is obtained by the following formula G(q,p)=g(q x ,p x )·g(q y ,p y ), where g(a,b)=max(0,1-|ab|).

7. A highway cone target detection method according to claim 6, characterized in that: The deformable convolution DCNv3 module uses a 3×3 convolution with padding=3 on the original feature map to obtain a feature map with the same length and width as the original feature map and 2N channels. Each feature point on the map has 2N values: 2 corresponds to the offset of the x and y axes, and N corresponds to N ΔP n For each P0, P n There are N values, corresponding to the size of the convolution kernel, ΔP n There are also N values, corresponding to the N channels of the offset field feature map. For each point on the output feature map, the spatial position of the 3×3 feature point sampled on the original image can be determined individually.

8. The highway cone target detection method according to claim 1 is characterized by: The calculation method of applying the Inner-IOU loss function to the CIOU loss function is as follows: union=(w gt ×h gt )×(ratio) 2 +(w×h)×(ratio) 2 -inter L Inner-CIOU =L CIOU +IOU-IOU inner .

9. A highway cone target detection method according to claim 8, characterized in that: The Inner-IOU loss function controls the size of the auxiliary bounding box through the scale factor ratio, realizes dynamic adjustment for different detection tasks and detection targets, and makes the cone bucket detection model have better generalization ability.

Images