A multi-scale feature self-correcting YOLOv5 road small target detection method
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NANJING UNIV OF POSTS & TELECOMM
- Filing Date
- 2023-12-15
- Publication Date
- 2026-05-29
Smart Images
Figure CN117710652B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of target detection technology, specifically to a road small target detection method using multi-scale feature self-correction YOLOv5. Background Technology
[0002] With the rapid development of artificial intelligence technology, intelligent driving has become one of the important trends in modern transportation. However, despite significant progress, intelligent driving systems still face a series of challenges, including insufficient accuracy, missed detection of small targets, and false detections. These challenges hinder the safety, reliability, and practical feasibility of intelligent driving vehicles; therefore, solving these problems is crucial.
[0003] Intelligent driving systems in automobiles rely on highly accurate perception, decision-making, and control systems to ensure safe driving in various complex environments. However, current intelligent driving systems still perform poorly in certain situations, such as in adverse weather conditions, low-light nighttime driving, or complex urban traffic. This lack of precision can lead to inaccurate predictions and decision-making regarding vehicle behavior, thereby increasing the risk of accidents.
[0004] On urban roads and highways, intelligent driving systems for automobiles must be able to accurately detect and identify targets of various sizes, including pedestrians, bicycles, motorcycles, and other vehicles. However, the detection of small targets has always been a challenging problem. Existing algorithms still have significant issues in detecting small targets, and are prone to both false negatives (missing detection of small targets) and false positives (incorrectly classifying non-target objects as targets). Summary of the Invention
[0005] Purpose of the invention: The purpose of this invention is to provide a road small target detection method of YOLOv5 with multi-scale feature self-correction that can maintain real-time performance and improve the accuracy of target detection on the road, thus providing safety, reliability and practical feasibility for intelligent driving; another purpose of this invention is to provide a new and effective method for small target detection.
[0006] Technical solution: The present invention provides a road small target detection method using multi-scale feature self-correction YOLOv5, comprising the following steps:
[0007] (1) Acquire road image data;
[0008] (2) Preprocess the input image;
[0009] (3) Multi-scale feature extraction and feature fusion are performed on the preprocessed image using the improved YOLOv5 network; the YOLOv5 network includes: Backbone, Neck, feature adaptive correction convolution, small target detection layer and output end;
[0010] (4) Output the detection results and evaluate their performance.
[0011] Furthermore, step (1) is as follows: image data is obtained using the KITTI road autonomous driving dataset.
[0012] Furthermore, step (2) is as follows: Mosaic data augmentation is applied to the input image; the input image is uniformly scaled.
[0013] Furthermore, step (3) includes the following steps:
[0014] (31) Backbone: The CSPNet network structure is used to extract features from the input image, and feature maps with scales of 80×80, 40×40 and 20×20 are obtained and output to the neck; the feature map with scale of 160×160 is output to the small target detection layer.
[0015] (32) Neck: The input 20×20 feature map is processed by feature adaptive correction convolution, and the output is a corrected 20×20 feature map. Then, it is upsampled to obtain a scale of 40×40. After concatenation with the 40×40 feature map output by Backbone, feature adaptive correction convolution is performed to obtain a corrected 40×40 feature map. Following the above steps, the 40×40 feature map is upsampled, concatenated, and processed by feature adaptive correction convolution to obtain an 80×80 feature map, which is then output to the small target detection layer. The 160×160 feature map output by the small target detection layer is then processed by continuous feature adaptive correction convolution and concatenation to obtain feature maps of different scales of 80×80, 40×40, and 20×20, which are then output to the output end.
[0016] (33) Small target detection layer: The 80×80 feature map is processed by feature adaptive correction convolution to obtain a 160×160 feature map that has been corrected multiple times at different scales and in space. The feature map is then output to Neck and the output end.
[0017] (34) Output: The feature maps of different scales are obtained for prediction and classification, and the regression loss is calculated. The regression loss includes classification loss, target loss and target box loss. Among them, the SEIoU loss function is used to calculate the loss of the target box.
[0018] Furthermore, the feature-adaptive correction convolution is as follows:
[0019] The input feature map X is downsampled through an average pooling layer to increase the receptive field of the features, and the output T is calculated as follows:
[0020] T = AvgPoolr (X)
[0021] After further convolution with kernel K1 and nearest neighbor interpolation upsampling, the output X' is as follows:
[0022] X' = Up(F1(T)) = Up(T*K1)
[0023] Set an adaptive weight p, add X' to the input X according to the weights of the performance index, and obtain the correction index a. The calculation formula is as follows:
[0024] a=(1-p)×X+p×X'
[0025] The correction index 'a' is processed by the Sigmoid function, as shown in the following formula:
[0026]
[0027] The original image X is then filtered by a K2 convolution kernel, and the result is multiplied element-wise to output Y', which further highlights the target features, as shown in the following formula:
[0028] Y'=X*K2·S(a)
[0029] Finally, after K3 filtering, the feature information of the target region after self-correction and enhancement is obtained, as shown in the following formula, which outputs feature Y.
[0030] Y = Y' * K3.
[0031] Furthermore, the adaptive weight p is the average value of the detection capabilities of each category in the road target, and the calculation formula is as follows:
[0032]
[0033] In the formula, n represents the number of road target categories, A i and R i A and R represent the detection precision and recall for each category, respectively. They can effectively improve feature extraction capabilities based on different targets and the balance of various performance indicators. The formulas for calculating A and R are as follows:
[0034]
[0035] Wherein, TP indicates that the predicted value is consistent with the actual value, and the predicted value of TP is a positive sample, i.e., the road target; FP and FN indicate that the predicted value is inconsistent with the actual value, FP indicates that the predicted value is a positive sample and the actual value is a negative sample, i.e., the road background; FN indicates that the predicted value is a negative sample and the actual value is a positive sample.
[0036] Furthermore, the SEIoU loss function is calculated as follows:
[0037]
[0038] In the formula, d represents the position loss, and the calculation formula is as follows:
[0039]
[0040]
[0041] Among them, the location loss mainly consists of factors such as the orientation and distance between the predicted bounding box and the ground truth bounding box; where ρ t The distance factor representing the distance between the two frames, where R refers to the x-coordinate of the center point of the predicted bounding box and the ground truth bounding box, respectively. w R h The meaning represented is the width and height of the circumscribed rectangle of the two frames; in the formula, α represents the orientation factor of the two frames. When the two frames are parallel or perpendicular to each other, that is, when α approaches 0 or π / 2, more emphasis is placed on the distance factor between the two frames. The orientation factor is calculated as follows:
[0042]
[0043] Among them, c w σ represents the width between the center points of the predicted bounding box and the ground truth bounding box, and c represents the distance between the center points of the two boxes. w The calculation formula is as follows:
[0044]
[0045] in, These refer to the x-coordinates of the center points of the predicted bounding box and the ground truth bounding box, respectively.
[0046] Furthermore, the formula for calculating the width and height loss of SEIoU is as follows:
[0047]
[0048] Among them, w,w gt H and h represent the widths of the predicted bounding box and the ground truth bounding box, respectively. gt These represent the heights of the predicted bounding box and the ground truth bounding box, respectively.
[0049] An electronic device according to the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded onto the processor, it implements a road small target detection method of YOLOv5 with multi-scale feature self-correction as described in any one of the claims.
[0050] The present invention provides a storage medium storing a computer program, which, when executed by a processor, implements a road small target detection method of YOLOv5 with multi-scale feature self-correction as described in any one of the claims.
[0051] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: (1) Setting an adaptive parameter P can effectively improve the feature extraction capability based on different road targets and the balance of various performance indicators, thereby improving the detection accuracy of the algorithm; (2) Feature adaptive correction can effectively enhance the feature information of small targets, improve the detection capability of small targets, and increase the stability and robustness of the algorithm; (3) The target box regression loss calculation adopts the SEIoU loss function, which can further improve the detection capability of small targets. The width and height loss adopts the accelerated convergence method, which reduces the inference time of the algorithm. Attached Figure Description
[0052] Figure 1 This is the network structure of the multi-scale feature self-correction YOLOv5 detection algorithm of the present invention;
[0053] Figure 2 This invention features an adaptive correction network structure.
[0054] Figure 3 For calculating the position loss between the predicted bounding box and the ground truth bounding box in this invention Detailed Implementation
[0055] The technical solution of the present invention will be further described below with reference to the accompanying drawings.
[0056] like Figure 1-2 As shown, this embodiment of the invention provides a road small target detection method using multi-scale feature self-correction YOLOv5, comprising the following steps:
[0057] (1) Obtain road image data; specifically: use the KITTI road autonomous driving dataset to obtain image data.
[0058] (2) Preprocess the input image; apply Mosaic data augmentation to the input image; perform uniform scaling operation on the input image; and output to the backbone network structure.
[0059] (3) Multi-scale feature extraction and feature fusion are performed on the preprocessed image using an improved YOLOv5 network; the YOLOv5 network includes: Backbone, Neck, feature adaptive correction convolution, small object detection layer and output; including the following steps:
[0060] (31) Backbone: The CSPNet network structure is used to extract features from the input image, resulting in feature maps of scales 80×80, 40×40, and 20×20, which are output to the neck. A feature map of scale 160×160 is output to the small object detection layer. The core idea of the CSPNet network structure is to divide the feature map into two parts. One part directly enters the next stage during forward propagation, while the other part is fused with the feature maps of subsequent stages through cross-stage connections. This structure helps to improve the expressive power and information flow of features. Three Bottleneck layers are concatenated with convolutions to form a C3 module, which enhances the learning ability of images while maintaining lightweight design.
[0061] (32) Neck: The input 20×20 feature map is processed by feature adaptive correction convolution, and the output is a corrected 20×20 feature map. Then, it is upsampled to obtain a scale of 40×40. After concatenation with the 40×40 feature map output by Backbone, feature adaptive correction convolution is performed to obtain a corrected 40×40 feature map. Following the above steps, the 40×40 feature map is upsampled, concatenated, and processed by feature adaptive correction convolution to obtain an 80×80 feature map, which is then output to the small target detection layer. The 160×160 feature map output by the small target detection layer is then processed by continuous feature adaptive correction convolution and concatenation to obtain feature maps of different scales of 80×80, 40×40, and 20×20, which are then output to the output end.
[0062] The upsampling method is nearest-neighbor interpolation, which selects the value of the original pixel closest to the target pixel location as the target pixel value. First, the position (x', y') of each pixel in the target image is calculated, where (x', y') are the coordinates in the target image. Then, based on (x', y'), the nearest pixel position (x, y) in the source image is found, where (x, y) are the coordinates in the source image. Nearest-neighbor interpolation sets the value of the pixel (x', y') in the target image to the pixel value at position (x, y) in the source image.
[0063] like Figure 2 As shown, the feature-adaptive correction convolution is specifically as follows:
[0064] The input feature map X is downsampled through an average pooling layer to increase the receptive field of the features, and the output T is calculated as follows:
[0065] T = AvgPool r (X)
[0066] After further convolution with kernel K1 and nearest neighbor interpolation upsampling, the output X' is as follows:
[0067] X' = Up(F1(T)) = Up(T*K1)
[0068] Set an adaptive weight p, add X' to the input X according to the weights of the performance index, and obtain the correction index a. The calculation formula is as follows:
[0069] a=(1-p)×X+p×X'
[0070] The calibration index 'a' enhances the ability to extract features from small targets. Based on the dynamic change of weight 'p', it adaptively strengthens feature information. When 'p' is initially small, it indicates that the feature extraction part extracts less feature information, so the weight of feature extraction X' is increased. As 'p' increases, while maintaining the increased weight of feature extraction X', the weight of the input X part is slowly reduced. When 'p' converges to a certain value and then decreases, it's because downsampling during feature extraction increases the receptive field, making it easy to ignore the feature information of small targets, which is detrimental to the feature extraction of some small targets. Therefore, it is necessary to increase the weight of the original input X part to recover the features of small targets.
[0071] The correction index 'a' is processed by the Sigmoid function, as shown in the following formula:
[0072]
[0073] The original image X is then filtered by a K2 convolution kernel, and the result is multiplied element-wise to output Y', which further highlights the target features, as shown in the following formula:
[0074] Y'=X*K2·S(a)
[0075] Finally, after K3 filtering, the feature information of the target region after self-correction and enhancement is obtained, as shown in the following formula, which outputs feature Y.
[0076] Y = Y' * K3.
[0077] Wherein, the adaptive weight p is the average value of the detection capability of each category in the road target, and the calculation formula is as follows:
[0078]
[0079] In the formula, n represents the number of road target categories, A i and R i A and R represent the detection precision and recall for each category, respectively. They can effectively improve feature extraction capabilities based on different targets and the balance of various performance indicators. The formulas for calculating A and R are as follows:
[0080]
[0081] Wherein, TP indicates that the predicted value is consistent with the actual value, and the predicted value of TP is a positive sample, i.e., the road target; FP and FN indicate that the predicted value is inconsistent with the actual value, FP indicates that the predicted value is a positive sample and the actual value is a negative sample, i.e., the road background; FN indicates that the predicted value is a negative sample and the actual value is a positive sample.
[0082] (33) Small target detection layer: The 80×80 feature map is processed by feature adaptive correction convolution to obtain a 160×160 feature map that has been corrected multiple times at different scales and in space. The feature map is then output to Neck and the output end.
[0083] (34) Output: The feature maps at different scales are obtained for prediction and classification, and the regression loss is calculated. The regression loss includes: classification loss, target loss, and target box loss; among them, the SEIoU loss function is used to calculate the loss of the target box. Figure 3 As shown, the SEIoU loss function is calculated using the following formula:
[0084]
[0085] In the formula, d represents the position loss, and the calculation formula is as follows:
[0086]
[0087]
[0088] Among them, the location loss mainly consists of factors such as the orientation and distance between the predicted bounding box and the ground truth bounding box; where ρ t The distance factor representing the distance between the two frames, where R refers to the x-coordinate of the center point of the predicted bounding box and the ground truth bounding box, respectively. w R h The meaning represented is the width and height of the circumscribed rectangle of the two frames; in the formula, α represents the orientation factor of the two frames. When the two frames are parallel or perpendicular to each other, that is, when α approaches 0 or π / 2, more emphasis is placed on the distance factor between the two frames. The orientation factor is calculated as follows:
[0089]
[0090] Among them, c w σ represents the width between the center points of the predicted bounding box and the ground truth bounding box, and c represents the distance between the center points of the two boxes. w The calculation formula is as follows:
[0091]
[0092] in, These refer to the x-coordinates of the center points of the predicted bounding box and the ground truth bounding box, respectively.
[0093] The formula for calculating the width and height loss of SEIoU is as follows:
[0094]
[0095] Among them, w,w gt H and h represent the widths of the predicted bounding box and the ground truth bounding box, respectively. gt These represent the heights of the predicted bounding box and the ground truth bounding box, respectively.
[0096] (4) Output the detection results and evaluate their performance.
[0097] This invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded onto the processor, it implements the road small target detection method of YOLOv5 with multi-scale feature self-correction as described in any one of the embodiments.
[0098] This invention also provides a storage medium storing a computer program, which, when executed by a processor, implements any of the road small target detection methods described in the embodiments of the present invention, namely, the multi-scale feature self-correction YOLOv5 method.
Claims
1. A road small target detection method using multi-scale feature self-correction YOLOv5, characterized in that, Includes the following steps: (1) Acquire road image data; (2) Preprocess the input image; (3) Multi-scale feature extraction and feature fusion are performed on the preprocessed image using an improved YOLOv5 network; wherein the YOLOv5 network includes: Backbone, Neck, feature adaptive correction convolution, small object detection layer and output end; including the following steps: (31) Backbone: The CSPNet network structure is used to extract features from the input image, and feature maps with scales of 80×80, 40×40 and 20×20 are obtained and output to the neck; the feature map with scale of 160×160 is output to the small target detection layer. (32) Neck: The input 20×20 feature map is processed by feature adaptive correction convolution, and the output is a corrected 20×20 feature map. Then, it is upsampled to obtain a scale of 40×40. After concatenation with the 40×40 feature map output by Backbone, feature adaptive correction convolution is performed to obtain a corrected 40×40 feature map. Following the above steps, the 40×40 feature map is upsampled, concatenated, and processed by feature adaptive correction convolution to obtain an 80×80 feature map, which is then output to the small target detection layer. The 160×160 feature map output by the small target detection layer is then processed by continuous feature adaptive correction convolution and concatenation to obtain feature maps of different scales of 80×80, 40×40, and 20×20, which are then output to the output end. (33) Small target detection layer: The 80×80 feature map is processed by feature adaptive correction convolution to obtain a 160×160 feature map that has been corrected multiple times in different scale spaces. The feature map is then output to the Neck and the output end. (34) Output: The feature maps of different scales are obtained for prediction and classification, and the regression loss is calculated. The regression loss includes classification loss and target box loss. Among them, the SEIoU loss function is used to calculate the loss of the target box. The feature adaptive correction convolution is as follows: The input feature map X is downsampled by an average pooling layer to increase the receptive field of the features, and the output T is calculated as follows: ； After further convolution with kernel K1 and nearest neighbor interpolation upsampling, the output X' is as follows: ； Set an adaptive weight p, add X' to the input X according to the weights of the performance index, and obtain the correction index a. The calculation formula is as follows: ； The correction index 'a' is processed by the Sigmoid function, as shown in the following formula: ； The original image X is then filtered by a K2 convolution kernel, and the result is multiplied element-wise to output Y', which further highlights the target features, as shown in the following formula: ； Finally, after K3 filtering, the feature information of the target region after self-correction and enhancement is obtained, as shown in the following formula, which outputs feature Y. The adaptive weight p is the average detection capability of each category in the road target, and the calculation formula is as follows: ； In the formula, n represents the number of road target categories, A i and R i Let A and R represent the detection precision and recall for each category, respectively. The formulas for calculating A and R are as follows: ； Wherein, TP indicates that the predicted value is consistent with the actual value, and the predicted value is a positive sample, i.e., the road target; FP and FN indicate that the predicted value is inconsistent with the actual value, FP indicates that the predicted value is a positive sample and the actual value is a negative sample, i.e., the road background; FN indicates that the predicted value is a negative sample and the actual value is a positive sample. (4) Output the detection results and evaluate their performance.
2. The road small target detection method of YOLOv5 with multi-scale feature self-correction according to claim 1, characterized in that, The specific steps (1) are as follows: obtain image data using the KITTI road autonomous driving dataset.
3. The road small target detection method of YOLOv5 with multi-scale feature self-correction according to claim 1, characterized in that, The specific steps (2) are as follows: Mosaic data augmentation is applied to the input image; the input image is uniformly scaled.
4. The road small target detection method of YOLOv5 with multi-scale feature self-correction according to claim 1, characterized in that, The formula for calculating the SEIoU loss function is as follows: ； In the formula, d represents the position loss, and the calculation formula is as follows: ； ； Wherein, the position loss factor includes the orientation and distance between the predicted bounding box and the ground truth bounding box; in the formula The distance factor representing the distance between the two frames, where , R refers to the x-coordinate of the center point of the predicted bounding box and the ground truth bounding box, respectively. w R h The meaning represented is the width and height of the circumscribed rectangle of the two frames; in the formula, α represents the orientation factor of the two frames. When the two frames are parallel or perpendicular to each other, that is, when α is close to 0 or π / 2, the focus is on the distance factor between the two frames. The orientation factor is calculated as follows: ； Among them, c w σ represents the width between the center points of the predicted bounding box and the ground truth bounding box, and c represents the distance between the center points of the two boxes. w The calculation formula is as follows: 。 5. The road small target detection method of YOLOv5 with multi-scale feature self-correction according to claim 4, characterized in that, The formula for calculating the width and height loss of SEIoU is as follows: ； Among them, w, w gt Let h and h represent the widths of the predicted bounding box and the ground truth bounding box, respectively. gt These represent the heights of the predicted bounding box and the ground truth bounding box, respectively.
6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the computer program is loaded into the processor, it implements a road small target detection method of YOLOv5 with multi-scale feature self-correction as described in any one of claims 1-5.
7. A storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements a road small target detection method of YOLOv5 with multi-scale feature self-correction as described in any one of claims 1-5.