A city street scene target detection method based on YOLOv5
By improving the backbone extraction network and feature fusion module of the YOLOv5 model, and combining the SOCA module and SIOU loss function, the problem of balancing detection accuracy and efficiency in urban street scenes is solved, achieving efficient and accurate target detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-04-11
- Publication Date
- 2026-03-17
AI Technical Summary
Existing target detection technologies for urban street scenes struggle to simultaneously guarantee high detection accuracy and high detection efficiency, especially when dealing with overlapping and interactive situations between motor vehicles and non-motor vehicles, or between motor vehicles and pedestrians, where recognition becomes quite difficult.
The downsampling module of the backbone extraction network and feature fusion module of the YOLOv5 model is improved by adopting deformable convolutional units and focus units, and an SOCA module is added after the prediction feature layer. The SIOU loss function is used, and the SIOU function is used during model training. Combined with the feature fusion module, multi-scale feature maps are output for object detection.
It achieves higher detection accuracy while maintaining high detection efficiency, and can accurately identify multiple targets in complex street scenes, including vehicle subcategories and overlapping information.
Smart Images
Figure CN116385860B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of target detection technology, and specifically to a target detection method for urban street scenes based on YOLOv5. Background Technology
[0002] Urban street scene object detection is a branch of computer vision object detection, widely used in fields such as autonomous driving and traffic safety inspection. Like other object detection problems, the goal of urban street scene object detection is to automate detection through computer vision technology, thereby reducing labor costs and improving detection efficiency and accuracy. Specifically, compared to other object detection problems, urban street scenes are unique in that there is overlap and interaction between motor vehicles, between motor vehicles and non-motor vehicles, and between motor vehicles and pedestrians, making detection more difficult and challenging to accurately and comprehensively identify targets within the scene.
[0003] With the development of deep learning technology, existing object detection techniques for urban street scenes typically employ one-stage object detection techniques based on the YOLO framework and two-stage object detection techniques based on the Faster R-CNN framework. For example, a lightweight object detection method based on YOLOv5 can reduce the number of model parameters and achieve high detection efficiency by setting depthwise separable convolutional modules, but its detection accuracy is correspondingly reduced due to this limitation. Another detection method based on the Faster R-CNN model incorporates an RPN (Region Generation Network) structure, which can help achieve higher detection accuracy, but the large number of model parameters results in very low detection efficiency. Overall, the latter method has significantly better detection accuracy than the former, while the former method has significantly better detection efficiency. Therefore, both existing methods have their advantages and disadvantages, and neither can simultaneously guarantee both detection accuracy and detection efficiency. Summary of the Invention
[0004] The present invention aims to provide a target detection method for urban street scenes based on YOLOv5, which can achieve high detection accuracy while ensuring high detection efficiency.
[0005] The basic solution provided by this invention is: a target detection method for urban street scenes based on YOLOv5, comprising the following steps:
[0006] Step 1: Construct an object detection model based on the improved YOLO-v5; the object detection model includes a backbone feature extraction network, a feature fusion module, and an output module; the downsampling module in the backbone feature extraction network consists of deformable convolutional units and focus units; the downsampling module of the feature fusion module consists of convolutional units and focus units with a preset stride.
[0007] Step 2: Train the object detection model;
[0008] Step 3: Scene target detection is performed by the trained target detection model; wherein, the backbone feature extraction network extracts features of the scene image to be detected; the feature fusion module outputs feature maps at different scales; and the output module outputs target detection feature maps based on the feature maps.
[0009] Furthermore, the convolutional unit with the preset stride is a convolutional unit with a stride of 1.
[0010] Furthermore, both the backbone extraction network and the feature fusion module include C3 units, deformable convolutional units, Focus units, and SPPF units.
[0011] Furthermore, the output module has an SOCA module connected to its predicted feature layer.
[0012] Furthermore, the SIOU function is used as the loss function when training the object detection model.
[0013] Furthermore, when training the target detection model, the backpropagation algorithm is used to update the detection model parameters based on the calculated loss, and the process is iterated until the detection model converges.
[0014] Furthermore, when the feature fusion module outputs feature maps, it outputs four feature maps of different scales, which are applicable to the detection of small targets, medium targets, medium-large targets and large targets.
[0015] Furthermore, the output module outputs four sets of target bounding box regression parameters and target categories at different scales based on the feature map. The target bounding boxes and target categories are then drawn on the image to be detected according to the confidence threshold, thereby forming a target detection feature map.
[0016] The working principle and advantages of this invention are as follows:
[0017] This solution starts with the original YOLO-v5 model and improves upon it. First, it uses deformable convolutional units combined with a focus module as the downsampling module in the backbone extraction network of the object detection model, replacing the downsampling module in the original YOLO-v5 backbone extraction network. Then, it uses convolutional units with a preset stride and focus units as the downsampling module in the backbone extraction network, replacing the downsampling module in the original YOLO-v5 feature fusion module. Next, it uses the SIOU loss function instead of the CIOU loss function used in the original YOLO-v5. Finally, it adds a SOCA module after the prediction feature layer, thus forming a new object detection model based on the improved YOLO-v5, achieving higher object detection efficiency and accuracy.
[0018] The deformable convolutional units adaptively extract features, while the Focus module preserves global information, thus improving detection accuracy. Combined with YOLO's inherent one-stage detection capabilities, this ensures high detection efficiency. The SIOU loss function, compared to the CIOU loss function, offers better training performance, faster training speed, and higher inference accuracy. The SOCA module utilizes higher-order feature statistics than the first-order module to adaptively readjust features, allowing the network to focus on more informational features and enhancing the model's overall discriminative learning ability, thereby achieving higher object detection accuracy.
[0019] Furthermore, compared to conventional object detection schemes based on standard YOLO-v5, this scheme achieves higher detection accuracy. Compared to existing object detection schemes based on Faster R-CNN, this scheme is not limited by the RPN structure and achieves higher detection efficiency; that is, this scheme can simultaneously maintain high detection accuracy and detection efficiency. By replacing and improving the downsampling module in the backbone extraction network and feature fusion module, this scheme can significantly improve the model's feature extraction and fusion capabilities, enabling the model to identify more refined target information. This allows target identification to move beyond broad categories such as vehicles to more specific subcategories such as bicycles, motorcycles, cars, and vans. It also enables accurate identification of overlapping information that is difficult to recognize in conventional methods (including overlaps and interactions between motor vehicles, between motor vehicles and non-motor vehicles, and between motor vehicles and pedestrians). Combined with the SOCA module located at specific positions in the model, it endows the model with a higher-order feature statistical capability, allowing the model to maintain the high recognition efficiency of YOLOv5's first-order characteristics while possessing a higher-order feature statistical capability, thus enabling the identification of more comprehensive and richer target information. Attached Figure Description
[0020] Figure 1 This is a schematic diagram of the method flow of an embodiment of the target detection method for urban street scenes based on YOLOv5 of the present invention;
[0021] Figure 2 This is a schematic diagram of the target detection model structure of an embodiment of the target detection method for urban street scenes based on YOLOv5 according to the present invention;
[0022] Figure 3 This is a first comparison image of the model detection performance of an embodiment of the target detection method for urban street scenes based on YOLOv5 of the present invention;
[0023] Figure 4 This is a second comparison image showing the model detection performance of an embodiment of the target detection method for urban street scenes based on YOLOv5 according to the present invention;
[0024] Figure 5 This is a third comparison image showing the model detection performance of an embodiment of the target detection method for urban street scenes based on YOLOv5 according to the present invention;
[0025] Figure 6 This is the fourth comparison image showing the model detection performance of an embodiment of the target detection method for urban street scenes based on YOLOv5 according to the present invention. Detailed Implementation
[0026] The following detailed explanation illustrates the specific implementation methods:
[0027] The basic implementation examples are as follows: Figure 1 As shown: A target detection method for urban street scenes based on YOLOv5 includes the following steps:
[0028] Step 1: Construct an object detection model based on the improved YOLO-v5.
[0029] As attached Figure 2 As shown, the target detection model includes a backbone feature extraction network, a feature fusion module, and an output module. The downsampling module in the backbone feature extraction network consists of deformable convolutional units and focus units; the downsampling module of the feature fusion module consists of convolutional units with a preset stride and focus units. Specifically, in this embodiment, the convolutional unit with the preset stride is a convolutional unit with a stride of 1.
[0030] The backbone extraction network and feature fusion module both include C3 units, deformable convolutional units, focus units, and SPPF units. Specifically, the C3 unit consists of three convolutional modules (ConvBnSiLU) and one bottleneck layer (BottleNeck); its convolutional modules consist of convolutional layers, one batch normalization (BN) layer, and a sigmoid weighted linear combination live function (SiLU). The SPPF unit is an improved spatial pyramid pooling unit. Compared to conventional pooling units, this unit can achieve adaptive pooling size with lower computational cost, helping to ensure that the model has high detection efficiency and accuracy. The specific workflow of the SPPF unit is as follows: First, the feature map is input into a network structure consisting of one convolutional module (ConvBNSiLU) and three max pooling layers (MaxPool2d). Then, the inputs of each module (one convolutional module and three max pooling layers) are concatenated in the depth direction to obtain a new output feature map. Finally, the new output feature map is passed through a convolutional module to obtain the final output feature map.
[0031] Among them, the deformable convolutional unit setting, compared with the standard convolutional unit in the conventional setting, adds an offset to its sampling position. Its convolutional kernel can be expanded to a large range during training, resulting in better convolution effect. It can be closer to the shape and size of the object during sampling, making it more robust and helping to achieve higher target detection accuracy.
[0032] The output module's predicted feature layer is connected to an SOCA module. Specifically, SOCA refers to a second-order channel attention mechanism. In this embodiment, the main operations of the SOCA module include the following steps:
[0033] S1: Calculate the normalized covariance matrix of the input feature map:
[0034] S1.1: Reshape the input H×W×C feature map into an s×C feature matrix X, where s=H×W;
[0035] S1.2: Calculate the sample covariance matrix Σ=X(I / s-1 / s) of X. 2 )X T Where I is the identity matrix and 1 is an all-one square matrix;
[0036] S1.3: Perform orthogonal decomposition on Σ to obtain Σ=UΛU T , where U is an orthogonal matrix composed of eigenvectors and Λ is a diagonal matrix composed of eigenvalues;
[0037] S1.4: Let Obtain the normalized covariance matrix of X. The dimensions are C×C.
[0038] S2: Calculate channel attention:
[0039] S2.1 Order
[0040] The c-th component of z
[0041] S2.2: Let w = f(W) U δ(W D z)), where W D The kernel is a one-dimensional convolution kernel of size C / r, δ is the ReLU function, and W... U is a one-dimensional convolution kernel of size C, and f is the sigmoid function.
[0042] S2.3: Finally, perform a dot product between the channel attention matrix w and the input feature map along the channel direction, i.e. Obtain the output feature map
[0043] Step 2: Train the object detection model;
[0044] When training the object detection model, the SIOU function is used as the loss function. Specifically, in this embodiment, the SIOU function is calculated as follows:
[0045]
[0046] Where c h σ represents the height difference between the ground truth bounding box and the predicted center point, and σ represents the distance between the ground truth bounding box and the predicted center point.
[0047]
[0048] in: γ=2-Λ
[0049] here (c w ,c h ) are the center coordinates of the ground truth bounding box, the center coordinates of the predicted bounding box, and the width and height of the minimum bounding rectangle of the ground truth bounding box and the predicted bounding box, respectively.
[0050]
[0051] in:
[0052] Here, (w,h) and (w) gt ,h gt ) represent the width and height of the predicted bounding box and the ground truth bounding box, respectively.
[0053] Loss function:
[0054] Furthermore, during the training of the target detection model, the backpropagation algorithm is used to update the detection model parameters based on the calculated loss, and equation (1) is returned, iterating until the detection model converges. In this embodiment, the calculated loss specifically refers to the loss L calculated as shown in equation (4) above.
[0055] Step 3: Scene target detection is performed by the trained target detection model; wherein, the backbone feature extraction network extracts features of the scene image to be detected; the feature fusion module outputs feature maps at different scales; and the output module outputs target detection feature maps based on the feature maps.
[0056] When the feature fusion module outputs feature maps, it outputs four feature maps at different scales, applicable to the detection of small, medium, medium-large, and large targets. The output module outputs four sets of target bounding box regression parameters and target categories based on the feature maps at different scales. According to the confidence threshold, the target bounding boxes and target categories are drawn on the image to be detected, thus forming the target detection feature map.
[0057] In practical applications, as shown in the appendix Figure 3 , Figure 4 , Figure 5 and Figure 6 As shown, the left images represent the prediction outputs of the original YOLO-v5 model, while the right images represent the prediction outputs of the improved object detection model provided by this solution. It is evident from the images that the object detection model provided by this solution can effectively detect objects that the original YOLO-v5 model failed to detect, and it achieves higher recognition accuracy for each object. The model performance is significantly improved, resulting in higher detection accuracy. For example, in the attached... Figure 4 In the original model, all different types of vehicles were identified as "cars," while this model can further distinguish between "cars," "riders," and "motorcycles" more accurately; that is, it can further differentiate between cars, bicycles, and motorcycles from vehicle categories, with significantly better recognition accuracy than the original model. For example, in the attached... Figure 6 In the original model, the truck in the distance was not identified because the original model's recognition accuracy was not high enough to accurately identify road scenes with overlapping information, while this model successfully identified it (corresponding to truck 0.9).
[0058] This embodiment provides a target detection method for urban street scenes based on YOLOv5. By improving the YOLO-v5 model, the downsampling module of the backbone extraction network and feature fusion module is modified. In particular, an SOCA module is added after the prediction feature layer, and the SIOU loss function is set for model training. This results in a new target detection model based on the improved YOLO-v5 that can be used for target detection in urban street scenes. The model has a small number of parameters and can achieve higher target detection efficiency and accuracy. It has high target accuracy for road scenes with overlapping information.
[0059] Furthermore, after being trained on a massive number of samples, the model can effectively detect scene targets of different scales, including cars, trucks, motorcycles, bicycles, city trains, pedestrians, and motorcycle / bicycle riders. It has a high degree of comprehensiveness in target detection for road scenes with overlapping information, and the detection is comprehensive and accurate. It can be deployed on intelligent vehicles and traffic detection equipment, which will help promote the development of autonomous driving, traffic safety detection, and other related fields.
[0060] The above descriptions are merely embodiments of the present invention. Commonly known structures and characteristics of the solutions are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are aware of all existing technologies in that field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can, under the guidance of this application, improve and implement this solution in combination with their own capabilities. Some typical known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention. These should also be considered within the scope of protection of the present invention, and will not affect the effectiveness of the implementation of the present invention or the practicality of the patent.
Claims
1. A YOLOv5-based urban street scene target detection method, characterized in that, The method comprises the following steps: Step 1: constructing a target detection model based on improved YOLO-v5; the target detection model comprises a backbone feature extraction network, a feature fusion module and an output module; and the down-sampling module in the backbone extraction network is composed of a deformable convolution unit and a Focus unit connected in sequence; the down-sampling module of the feature fusion module is composed of a convolution unit with a preset step distance and a Focus unit connected in sequence; and the SOCA module is connected to the minimum scale prediction feature layer of the output module; Step 2: training the target detection model; Step 3: performing scene target detection by the trained target detection model; wherein the features of the scene image to be detected are extracted by the backbone feature extraction network; the feature maps of different scales are output by the feature fusion module; and the target detection feature map is output by the output module based on the feature map.
2. The YOLOv5-based urban street scene target detection method according to claim 1, wherein, The convolution unit with a preset step distance is a convolution unit with a step distance of 1.
3. The YOLOv5-based urban street scene target detection method according to claim 1, characterized in that, The backbone extraction network comprises a C3 unit, a deformable convolution unit, a Focus unit and an SPPF unit.
4. The YOLOv5-based urban street scene target detection method according to claim 1, characterized in that, In the training of the target detection model, the loss function adopts the SIOU function.
5. The YOLOv5-based urban street scene target detection method according to claim 4, characterized in that, In the training of the target detection model, based on the calculated loss, the detection model parameters are updated using the back propagation algorithm, and the iteration is performed until the detection model converges.
6. The YOLOv5-based urban street scene target detection method according to claim 1, characterized in that, When the feature fusion module outputs the feature map, four feature maps of different scales are output respectively, and are applied to the detection of small targets, medium targets, medium-large targets and large targets.
7. The YOLOv5-based urban street scene target detection method according to claim 6, characterized in that, The output module outputs four groups of target bounding box regression parameters and target categories of different scales based on the feature map, draws the target bounding box and the target category on the image to be detected according to the confidence threshold, and then forms the target detection feature map.