Vehicle detection method based on improved YOLOv4 model
By improving the YOLOv4 model, deep separation convolution, increase output layer, feature map splicing and K-means clustering algorithm are adopted, combined with the Inceptionv3 structure, the problem of insufficient accuracy and real-time in vehicle detection is solved, and efficient vehicle detection performance is achieved.
Patent Information
- Application Number
- CN202111677392.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-31
- Publication Date
- 2025-03-28
- Estimated Expiration
- 2041-12-31
Smart Images
Figure CN114283390B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of vehicle detection, and in particular to a vehicle detection method based on an improved YOLOv4 model. Background Art
[0002] In the past decade, the number of civilian cars in my country has grown rapidly. By the end of 2020, the total number of civilian cars in my country reached 273.3856 million, an increase of 4.5% compared with 2019. Although the emergence of cars has brought great convenience to people's travel, with the continuous increase in the total number of civilian cars and the growing number of drivers, the road transportation safety situation has become more severe, traffic accidents have occurred frequently, and traffic congestion often occurs, which greatly affects the efficiency of road transportation.
[0003] By effectively detecting road vehicles, road safety can be improved to a certain extent, and urban traffic congestion can be effectively prevented and solved. At the same time, data support can be provided for traffic violations. However, at present, vehicle detection based on traditional methods is difficult to meet the high accuracy requirements for vehicle detection and tracking in various fields. In recent years, with the rapid development of deep learning, the application of deep learning to vehicle detection is one of the current hot topics. Therefore, it is necessary to study vehicle detection methods based on deep learning to improve the accuracy and real-time performance of vehicle detection. Summary of the invention
[0004] In view of this, the purpose of the present invention is to overcome the defects in the prior art and provide a vehicle detection method based on an improved YOLOv4 model, which can improve the vehicle detection accuracy, reduce the number of parameters and calculation amount of the model, have good robustness, and meet the real-time requirements of vehicle detection.
[0005] The vehicle detection method based on the improved YOLOv4 model of the present invention comprises the following steps:
[0006] S1. Obtain a vehicle image dataset; the vehicle image dataset includes vehicle image information of several vehicle types;
[0007] S2. Preprocess the vehicle image dataset to obtain a processed image dataset;
[0008] S3. Build an improved YOLOv4 model, and input the processed image data set into the improved YOLOv4 model for network model training to obtain a trained improved YOLOv4 model;
[0009] S4. Input the image of the vehicle to be tested into the trained improved YOLOv4 model for vehicle detection, and output the detection result of the vehicle to be tested.
[0010] Further, the step S2 specifically includes:
[0011] The vehicle image dataset is labeled to obtain a labeled vehicle image dataset;
[0012] The marked vehicle image dataset is subjected to image flipping, brightness adjustment, image noise addition and image blurring processing to obtain a processed image dataset.
[0013] Furthermore, the vehicle image dataset is labeled, including:
[0014] Set the vehicle image dataset to the Pascal VOC dataset format;
[0015] Put the vehicle image dataset in the Pascal VOC dataset format into the target folder;
[0016] Use the LabelImg annotation tool to annotate the vehicle image dataset in the target folder to obtain the XML file corresponding to the vehicle image and the vehicle image number information.
[0017] Furthermore, an improved YOLOv4 model is constructed, including:
[0018] Use depth-wise separable convolutions instead of the standard 3×3 convolutions in the YOLOv4 model;
[0019] While retaining the output layer of the YOLOv4 model, an output layer with a grid of n×n is added, and the K-means clustering algorithm is used to redefine several anchor boxes;
[0020] The Inceptionv3 structure is introduced into the Head of the YOLOv4 model, so that the Head includes a plurality of parallel lines; wherein, after the data information input into the Head is processed in parallel by the plurality of parallel lines, a plurality of processing results are obtained, and the plurality of processing results are stacked and output.
[0021] Furthermore, after adding an output layer with a grid of n×n, the feature maps of the same size in the network structure are concatenated according to the following method:
[0022] The grid in the path aggregation network PANet is The feature layer of the previous layer is upsampled to obtain a feature layer with a grid of n×n, and the feature layer with a grid of n×n is stacked with the shallow feature layer with a grid of n×n and convolved several times to obtain a new feature layer.
[0023] Furthermore, several anchor boxes are re-determined according to the following K-means clustering algorithm:
[0024] a. Randomly select K cluster centers from the cluster objects;
[0025] b. Calculate the distance from each cluster object to the cluster center, and cluster each cluster object into the nearest cluster;
[0026] c. Calculate the average coordinates of all objects in each cluster and use the average as the new coordinates of the cluster center;
[0027] d. Repeat steps bc until the moving range of the cluster center coordinates is within the set threshold or the number of clustering times reaches the set upper limit.
[0028] Further, the plurality of parallel lines include a first line, a second line, a third line and a fourth line;
[0029] The first circuit uses a 1×1 convolution kernel to perform a convolution operation on the input data information once, and obtains a result 1;
[0030] The second circuit first uses a 1×1 convolution kernel to perform a convolution operation on the input data information once to obtain a convolution result, and then uses a 5×5 convolution kernel to perform a convolution operation on the convolution result once to obtain a result 2;
[0031] The third circuit first performs a convolution operation on the input data information using a 1×1 convolution kernel to obtain a convolution result, and then performs two convolution operations on the convolution result using a 3×3 convolution kernel to obtain a result 3;
[0032] The fourth circuit first performs a 3×3 average pooling operation on the input data information to obtain a pooled result, and then performs a convolution operation on the pooled result using a 1×1 convolution kernel to obtain result 4.
[0033] The beneficial effects of the present invention are as follows: the vehicle detection method based on the improved YOLOv4 model disclosed by the present invention greatly reduces the number of model parameters and calculations by introducing depthwise separable convolution to replace the 3×3 standard convolution in the YOLOv4 model, thereby improving the model detection speed. While retaining the output layer of the YOLOv4 model, an output layer with a grid of 104×104 is added, and the feature maps of the same size in the network structure are spliced, and the anchor frame is re-determined using the K-means clustering algorithm, thereby improving the model's detection performance for target vehicles. By introducing the Inceptionv3 structure in the Head module, the input features are convolved using a variety of convolution kernels of different sizes, and the outputs are finally stacked, thereby further improving the model's detection performance for target vehicles. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] The present invention will be further described below in conjunction with the accompanying drawings and embodiments:
[0035] Figure 1It is a schematic diagram of the method flow of the present invention;
[0036] Figure 2 It is a convolution flow chart of the depth-wise separable convolution of the present invention;
[0037] Figure 3 This is a schematic diagram of the Head module structure that introduces the Inceptionv3 structure of the present invention;
[0038] Figure 4 It is a structural schematic diagram of the improved YOLOv4 model of the present invention. DETAILED DESCRIPTION
[0039] The present invention is further described below in conjunction with the accompanying drawings, as shown in the drawings:
[0040] The vehicle detection method based on the improved YOLOv4 model of the present invention comprises the following steps:
[0041] S1. Obtain a vehicle image dataset; the vehicle image dataset includes vehicle image information of several vehicle types; wherein, vehicles on the road are used as sample vehicles to obtain the vehicle image dataset; due to the different types of vehicles on the road, vehicle images in urban areas, towns, highways and other places are collected by taking photos, downloading from the Internet and other means, and the vehicles include common vehicles on the road such as cars, buses, trucks, motorcycles, etc.; at the same time, it also covers vehicle images in scenes including sunny days, cloudy days, nights and situations where vehicles are blocked from each other, the size of the vehicle image is 416×416, and the vehicle image dataset is divided into a training set and a test set at a ratio of 9:1;
[0042] S2. Preprocess the vehicle image dataset to obtain a processed image dataset;
[0043] S3. Build an improved YOLOv4 model, and input the processed image data set into the improved YOLOv4 model for network model training to obtain a trained improved YOLOv4 model; wherein, according to the computer memory and video memory size and the user's final detection effect requirements, set the batch size of the network model training, the number of vehicle recognition types, the number of training iterations, and whether to use CUDA acceleration, etc.;
[0044] S4. Input the image of the vehicle to be tested into the trained improved YOLOv4 model for vehicle detection, and output the detection result of the vehicle to be tested.
[0045] In this embodiment, the step S2 specifically includes:
[0046] The dataset obtained in step S1 is placed in the format of the Pascal VOC dataset and annotated using labelImg. Before annotating the dataset using the LabelImg annotation tool, the vehicle images in the dataset need to be placed in the folder JPEGImages, and then the LabelImg annotation tool is used to annotate the vehicle images in the dataset. At this time, the corresponding XML files will be generated in the folder Annotations in the order of the vehicle images, and the Main folder under the folder ImageSets is used to store the numbers of the vehicle photos in the dataset. The vehicles in the dataset are annotated into four types: small passenger cars are annotated as car; buses are annotated as bus; motorcycles are annotated as motorbikes; and trucks are annotated as trucks.
[0047] The marked vehicle image dataset is subjected to image flipping, brightness adjustment, image noise addition and image blurring to obtain a processed image dataset, which is a data enhanced image dataset. The image flipping, brightness adjustment, image noise addition and image blurring all adopt existing technologies and will not be described in detail here.
[0048] In this embodiment, an improved YOLOv4 model is constructed, specifically including:
[0049] Introducing deep separable convolution to replace the 3×3 standard convolution in the YOLOv4 model; by using deep separable convolution, the number of model parameters and the amount of calculation can be greatly reduced, and the model detection speed is improved. Figure 4 The depth-separable convolution is represented by convolution_DW, and the convolution process of depth-separable convolution is as follows Figure 2 As shown;
[0050] While retaining the output layer of the YOLOv4 model, an output layer with a grid of 104×104 is added. After adding an output layer with a grid of 104×104, the feature maps of the same size in the network structure are spliced according to the following method:
[0051] like Figure 4 As shown in the figure, the feature layer with a grid of 52×52 in the path aggregation network PANet is further upsampled to obtain a feature layer with a grid of 104×104, and the feature layer with a grid of 104×104 is stacked with the feature layer with a shallow grid of 104×104 and convolved 5 times to obtain a new feature layer; among them, the new feature layer is input into the Head module for detection on the one hand, and corresponds to the output layer with a grid of 104×104; on the other hand, it continues to be downsampled and stacked with the feature layer of 52×52 and convolved 5 times, thereby providing more feature information for the detection network and improving the model's detection performance for the target vehicle.
[0052] The K-means clustering algorithm is used to redefine 12 anchor boxes, which improves the model's detection performance for target vehicles. The 12 anchor boxes are redetermined according to the following K-means clustering algorithm:
[0053] a. Randomly select K cluster centers from the cluster objects;
[0054] b. Calculate the distance from each cluster object to the cluster center, and cluster each cluster object into the nearest cluster;
[0055] c. Calculate the average coordinates of all objects in each cluster and use the average as the new coordinates of the cluster center;
[0056] d. Repeat steps bc until the moving range of the cluster center coordinates is within the set threshold or the number of clustering times reaches the set upper limit. The threshold and the upper limit are set according to actual working conditions.
[0057] like Figure 3 As shown, the Inceptionv3 structure is introduced into the Head of the YOLOv4 model, so that the Head includes several parallel lines; wherein, after the data information input into the Head is processed in parallel by the several parallel lines, several processing results are obtained, and the several processing results are stacked and output, which further improves the detection performance of the model for the target vehicle. The several processing results include result 1, result 2, result 3 and result 4;
[0058] The plurality of parallel lines include a first line, a second line, a third line and a fourth line;
[0059] The first circuit uses a 1×1 convolution kernel to perform a convolution operation on the input data information once, and obtains a result 1;
[0060] The second circuit first uses a 1×1 convolution kernel to perform a convolution operation on the input data information once to obtain a convolution result, and then uses a 5×5 convolution kernel to perform a convolution operation on the convolution result once to obtain a result 2;
[0061] The third circuit first performs a convolution operation on the input data information using a 1×1 convolution kernel to obtain a convolution result, and then performs two convolution operations on the convolution result using a 3×3 convolution kernel to obtain a result 3;
[0062] The fourth circuit first performs a 3×3 average pooling operation on the input data information to obtain a pooled result, and then performs a convolution operation on the pooled result using a 1×1 convolution kernel to obtain result 4.
[0063] In this embodiment, in step S3, when the network model is trained, the data set is divided into a training set and a test set at a ratio of 9:1, and the sample images in the training set are input into the improved YOLOv4 model, and the optimal weight file for vehicle detection is obtained through continuous iterative training. Its essence is to use the loss function to continuously adjust the weights in the network, and then use the verification set to calculate the average precision mean mAP and the detection speed FPS, so as to verify whether the training results meet the standards, until a weight file that meets the standards is obtained, and the weight file is loaded into the improved YOLOv4 model, and finally a trained improved YOLOV4 model is obtained.
[0064] In this embodiment, in step S4, a vehicle image to be tested is prepared on a computer, and a detection command is input in a command line in a python environment, and the improved YOLOv4 model detects the input vehicle image to be tested, and finally obtains the detection result of the vehicle image to be tested. The detection result includes the type of vehicle and the confidence information corresponding to the type.
[0065] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.
Claims
1. A vehicle detection method based on an improved YOLOv4 model, characterized in that: The steps include: S1. Obtain a vehicle image dataset; the vehicle image dataset includes vehicle image information of several vehicle types; S2. Preprocess the vehicle image dataset to obtain a processed image dataset; S3. Build an improved YOLOv4 model, and input the processed image data set into the improved YOLOv4 model for network model training to obtain a trained improved YOLOv4 model; Build an improved YOLOv4 model, including: Use depth-wise separable convolutions instead of the standard 3×3 convolutions in the YOLOv4 model; While retaining the output layer of the YOLOv4 model, an output layer with a grid of n×n is added, and the K-means clustering algorithm is used to redefine several anchor boxes; In the Head of the YOLOv4 model, the Inceptionv3 structure is introduced, so that the Head includes a plurality of parallel lines; wherein, after the data information input into the Head is processed in parallel by the plurality of parallel lines, a plurality of processing results are obtained, and the plurality of processing results are stacked and output; Among them, after adding an output layer with a grid of n×n, the feature maps of the same size in the network structure are spliced according to the following method: The grid in the path aggregation network PANet is The feature layer of the upper layer is upsampled to obtain a feature layer with a grid of n×n, and the feature layer with a grid of n×n is stacked with the feature layer of the shallow layer with a grid of n×n and convolved several times to obtain a new feature layer; The plurality of parallel lines include a first line, a second line, a third line and a fourth line; The first circuit uses a 1×1 convolution kernel to perform a convolution operation on the input data information once, and obtains a result 1; The second circuit first uses a 1×1 convolution kernel to perform a convolution operation on the input data information once to obtain a convolution result, and then uses a 5×5 convolution kernel to perform a convolution operation on the convolution result once to obtain a result 2; The third circuit first performs a convolution operation on the input data information using a 1×1 convolution kernel to obtain a convolution result, and then performs two convolution operations on the convolution result using a 3×3 convolution kernel to obtain a result 3; The fourth circuit first performs a 3×3 average pooling operation on the input data information to obtain a pooled result, and then performs a convolution operation on the pooled result using a 1×1 convolution kernel to obtain a result 4; S4. Input the image of the vehicle to be tested into the trained improved YOLOv4 model for vehicle detection, and output the detection result of the vehicle to be tested.
2. The vehicle detection method based on the improved YOLOv4 model according to claim 1, characterized in that: The step S2 specifically includes: The vehicle image dataset is labeled to obtain a labeled vehicle image dataset; The marked vehicle image dataset is subjected to image flipping, brightness adjustment, image noise addition and image blurring processing to obtain a processed image dataset.
3. The vehicle detection method based on the improved YOLOv4 model according to claim 2, characterized in that: The vehicle image dataset is labeled, including: Set the vehicle image dataset to the Pascal VOC dataset format; Put the vehicle image dataset in the Pascal VOC dataset format into the target folder; Use the LabelImg annotation tool to annotate the vehicle image dataset in the target folder to obtain the XML file corresponding to the vehicle image and the vehicle image number information.
4. The vehicle detection method based on the improved YOLOv4 model according to claim 1, characterized in that: Re-determine several anchor boxes according to the following K-means clustering algorithm: a. Randomly select K cluster centers from the cluster objects; b. Calculate the distance from each cluster object to the cluster center, and cluster each cluster object into the nearest cluster; c. Calculate the average coordinates of all objects in each cluster and use the average as the new coordinates of the cluster center; d. Repeat steps bc until the moving range of the cluster center coordinates is within the set threshold or the number of clustering times reaches the set upper limit.