A road vehicle detection system and method based on YOLOv4

Abstract

The present application relates to a road vehicle detection system and method based on YOLOv4, wherein the road vehicle detection system comprises a road vehicle data acquisition module, a road vehicle detection model and a terminal device, wherein the road vehicle detection model is an improved YOLOv4 algorithm model, wherein the backbone network of the improved YOLOv4 algorithm model uses the MobileVit network as the backbone feature extraction network, and the neck Neck comprises a spatial pyramid pooling layer SPP and a prediction layer PANet; and the positioning loss function of the model is a CDIOU loss function. The present invention adopts the GridMask image enhancement method for image preprocessing, and performs road vehicle detection through a trained road vehicle detection model, thereby improving the existing YOLOv4 model's low accuracy, slow detection speed, and insufficient lightness, and ensuring the detection accuracy of the model while improving the detection efficiency.

Description

Technical Field

[0001] The present application relates to the field of vehicle detection technology, and in particular to a road vehicle detection system and method based on YOLOv4. Background Art

[0002] In recent years, deep learning-based object detection technology has been widely used in intelligent video surveillance, unmanned driving, robot navigation and other fields. With the rapid growth of China's car ownership, the number of traffic violations is increasing, which has brought great pressure to traffic management personnel. Rapid and accurate detection of vehicles on the road through surveillance video is an indispensable part of building an intelligent transportation system.

[0003] The mainstream vehicle detection algorithms based on deep learning can be roughly divided into one-stage methods and two-stage methods. The one-stage method directly predicts and regresses the category of the prior frame in the image. This method has a fast detection speed, but the detection accuracy is relatively low. Representative algorithms include the YOLO series. The two-stage method extracts candidate frames from the image prior frame through a candidate frame generator, and then classifies and regresses the obtained candidate frames to predict the results. The two-stage algorithm has high detection accuracy, but the real-time performance is weak due to the complex calculation of the candidate frame extraction process. Representative algorithms include the R-CNN series. The existing method uses the intelligent vehicle road ahead vehicle detection method based on Faster RCNN to achieve high detection accuracy, but the detection speed is low and it is difficult to meet the real-time requirements. The existing method uses the improved SSD algorithm to realize the automatic recognition and positioning of aerial photography urban road vehicles. Although it can basically meet the real-time requirements, the detection accuracy of the model is low. The YOLO series of algorithms are deeply loved in the engineering field because of their strong real-time performance, but the detection accuracy and model size limit the application scope of the YOLO series of algorithms. Summary of the invention

[0004] The purpose of the present invention is to provide a road vehicle detection system and method based on YOLOv4, adopting MobileVit network to replace CSPDarknet53 network as the backbone network of the improved YOLOv4 algorithm model, and introducing CDIOU loss function as the positioning loss function of the improved YOLOv4 algorithm model, so as to improve the real-time performance and detection accuracy of the improved YOLOv4 algorithm model.

[0005] The technical solution adopted by the present invention is: a road vehicle detection system based on YOLOv4, including a road vehicle data acquisition module, a road vehicle detection model and a terminal device; the road vehicle data acquisition module is a color camera, which is used to collect road vehicle image data in real time, and input the collected image data into the road vehicle detection model; the road vehicle detection model recognizes and outputs vehicle information in the image, the road vehicle detection model is an improved YOLOv4 algorithm model, the backbone network of the improved YOLOv4 algorithm model uses a MobileVit network as a basic network, the number of layers of the MobileVit network is ten, and the MobileVit network includes MobileNetv2 modules and MobileVit modules stacked on each other; the outputs of the seventh layer, the ninth layer and the tenth layer of the MobileVit network are connected to the prediction layer PANet of the improved YOLOv4 algorithm model; the loss function Loss of the improved YOLOv4 algorithm model includes a positioning loss function loss cdiou , confidence loss function loss confidence And category loss function loss class , where the positioning loss function loss cdiou is the CDIOU loss function; the terminal device includes a processor and a memory, the memory stores the program required for the road vehicle detection model to perform road vehicle detection, and the processor is used to run the program stored in the memory.

[0006] Furthermore, the loss function Loss and the positioning loss function loss cdiou , confidence loss function loss confidence And category loss function loss class The specific expression is as follows:

[0007] Loss = loss cdiou +loss confidence +loss class

[0008]

[0009]

[0010]

[0011] Among them, S 2 is the number of grids divided in the image; B is the number of prior boxes contained in each grid; Is there an object in the j-th prior box of the i-th grid? If there is an object, it takes 1; if there is no object, it takes 0; Is the j-th prior box of the i-th grid without an object, 1 if there is no object, and 0 if there is a target; λ noobj is the confidence error weight of the prior box without the target; C i j is the confidence of the j-th prediction box of the i-th grid; is the confidence of the jth ground truth box of the i-th grid; P i j is the confidence of the j-th prediction box of the i-th grid; is the confidence of the jth true box of the ith grid; classes represents the set of all target categories, c represents a target category in classes; AE, BF, CG, DH represent the distances between the four corresponding vertices of the predicted box ABCD and the true box EFGH, respectively, and AG is the diagonal length of the minimum rectangle enclosing the predicted box ABCD and the true box EFGH.

[0012] Furthermore, the activation functions of the MobileVit network are SILU activation function and RELU activation function.

[0013] Another technical solution adopted by the present invention is: a road vehicle detection method based on YOLOv4, characterized in that it includes the following steps:

[0014] Step S1: constructing the improved YOLOv4 algorithm model as described in the previous technical solution;

[0015] Step S2: Divide the road surface images into vehicle information training images, vehicle information verification images and vehicle information test images, construct a vehicle information dataset in VOC format, and obtain a training set, a verification set and a test set;

[0016] Step S3: Use the training set obtained in step S2 to train the improved YOLOv4 algorithm model. The training process includes two stages. In the first training stage, the backbone network of the improved YOLOv4 algorithm model is frozen, and no parameter update is performed. The remaining network layers are trained and the parameters are updated. In the second training stage, all network layers are trained and the parameters are updated. The training methods of the first training stage and the second training stage are the same. The specific training steps are as follows:

[0017] Step S301: Preprocess the images in the training set using the GridMask image enhancement method to delete the information in the image in a discrete and orderly manner, reducing excessive deletion of regions and excessive retention;

[0018] Step S302: cluster the real frames in the training set through the K-means algorithm to obtain the size of the prior frame; then input the preprocessed training set pictures into the improved YOLOv4 algorithm model constructed in step S1, and fully extract the local features and global features of the pictures through the MobileVit network, increase the network receptive field through the spatial pyramid pooling layer SPP to obtain richer features, and integrate the shallow features and deep features through the prediction layer PANet, and optimize the loss function. Finally, the head Head outputs the judgment and adjustment data of the prior frame in the picture, and adjusts the prior frame to obtain the predicted frame;

[0019] Step S303: After setting the confidence threshold and classification threshold for the prediction box using the DIOU NMS algorithm, sort them from high to low according to the scores, select the detection box with the highest score, and then remove the detection box with a low score to complete the training of the improved YOLOv4 algorithm model;

[0020] Step S4: Use the trained improved YOLOv4 algorithm model to detect the road vehicle image data collected in real time and identify the vehicle targets in the image.

[0021] Furthermore, in step S3, the number of iterations in the first training stage is set to 37400, and the learning rate is modified to 0.001; the initial learning rate in the second training stage is set to 0.0001, and the number of iterations is uncertain, until the improved YOLOv4 algorithm model converges, that is, when the Loss variable in the output log of the improved YOLOv4 algorithm model no longer decreases, the second training stage is stopped.

[0022] Furthermore, in step S2, the vehicle information includes category information and position information. The category information includes three categories: bus, car and truck. The position information includes the center point coordinates of the real frame, and the width and height of the real frame.

[0023] Furthermore, the specific method of step S2 is:

[0024] Step S201: Create a file voc_classes.txt, add vehicle dataset labels, use the labellmg image annotation tool to complete the image annotation, and generate an xml annotation file in PASCAL VOC format;

[0025] Step S202: Use the directory structure of the PASCAL VOC dataset to create a vehicle information folder Vehicle. The Vehicle folder contains three subfolders, namely, the Annotations folder, the ImageSets folder, and the JPEGImages folder. Among them, the Annotations folder contains all the xml tag files; the ImageSets file stores the txt files of the dataset division; the JPEGImages folder contains all the vehicle information training images, vehicle information verification images, and vehicle information test images;

[0026] Step S203: Name the images in the JPEGImages folder one by one according to the format requirements of PASCAL VOC, annotate the category information and location information of the annotation file corresponding to each image, save it as an XML file with the same file name as the corresponding image, and then put it into the Annotations folder;

[0027] Step S204: Generate training set, validation set and test set files, create a labels subfolder under the vehicle information folder Vehicle, and the labels subfolder contains a yolo format annotation file of each image, and the yolo format annotation file is converted from the xml annotation file;

[0028] Step S205: In the labels subfolder, create 2007_train.txt file, 2007_val.text file and 2007_test.txt file. The three files correspond to the lists of training set, validation set and test set files respectively, and the lists contain the file paths and file names of all images; create a folder Main in the ImageSets folder, and create train.txt file, val.text file and test.txt file in the folder Main. The three files contain the file name of each image, but do not contain file path and extension information.

[0029] Furthermore, in step S4, the MobileNetv2 module performs a 1×1 convolution operation on the input feature map to expand the number of channels, performs a 3×3 depth-separable convolution operation to reduce the amount of calculation, and then performs a 1×1 convolution operation to reduce the number of channels and add the result to the skip chain at the input end of the MobileNetv2 module to obtain the feature map output by the MobileNetv2 module.

[0030] Furthermore, in step S4, the MobileVit module encodes the local spatial information and the number of channels of the expanded features through an n×n and a 1×1 convolution operation on the input feature map, extracts the global features through L Transformer operations, compresses the number of channels through a 1×1 convolution, and finally fuses the features input by the MobileVit module, and then performs an n×n convolution to reduce the number of channels to obtain the final output features.

[0031] The beneficial technical effects of the present invention are:

[0032] (1) The MobileVit network is used to replace the CSPDarknet53 network as the backbone network of the improved YOLOv4 algorithm model for feature extraction, so that the improved YOLOv4 algorithm model is more lightweight and the detection speed of the model is improved; the convolutional neural network has the ability to learn local feature extraction with only a small number of parameters, and the VisionTransformer can learn the ability of global feature extraction, but requires a large number of parameters for learning. The MobileVit network combines the advantages of the convolutional neural network and the Vision Transformer to improve the existing YOLOv4 algorithm model, and obtains the improved YOLOv4 algorithm model used in the present invention, which takes into account the real-time and accuracy of model detection;

[0033] (2) The present invention introduces the GridMask image enhancement method into the data preprocessing stage, and performs discrete and orderly deletion of information in the image, thereby reducing the situation of excessive deletion and excessive retention of regions, and further simulating partial occlusion of vehicle targets; the improved YOLOv4 algorithm model is used to train image data containing partially occluded targets, so as to improve the detection ability of the improved YOLOv4 algorithm model for occluded targets; the training process of the model is divided into two stages, so as to realize the transfer learning of the model and shorten the training time;

[0034] (3) The CDIOU loss function is used to replace the CIoU loss function. The CDIOU loss function includes the positioning loss function, the confidence loss function and the category loss function. The CDIOU loss function can speed up the process of minimizing the distance between the center point of the predicted box and the true box without increasing the amount of calculation, and improve the detection accuracy of the improved YOLOv4 algorithm model. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0036] Figure 1 A schematic diagram of the network structure of the improved YOLOv4 algorithm model according to an embodiment of the present invention;

[0037] Figure 2 Schematic diagram of the positional relationship between the prior box and the true box in the CDIoU loss function of an embodiment of the present invention;

[0038] Figure 3 is a flow chart of a method according to an embodiment of the present invention;

[0039] Figure 4 It is a schematic diagram of the GridMask image enhancement method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention can also be implemented in other ways different from those described herein, and therefore, the present invention is not limited to the limitations of the specific embodiments disclosed below.

[0041] Unless otherwise defined, the technical or scientific terms used herein shall have the usual meanings understood by persons of ordinary skill in the field described in this application. The words "first", "second" and similar terms used in this patent application specification and claims do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, words such as "one" or "one" do not indicate a quantity limitation, but rather indicate the presence of at least one. Words such as "connect" or "connected" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the object being described changes, the relative positional relationship also changes accordingly.

[0042] The embodiment of the present invention adopts a road vehicle detection system based on YOLOv4, including a road vehicle data acquisition module, a road vehicle detection model and a terminal device; the road vehicle data acquisition module is a color camera, which is used to collect road vehicle image data in real time and input the collected image data into the road vehicle detection model; the road vehicle detection model recognizes and outputs vehicle information in the image. The road vehicle detection model is an improved YOLOv4 algorithm model, and the specific network structure is as follows Figure 1 As shown, the improved YOLOv4 algorithm model includes a backbone network, a neck and a head. The backbone network uses the MobileVit network as the basic network. The MobileVit network has ten layers and includes stacked MobileNetv2 modules and MobileVit modules. The neck includes a spatial pyramid pooling layer SPP and a prediction layer PANet. The outputs of the seventh, ninth and tenth layers of the MobileVit network are connected to the prediction layer PANet of the improved YOLOv4 algorithm model. The head outputs the parameters of the prior frame adjustment. The loss function Loss of the improved YOLOv4 algorithm model includes the positioning loss function loss cdiou , confidence loss function loss confidence And category loss function loss class , where the positioning loss function loss cdiou is the CDIOU loss function; the terminal device includes a processor and a memory, the memory stores the program required for the road vehicle detection model to perform road vehicle detection, and the processor is used to run the program stored in the memory.

[0043] The MobileVit network is an image classification network. When performing road vehicle detection, semantic information and location information need to be extracted. The terminal operation of the downsampling module in the MobileVit network uses a 3×3 convolution kernel with a step size of 2 to downsample the input image, reduce feature parameters, and extract more semantic information. The feature information extracted by the MobileVit network at the tenth layer is already 32 times that of the downsampling. Continuing to extract features will lose a large amount of location information, and the subsequent network involves classification tasks. Therefore, the embodiment of the present invention discards the network structure after the tenth layer of the MobileVit network, and uses the remaining part as the backbone feature extraction network of the improved YOLOv4 algorithm model. In order to achieve multi-scale prediction, the features extracted from the seventh, ninth, and tenth layers of the MobileVit network are input into the PANet network for feature fusion to obtain rich feature information. The MobileVit network is used to replace the CSPDarknet53 network as the backbone feature extraction network of the improved YOLOv4 algorithm model for feature extraction, making the improved YOLOv4 algorithm model more lightweight and improving the detection speed of the model;

[0044] The MobileVit network is used instead of the CSPDarknet53 network as the backbone feature extraction network of the improved YOLOv4 algorithm model for feature extraction, making the improved YOLOv4 algorithm model more lightweight and improving the detection speed of the model; the convolutional neural network has the ability to learn local feature extraction with only a small number of parameters, and the VisionTransformer can learn the ability of global feature extraction, but it requires a large number of parameters to learn. The MobileVit network combines the advantages of the convolutional neural network and the Vision Transformer to improve the improved YOLOv4 algorithm model, taking into account the real-time and accuracy of model detection.

[0045] Table 1 shows the MobileVit network structure of an embodiment of the present invention, and the activation functions of the MobileVit network are SILU activation function and RELU activation function.

[0046] Table 1 MobileVit network structure of the embodiment of the present invention

[0047]

[0048] The spatial pyramid pooling layer SPP is connected to the end of the MobileVit network. Through four pooling layers of different scales, it expands the receptive field and separates the most significant contextual features. In order to improve the detection performance of the improved YOLOv4 algorithm model for small targets, the prediction layer network uses a feature fusion network based on PANet. In the feature extraction process, the feature information extracted by the shallow network and the deep network is different. The shallow feature learns more location information of the target, while the deep feature learns more semantic information of the target. The prediction layer PANet fuses the deep and shallow feature information to achieve feature enhancement, while avoiding a large amount of information loss caused by the use of single feature information. The PANet network designs two feature fusion paths, one from bottom to top and the other from top to bottom, so that the PANet network can fully fuse the shallow and deep features, improve the model's detection performance for small targets, and thus achieve multi-scale prediction. The data output by the head Head is the judgment and adjustment of the preset prior frame in the image, which involves judging whether the prior frame contains a target and what type of target it contains, as well as the adjustment data for the size and position of the prior frame containing the target.

[0049] In the embodiment of the present invention, the loss function Loss and the positioning loss function loss cdiou , confidence loss function loss confidence And category loss function loss class The specific expression is as follows:

[0050] Loss = loss cdiou +loss confidence +loss class (1)

[0051]

[0052]

[0053]

[0054] Among them, S 2 is the number of grids divided in the image; B is the number of prior boxes contained in each grid; Is there an object in the j-th prior box of the i-th grid? If there is an object, it takes 1; if there is no object, it takes 0; Is the j-th prior box of the i-th grid without an object, 1 if there is no object, and 0 if there is a target; λ noobj is the confidence error weight of the priori box without the target. Since there are very few priori boxes containing targets in actual training, the positive samples and negative samples in the priori boxes are extremely unbalanced, so in this embodiment of the present invention, λ noobj Set to 0.001; C ij is the confidence of the j-th prediction box of the i-th grid; is the confidence of the jth ground truth box of the i-th grid; P i j is the confidence of the j-th prediction box of the i-th grid; is the confidence of the jth ground truth box of the ith grid; classes represents the set of all target categories, c represents a target category in classes. In the embodiment of the present invention, classes includes three categories: bus, car and truck, and c is one of the categories of bus, car and truck; AE, BF, CG and DH represent Figure 2 The CDIOU loss function can speed up the process of minimizing the distance between the center points of the predicted box ABCD and the real box EFGH without increasing the amount of calculation, and improve the detection accuracy of the YOLOv4 algorithm model.

[0055] like Figure 3 As shown, the embodiment of the present invention also adopts a road vehicle detection method based on YOLOv4, and uses the above-mentioned road vehicle detection system based on YOLOv4 to perform road vehicle detection, including the following steps:

[0056] Step S1: construct an improved YOLOv4 algorithm model as described in the above technical solution.

[0057] Step S2: Divide the road surface images into vehicle information training images, vehicle information verification images and vehicle information test images, construct a vehicle information dataset in VOC format, and obtain a training set, a verification set and a test set; vehicle information includes category information and location information. The category information includes three categories: bus, car and truck. The location information includes the center point coordinates of the real frame, the width and height of the real frame. The specific method is:

[0058] Step S201: Create a file voc_classes.txt, add vehicle dataset labels, use the labellmg image annotation tool to complete the image annotation, and generate an xml annotation file in PASCAL VOC format;

[0059] Step S202: Use the directory structure of the PASCAL VOC dataset to create a vehicle information folder Vehicle. The Vehicle folder contains three subfolders, namely, the Annotations folder, the ImageSets folder, and the JPEGImages folder. Among them, the Annotations folder contains all the xml tag files; the ImageSets file stores the txt files of the dataset division; the JPEGImages folder contains all the vehicle information training images, vehicle information verification images, and vehicle information test images;

[0060] Step S203: Name the images in the JPEGImages folder one by one according to the format requirements of PASCAL VOC, annotate the category information and location information of the annotation file corresponding to each image, save it as an XML file with the same file name as the corresponding image, and then put it into the Annotations folder;

[0061] Step S204: Generate training set, validation set and test set files, create a labels subfolder under the vehicle information folder Vehicle, and the labels subfolder contains a yolo format annotation file of each image, and the yolo format annotation file is converted from the xml annotation file;

[0062] Step S205: In the labels subfolder, create 2007_train.txt file, 2007_val.text file and 2007_test.txt file. The three files correspond to the lists of training set, validation set and test set files, respectively. The lists contain the file paths and file names of all pictures. Create a folder Main in the ImageSets folder, and create train.txt file, val.text file and test.txt file in the folder Main. The three files contain the file names of each picture, but do not contain file path and extension information. The training set is used to train the model, the validation set is used to assess the effect of model training, and the test set is used to detect the final performance of the model.

[0063] Step S3: Use the training set obtained in step S2 to train the improved YOLOv4 algorithm model. The training process includes two stages. In the first training stage, the backbone network of the improved YOLOv4 algorithm model is frozen, and no parameter update is performed. The remaining network layers are trained and the parameters are updated. In the second training stage, all network layers are trained and the parameters are updated. The training methods of the first training stage and the second training stage are the same. The specific training steps are as follows:

[0064] Step S301: Use the GridMask image enhancement method to preprocess the images in the training set, and perform discrete and orderly deletion of information in the image to reduce excessive deletion and excessive retention of regions. The process of the GridMask image enhancement method is as follows Figure 4 As shown, the input image can be represented as X(X∈R H×W×C ), H, W, and C represent the height, width, and number of channels of the input image, respectively. GridMask generates a mask with the same size as the input image and represents it as M. The pixel value corresponding to the gray part in M ​​is 1, and the pixel value corresponding to the black part is 0. The GridMask data augmentation operation multiplies the input image by the mask matrix, and the result is that the information of the gray area is retained, and the information of the black area is deleted.

[0065] like Figure 4 As shown in (b), the dotted box is a unit of the mask, and the mask is defined by four parameters, namely r, d, x, and y. Assuming the image size is 1×1, r represents the proportion of the gray short edge in a unit, d represents the side length of a unit, and x and y represent the distance from the image boundary to the first complete unit.

[0066] The GridMask image enhancement method is used to discretely and orderly delete the information in the input image, achieving an effective balance between excessive information deletion and excessive information retention, thereby simulating partial occlusion of vehicle targets. The image data containing partially occluded targets is input into the improved YOLOv4 algorithm model for training to improve the improved YOLOv4 algorithm model's ability to detect occluded targets.

[0067] Step S302: cluster the real boxes in the training set through the K-means algorithm to obtain the size of the prior box; then input the preprocessed training set pictures into the improved YOLOv4 algorithm model constructed in step S1, firstly fully extract the local features and global features of the pictures through the MobileVit network, and the process of extracting features by the MobileNetv2 module and the MobileVit module in the MobileVit network is as follows:

[0068] The MobileNetv2 module performs a 1×1 convolution operation on the input feature map to expand the number of channels, then performs a 3×3 depth-wise separable convolution operation to reduce the amount of computation, and then performs a 1×1 convolution operation to reduce the number of channels and add the result to the skip chain at the input of the MobileNetv2 module to obtain the feature map output by the MobileNetv2 module.

[0069] The MobileVit module encodes the local spatial information and the number of channels of the expanded features through an n×n and a 1×1 convolution operation on the input feature map, and then extracts the global features through L Transformer operations, where the value of L is referred to Table 1; the number of channels is compressed through a 1×1 convolution, and finally the features are fused with the features input by the MobileVit module, and then an n×n convolution is performed to reduce the number of channels to obtain the final output features.

[0070] After fully extracting the local and global features of the image through the MobileVit network, the spatial pyramid pooling layer SPP is used to increase the network receptive field to obtain richer features, and the prediction layer PANet fuses shallow features and deep features and optimizes the loss function. Finally, the head Head outputs the judgment and adjustment data of the prior frame in the image, adjusts the prior frame, and obtains the predicted frame.

[0071] Step S303: After setting the confidence threshold and classification threshold for the prediction box using the DIOU NMS algorithm, sort them from high to low according to the scores, select the detection box with the highest score, and then remove the detection box with a low score to complete the training of the improved YOLOv4 algorithm model. By setting the confidence threshold and classification threshold for the prediction box, a large number of prediction boxes that do not contain targets or contain part of the target can be deleted. The DIOU NMS algorithm can select the most suitable prediction box for the current target while removing other redundant boxes, thereby improving the accuracy of the detection box.

[0072] Step S4: Use the trained improved YOLOv4 algorithm model to detect the road vehicle image data collected in real time and identify the vehicle targets in the image.

[0073] In the embodiment of the present invention, the number of iterations in the first training stage is set to 37400, and the learning rate is modified to 0.001; the initial learning rate in the second training stage is set to 0.0001, and the number of iterations is uncertain until the improved YOLOv4 algorithm model converges, that is, when the Loss variable in the output log of the improved YOLOv4 algorithm model no longer decreases, the second training stage is stopped.

[0074] The embodiment of the present invention uses an improved YOLOv4 algorithm model, replaces the CIoU loss function with the CDIOU loss function to optimize the YOLOv4 algorithm model, and uses the GridMask image enhancement method for data preprocessing, thereby improving the existing YOLOv4 model's low accuracy, slow detection speed, and insufficient lightness, while improving the detection efficiency and ensuring the model's detection accuracy. Compared with the existing YOLOv4 model, the embodiment of the present invention has higher detection accuracy, faster detection speed, and good effectiveness and precision.

[0075] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A road vehicle detection system based on YOLOv4, characterized in that: It includes a road vehicle data acquisition module, a road vehicle detection model and a terminal device; the road vehicle data acquisition module is a color camera, which is used to collect road vehicle image data in real time, and input the collected image data into the road vehicle detection model; the road vehicle detection model recognizes and outputs vehicle information in the image, and the road vehicle detection model is an improved YOLOv4 algorithm model, the backbone network of the improved YOLOv4 algorithm model uses the MobileVit network as the basic network, the number of layers of the MobileVit network is ten, and the MobileVit network includes MobileNetv2 modules and MobileVit modules stacked on each other; The outputs of the seventh, ninth and tenth layers of the MobileVit network are connected to the prediction layer PANet of the improved YOLOv4 algorithm model; the loss function Loss of the improved YOLOv4 algorithm model includes a positioning loss function , confidence loss function And the category loss function , where the positioning loss function is a CDIOU loss function; the terminal device includes a processor and a memory, the memory stores a program required for the road vehicle detection model to perform road vehicle detection, and the processor is used to run the program stored in the memory; The loss function Loss, positioning loss function , confidence loss function And the category loss function The specific expression is as follows: ； in, S 2 is the number of grids divided in the image; B is the number of prior boxes contained in each grid; For the i The grid j Whether the prior box has an object, if it has an object, it takes 1, if it does not have an object, it takes 0; For the i The grid j Whether there is no target in the prior box, 1 is taken if there is no target, and 0 is taken if there is a target; is the confidence error weight of the prior box that does not contain the target; For the i The grid j The confidence of the predicted box; For the i The grid j The confidence of the ground-truth box; For the i The grid j The confidence of the predicted box; For the i The grid j classes represents the set of all target categories, c represents a target category in classes; AE, BF, CG, DH represent the distances between the four corresponding vertices of the predicted box ABCD and the real box EFGH, respectively, and AG is the diagonal length of the minimum rectangle enclosing the predicted box ABCD and the real box EFGH.

2. A road vehicle detection system based on YOLOv4 according to claim 1, characterized in that: The activation functions of the MobileVit network are SILU activation function and RELU activation function.

3. A road vehicle detection method based on YOLOv4, characterized in that: The steps include: Step S1: constructing an improved YOLOv4 algorithm model as claimed in any one of claims 1 to 2; Step S2: Divide the road surface images into vehicle information training images, vehicle information verification images and vehicle information test images, construct a vehicle information dataset in VOC format, and obtain a training set, a verification set and a test set; Step S3: Use the training set obtained in step S2 to train the improved YOLOv4 algorithm model. The training process includes two stages. In the first training stage, the backbone network of the improved YOLOv4 algorithm model is frozen, and no parameter update is performed. The remaining network layers are trained and the parameters are updated. In the second training stage, all network layers are trained and the parameters are updated. The training methods of the first training stage and the second training stage are the same. The specific training steps are as follows: Step S301: Preprocess the images in the training set using the GridMask image enhancement method to delete the information in the image in a discrete and orderly manner, reducing excessive deletion of regions and excessive retention; Step S302: cluster the real frames in the training set through the K-means algorithm to obtain the size of the prior frame; then input the preprocessed training set pictures into the improved YOLOv4 algorithm model constructed in step S1, and fully extract the local features and global features of the pictures through the MobileVit network, increase the network receptive field through the spatial pyramid pooling layer SPP to obtain richer features, and integrate the shallow features and deep features through the prediction layer PANet, and optimize the loss function. Finally, the head Head outputs the judgment and adjustment data of the prior frame in the picture, and adjusts the prior frame to obtain the predicted frame; Step S303: After setting the confidence threshold and classification threshold for the prediction box using the DIOU NMS algorithm, sort them from high to low according to the scores, select the detection box with the highest score, and then remove the detection box with a low score to complete the training of the improved YOLOv4 algorithm model; Step S4: Use the trained improved YOLOv4 algorithm model to detect the road vehicle image data collected in real time and identify the vehicle targets in the image.

4. According to a road vehicle detection method based on YOLOv4 according to claim 3, in the step S3, the number of iterations in the first training stage is set to 37400, and the learning rate is modified to 0.001; the initial learning rate in the second training stage is set to 0.0001, and the number of iterations is uncertain, until the improved YOLOv4 algorithm model converges, that is, when the Loss variable in the output log of the improved YOLOv4 algorithm model no longer decreases, the second training stage is stopped.

5. A road vehicle detection method based on YOLOv4 according to claim 3, characterized in that: In step S2, the vehicle information includes category information and position information. The category information includes three categories: bus, car and truck. The position information includes the coordinates of the center point of the real frame, and the width and height of the real frame.

6. A road vehicle detection method based on YOLOv4 according to claim 3, characterized in that: The specific method of step S2 is: Step S201: Create a file voc_classes.txt, add vehicle dataset labels, use the labellmg image annotation tool to complete the image annotation, and generate an xml annotation file in PASCAL VOC format; Step S202: Use the directory structure of the PASCAL VOC dataset to create a vehicle information folder Vehicle. The Vehicle folder contains three subfolders, namely, the Annotations folder, the ImageSets folder, and the JPEGImages folder. Among them, the Annotations folder contains all the xml tag files; the ImageSets file stores the txt files of the dataset division; the JPEGImages folder contains all the vehicle information training images, vehicle information verification images, and vehicle information test images; Step S203: Name the images in the JPEGImages folder one by one according to the format requirements of PASCAL VOC, annotate the category information and location information of the annotation file corresponding to each image, save it as an XML file with the same file name as the corresponding image, and then put it into the Annotations folder; Step S204: Generate training set, validation set and test set files, create a labels subfolder under the vehicle information folder Vehicle, and the labels subfolder contains a yolo format annotation file of each image, and the yolo format annotation file is converted from the xml annotation file; Step S205: In the labels subfolder, create 2007_train.txt file, 2007_val.text file and 2007_test.txt file. The three files correspond to the lists of training set, validation set and test set files respectively, and the lists contain the file paths and file names of all images; create a folder Main in the ImageSets folder, and create train.txt file, val.text file and test.txt file in the folder Main. The three files contain the file name of each image, but do not contain file path and extension information.

7. A road vehicle detection method based on YOLOv4 according to claim 3, characterized in that: In step S4, the MobileNetv2 module performs a 1×1 convolution operation on the input feature map to expand the number of channels, performs a 3×3 depth-separable convolution operation to reduce the amount of calculation, and then performs a 1×1 convolution operation to reduce the number of channels and add the result to the skip chain at the input end of the MobileNetv2 module to obtain the feature map output by the MobileNetv2 module.

8. A road vehicle detection method based on YOLOv4 according to claim 3, characterized in that: In the step S4, the MobileVit module encodes the local spatial information and the number of channels of the expanded features through an n×n and a 1×1 convolution operation on the input feature map, extracts the global features through L Transformer operations, compresses the number of channels through a 1×1 convolution, and finally fuses the features input by the MobileVit module, and then reduces the number of channels through an n×n convolution to obtain the final output features.

Images