A lightweight vehicle detection method based on deep learning

Abstract

A kind of lightweight vehicle detection method based on deep learning, for vehicle detection task, the backbone network of the architecture of one-stage target detection algorithm based on Anchor-Free, feature fusion layer, prediction subnetwork respectively are improved lightweight, realize a kind of lightweight vehicle detection method based on deep learning;First, a kind of lightweight backbone network ResCSPNet-Tiny is established;Then a kind of lightweight feature fusion network LPANet is established;Finally, a kind of lightweight multi-scale prediction subnetwork is established in the mode of joint representation;At the same time, in order to improve the detection ability of the algorithm to vehicle target, during training, dynamic label assignment method, and generalized focal loss function and generalized intersection over union loss function are used, and finally the experimental results on BDD vehicle dataset show that: the speed of SFVD algorithm of the application can reach 448FPS, mAP can reach 37.1%, the speed and accuracy are better than the existing lightweight detection algorithm, can better handle the vehicle detection task in driving scene.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing and target detection, and specifically relates to a lightweight vehicle detection method based on deep learning. Background Art

[0002] Vehicle detection is a branch of general target detection. It detects vehicles such as cars, trucks, and vans by extracting and classifying features from video data from on-board cameras, and provides the IDS with information such as the location, category, and size of surrounding vehicles for intelligent decision-making.

[0003] The emergence of deep learning has greatly promoted the development of the object detection field. Leveraging the powerful feature extraction capabilities of deep neural networks (DNNs), deep learning-based object detection algorithms have rapidly become mainstream research. While these algorithms offer high accuracy, common algorithms often suffer from complex network structures, large number of parameters, and heavy computational overhead. The limited performance of automotive processors makes it difficult to meet the real-time requirements of running such complex algorithms. Currently, commonly used lightweight detection algorithms, such as YOLOv4-Tiny and NanoDet, have low accuracy for vehicle object detection. Summary of the Invention

[0004] In order to overcome the defects of the above-mentioned prior art, the purpose of the present invention is to propose a lightweight vehicle detection method based on deep learning, which can, on the one hand, reduce the number of parameters and computational complexity of the algorithm model, and on the other hand, improve the accuracy of the detection model. It has the advantages of good real-time performance, small computational complexity and high accuracy, and can well perform real-time vehicle detection tasks on vehicle-mounted equipment.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is:

[0006] A lightweight vehicle detection method based on deep learning, specifically comprising the following steps:

[0007] Step 1. Input the vehicle video image data to be detected;

[0008] Step 2. Preprocess the image input in step 1 by random flipping, brightness enhancement, contrast enhancement, saturation enhancement, standardization, scale scaling, and stretching methods;

[0009] Step 3. Input the image preprocessed in step 2 into the lightweight backbone network ResCSPNet-Tiny for multiple downsampling feature extraction, and output the multi-scale feature P after downsampling feature extraction. i ;

[0010] Step 4. The multi-scale features P obtained in step 3 are aggregated using a lightweight path aggregation network LPANet. i Fuse and output the fused multi-scale feature C i ;

[0011] Step 5. The multi-scale feature C obtained in step 4 is processed through the multi-scale prediction sub-network i Further processing is performed to obtain the final multi-scale prediction value O i ;

[0012] Step 6. Multi-scale prediction value O obtained in step 5 i Decode and get the predicted target category score Score and position information Pre box ;

[0013] Step 7. Perform positive and negative sample label division and calculate the labels of positive and negative samples; then calculate the loss through GFL and GIOU, use the backpropagation algorithm to iteratively update the model parameters, and finally complete the training of the algorithm model;

[0014] Step 8. In the actual use stage of the algorithm model, directly combine the target category score Score obtained in step 6 with the position information Pre box After NMS processing, the vehicle detection results are output.

[0015] The step 3 is specifically as follows: by performing multiple downsampling feature extraction on the feature extraction unit of the lightweight backbone network ResCSPNet-Tiny, the multi-scale feature P is output. i :

[0016] The first feature extraction unit is used to sequentially slice the input image and perform a standard convolutional layer to obtain a feature map P1;

[0017] A second feature extraction unit is used to pass the feature map P1 output by the first feature extraction unit through a shallow residual module and a downsampling module to obtain a feature map P2;

[0018] A third feature extraction unit is used to obtain a feature map P3 by passing the feature map P2 output by the second feature extraction unit through a shallow residual module and a downsampling module;

[0019] a fourth feature extraction unit, configured to obtain a feature map P4 by passing the feature map P3 output by the third feature extraction unit through a cross-level local module and a downsampling module;

[0020] The fifth feature extraction unit is used to obtain a feature map P5 by passing the feature map P4 output by the fourth feature extraction unit through a cross-level local module and a downsampling module.

[0021] The step 4 is specifically as follows:

[0022] The multi-scale feature maps P3, P4, and P5 obtained in step 3 are passed through the convolution layer to transform the channel dimension;

[0023] Construct a top-down feature fusion path, perform upsampling of the multi-scale feature maps P4 and P5 through bilinear interpolation, and perform fusion of features of different scales through addition to obtain multi-scale feature maps H3, H4, and H5.

[0024] A bottom-up feature fusion path is constructed, and the downsampling of the multi-scale feature maps H3 and H4 is completed through bilinear interpolation operations. The fusion of features of different scales is completed through addition operations to obtain multi-scale feature maps C3, C4, and C5.

[0025] The step 5 is specifically as follows:

[0026] The multi-scale feature maps C3, C4, and C5 are sequentially passed through the convolution layer, batch normalization layer, activation function layer, and convolution layer to obtain the final multi-scale prediction values ​​O3, O4, and O5.

[0027] The step 7 is specifically as follows:

[0028] 1). Customize the sample set of deep learning network training hyperparameters;

[0029] 2) Input the batch of images in the training sample set in step 7 (1) into the constructed network for forward propagation to obtain the predicted multi-scale features O i ;

[0030] 3) Decoding step 7 Multi-scale feature O obtained in step 2) i Prediction information of different scale features in:

[0031] 3.1) Multi-scale feature O i Divide in the channel dimension to obtain the quality prediction value Pre of dimension N cls and dimension is 4*RegMax border regression prediction value Pre box ;

[0032] 3.2) For a multi-scale feature O of size S×S i , this S 2 The feature points are mapped back to the original image to obtain the coordinates of the preset anchor point A;

[0033] 3.3) For quality prediction Pre cls , after the channel dimension is activated by the Sigmoid function, the classification prediction score Score of the category corresponding to each anchor position is obtained;

[0034] 3.4) For the bounding box regression prediction Pre box The value represents the current anchor point A xy The distance of the four edges of the prediction box 4 is represented by a one-dimensional vector V with a length of RegMax; after the Softmax function is performed on the distance vector V of each edge, the value at each position represents its probability at the current position, and the final actual distance distance is calculated in an expected manner;

[0035] 4) Perform positive and negative sample label division:

[0036] 4.1) Initialize the target label label of quality prediction cls The target label label of bounding box regression reg ;

[0037] 4.2) Obtain the distance vector (t, r, b, l) from the regression prediction value Pre obtained in step 7, step 3.1) box , and then decode it with P xy to obtain the target box anchor predicted by the algorithm;

[0038] 4.3) Select all P and its corresponding anchor that fall within the GT box , denoted as P candidate , anchor candidate , and set the label candidate of P cls to the GT box of the corresponding GT label ;

[0039] 4.4) Calculate the iou of all anchor candidate and GT box , denoted as score iou ; calculate the cost of iou cost iou = -log(score iou );

[0040] 4.5) Multiply the label value of P candidate and score iou to obtain label soft , and let label cls = label soft ;

[0041] 4.6) For anchor candidate , calculate the cross-entropy loss of its quality Score and label soft , denoted as cost cls ;

[0042] 4.7) Calculate anchor candidate The cost matrix M = cost cls +cost iou ;

[0043] 4.8) According to the set K value, for each GT box Select score iou The first K anchors candidate , recorded as anchor k , calculate the anchor k score iou The sum of , and rounded down, is obtained for each GT box The number of anchor boxes n should be assigned;

[0044] 4.9) According to each GT box The number n that needs to be allocated, for each GT box Select the first n smallest anchors in M candidate As positive samples, the rest are regarded as negative samples, and the label is selected according to the positive and negative samples. reg Filling

[0045] 4.10) If there is an anchor candidate By multiple GT box In the case of selection, select the GT corresponding to the minimum value in M box As its target, other GT box rounding;

[0046] 4.11) Return the divided sample label label cls ,label reg ;

[0047] 5) Define the generalized focal loss function GFL and calculate the loss value between the predicted box and the corresponding sample:

[0048] 5.1) For the multi-scale prediction value O i The classification prediction value in the classification is calculated using QFL.

[0049] The loss calculation of the quantity forecast is as shown in formula (1.1);

[0050] loss cls =loss QFL

[0051] loss QFL (σ)=-|y-σ|β((1-y)log(1-σ)+ylog(σ)) (1.1)

[0052] Where y is the true quality label, and its value is a continuous value between [0, 1]. For positive samples, its value is the IOU value between the predicted box and the true target box, and for negative samples, its value is 0; σ is the classification prediction score of the current network, that is, the value of the classification prediction value after the function is activated, and β is the weight coefficient;

[0053] 5.2) For multi-scale features O i The bounding box regression prediction value in is used to calculate the regression loss only for positive samples. The loss composition of the bounding box regression branch is shown in formula (1.2).

[0054] loss reg =loss DFL +loss GIOU (1.2)

[0055] 5.2.1) Use DFL to calculate the loss of the probability distribution of the bounding box regression prediction value. Learn the distribution of the bounding box probability and optimize the probability values ​​on both sides of the closest target position, so that the model can quickly focus on the distribution of the area near the target position. The calculation method of DFL is shown in formula (1.3):

[0056] loss DFL (S i ,S i+1 )=-((y i+1 -y)log(S i )+(yy i )log(S i+1 )) (1.3)

[0057] Among them, y represents the subscript of the target position, y i Indicates the position subscript on the left side of the target position, y i+1 Indicates the right position subscript, y i The probability of the position, y i+1 Probability of location;

[0058] 5.2.2) After decoding the bounding box regression prediction value, the position information of the target box is obtained. The GIOU loss function is used to directly calculate the loss between the predicted target box information and the actual target box information. The calculation process is shown in formula (1.4):

[0059]

[0060]

[0061] loss GIOU =1-GIoU

[0062] Where A and B are the predicted box and the real box respectively, and C is the minimum bounding rectangle containing A and B;

[0063] 5.3) Finally, the results of the QFL, DFL, and GIOU loss calculations are added together to obtain the final loss function, which is specifically constructed as shown in formula (1.5):

[0064] loss = W1loss QFL +W2loss DFL +W3loss GIOU (1.5)

[0065] Among them, W1, W2, and W3 represent the weights of different losses respectively.

[0066] The step 8 is specifically as follows:

[0067] 1) After adjusting the image in the actual usage scenario, i.e. the test sample set, to a custom size, the images are sequentially input into the trained deep learning network for forward propagation to obtain the predicted category and predicted box coordinates for each image;

[0068] 2) Use the non-maximum suppression method to filter out overlapping prediction boxes and use the final prediction box to mark the location and category of the target on the image:

[0069] 2.1) Set the screening threshold N;

[0070] 2.2) Use all prediction boxes to form the initial prediction box set B;

[0071] 2.3) Create the final prediction box set D and initialize it to an empty set;

[0072] 2.4) Multiply the IOU value of the predicted box and the true box by the predicted category probability as the confidence level, and sort the predicted boxes in set B in descending order according to the confidence level;

[0073] 2.5) Select the first predicted box, add it to set D, and then delete it from set B;

[0074] 2.6) Traverse all the predicted boxes in set B and calculate the IOU value between them and the final predicted box in set D. If the IOU value is greater than the threshold N, it will be deleted from set B;

[0075] 2.7) Repeat steps 2.5) to 2.6) until set B is empty;

[0076] 2.8) Output the elements in the set D as the final prediction box, and use the final prediction box to mark the location and category of the target in the image to complete the target detection.

[0077] The beneficial effects of the present invention are:

[0078] First, the present invention uses the ResCSPNet-Tiny network as the backbone network, which greatly reduces the number of parameters of the deep learning network and ensures the feature extraction capability of the backbone network, so that the present invention can be deployed and applied on resource-constrained devices, with high detection speed and high detection accuracy.

[0079] Second, the present invention uses the LPANet network as the feature fusion network, which greatly reduces the computational complexity of the feature fusion layer while ensuring the algorithm model's ability to detect objects with varying scales.

[0080] Third, the lightweight vehicle detection algorithm established by the present invention not only ensures the lightweight of the algorithm model, but also greatly improves the algorithm's ability to detect vehicle targets in complex driving scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 It is an implementation flow chart of the present invention.

[0082] Figure 2 This is the ResCSPNet-Tiny network structure diagram constructed by the present invention.

[0083] Figure 3 This is a comparison diagram of the shallow residual unit constructed by the present invention and the residual unit in ResNet.

[0084] Figure 4 It is a shallow cross-level local structure framework diagram constructed by the present invention.

[0085] Figure 5 This is the network structure diagram of the lightweight feature fusion LPANet in the present invention.

[0086] Figure 6 This is the structural diagram of the lightweight multi-scale prediction subnetwork in the present invention.

[0087] Figure 7 This is a comparison chart of the detection effects of the present invention and the prior art, where (a) is the detection effect of the algorithm of the present invention, (b) is the detection effect of the YOLOv4-Tiny-416 algorithm, and (c) is the detection effect of the NanoDet-m algorithm. DETAILED DESCRIPTION

[0088] The present invention will be described in further detail below with reference to the accompanying drawings.

[0089] See also Figure 1 , a lightweight vehicle detection method based on deep learning, including the following specific steps:

[0090] (1) Obtaining vehicle dataset

[0091] Download the BDD dataset from the Internet, which includes road target detection, which can implement four types of image annotations: area division, lane line marking, and instance segmentation; it includes six types of weather: snowy, foggy, sunny, cloudy, overcast, and rainy, with sunny days being the main ones; it includes six types of scenes: residential area, gas station, tunnel, highway, city, and parking lot, with city streets being the main ones; it includes three time stages: dawn (dusk), daytime, and night, with daytime and night being the main ones; it includes a total of 10 types of objects: bus, light, sign, person, bike, truck, motor, car, train, and rider.

[0092] For the specific task of vehicle detection, this paper extracts annotated images containing three vehicle categories: Car, Truck, and Bus from the road object detection label. Since the official test set does not provide annotated files, this paper uses the validation set as the test set.

[0093] After extraction, the training set used in the present invention contains 69,133 images containing vehicles, and the test set contains 9,902 images containing vehicles.

[0094] (2) Building a lightweight backbone network ResCSPNet-Tiny

[0095] The backbone network consists of five feature extraction units, such as Figure 2 shown.

[0096] The first feature extraction unit is used to slice the input image in sequence and then pass it through a set of 3*3 standard convolutional layers to obtain a feature map P1 with a height of 160, a width of 160, and 32 channels;

[0097] The second feature extraction unit is used to pass the feature map output by the first feature extraction unit through a shallow residual module and a downsampling module to obtain a feature map P2 with a height of 80, a width of 80, and 64 channels;

[0098] The third feature extraction unit is used to pass the feature map output by the second feature extraction unit through a shallow residual module and a downsampling module to obtain a feature map P3 with a height of 40, a width of 40, and a channel number of 128;

[0099] The fourth feature extraction unit is used to sequentially pass the feature map output by the third feature extraction unit through the cross-level local module and the downsampling model to obtain a feature map P4 with a height of 20, a width of 20, and a channel number of 256;

[0100] A fifth feature extraction unit is configured to sequentially pass the feature map output by the third feature extraction unit through a cross-level local module and a downsampling model to obtain a feature map P5 with a height of 10, a width of 10, and a channel number of 512;

[0101] The second and third shallow residual modules are as follows Figure 3 As shown in the figure, the corresponding N and C are 160, 32 and 80, 64 respectively. Figure 4 As shown, the corresponding N and C are 40, 128 and 20, 256 respectively.

[0102] The output of the third, fourth, and fifth feature extraction units is a set of multi-scale feature maps P3, P4, and P5. The number of channels of this set of feature maps is 128, 256, and 512, and the downsampling ratio relative to the image is 8, 16, and 32.

[0103] (3) Constructing a lightweight multi-scale feature fusion network

[0104] The input multi-scale feature maps P3, P4, and P5 are passed through a 1*1 convolutional layer with a step size of 1 to transform the channel dimension to 256;

[0105] Construct a top-down feature fusion path, perform upsampling of the multi-scale feature maps P4 and P5 through bilinear interpolation, and perform fusion of features of different scales through addition to obtain multi-scale feature maps H3, H4, and H5.

[0106] Construct a bottom-up feature fusion path, complete the downsampling of the multi-scale feature maps H3 and H4 through bilinear interpolation operations, and complete the fusion of different scale features through addition operations to obtain multi-scale feature maps C3, C4, and C5;

[0107] The process of feature fusion is as follows Figure 5 shown.

[0108] (4) Constructing a lightweight multi-scale prediction head

[0109] The multi-scale feature maps C3, C4, and C5 are first passed through a 3*3 convolutional layer with a stride of 1, a batch normalization layer, and a LeakyReLU activation function layer, and then through a 1*1 convolutional layer with a stride of 1, resulting in the predicted multi-scale features O3, O4, and O5. The output channels of the first convolutional layer are 256, and the output channels of the second convolutional layer are 35.

[0110] Lightweight multi-scale prediction sub-network such as Figure 6 shown.

[0111] (5) Training vehicle detection model

[0112] 5.1) Preprocessing the training sample set: resize the image to 320×320 and use the data augmentation method shown in the following table.

[0113]

[0114]

[0115] 5.2) Set the deep learning network training hyperparameters: Set the loss function coefficients to: W1 = 1, W2 = 0.25, W3 = 2; use the AdamW optimizer, set the initial learning rate to 0.001, the weight decay coefficient to 0.05, and the batch size to 160; set the total training cycle to 300, of which the first 500 iterations use a linear warm-up strategy; use a cosine annealing learning rate decay strategy, with the minimum learning rate set to 0.00005.

[0116] 5.3) Input the batch of images in the training sample set into the constructed network for forward propagation to obtain the predicted multi-scale features O i .

[0117] 5.4) Adopt an adaptive training sample allocation strategy to assign positive and negative samples to each prediction box.

[0118] 5.4.1) Decoding prediction information of features at different scales:

[0119] 5.4.1.1) Multi-scale feature O i Divide in the channel dimension to obtain the quality prediction value Pre of dimension N cls and dimension is 4*RegMax border regression prediction value Pre box In the present invention, N=3 and RegMax=8.

[0120] 5.4.1.2) For a multi-scale feature O of size S×S i , this S 2 The feature points are mapped back to the original image to obtain the coordinates of the preset anchor point P.

[0121] 5.4.1.3) For quality prediction Pre cls After the channel dimension is activated by the Sigmoid function, the classification prediction score Score of the category corresponding to each anchor position is obtained.

[0122] 5.4.1.4) For bounding box regression prediction Pre box , whose value represents the current anchor point P xyThe distances (t, r, b, l) to the four edges of the predicted box are represented by a one-dimensional vector V of length RegMax. After applying the Softmax function to the distance vector V for each edge, the value at each position represents the probability of being at the current position. The actual distance is calculated using the expected method.

[0123] 5.4.2) Perform positive and negative sample label division

[0124] 5.4.2.1) Initialize the target label for quality prediction cls , target label label for bounding box regression reg ;

[0125] 5.4.2.2) For (t, r, b, l) and P xy Decode and get the algorithm predicted target box anchor;

[0126] 5.4.2.3) Select all the center points P that fall into GT box The P and its corresponding anchor are denoted as P candidate , anchor candidate , P candidate label cls Set to corresponding GT box GT label ;

[0127] 5.4.2.4) Calculate all anchors candidate With GT box IOU, recorded as score iou ; Calculate the cost of iou iou =-log(score iou );

[0128] 5.4.2.5) For P candidate The label value and score iou Multiply to get label soft , and let label cls =label soft ;

[0129] 5.4.2.6) For anchor candidate , calculate its quality Score and label soft The cross entropy loss is denoted as cost cls ;

[0130] 5.4.2.7) Calculating anchors candidate The cost matrix M = cost cls +costiou ;

[0131] 5.4.2.8) According to the set K value, for each GT box Select score iou The first K anchors candidate (In this invention, K is 13), recorded as anchor k , calculate the anchor k score iou The sum of , and rounded down, is obtained for each GT box The number of anchor boxes n should be assigned;

[0132] 5.4.2.9) Then according to each GT box The number n that needs to be allocated, for each GT box Select the first n smallest anchors in M candidate As positive samples, the rest are regarded as negative samples, and the label is selected according to the positive and negative samples. reg Filling

[0133] 5.4.2.10) If there is an anchor candidate By multiple GT box In the case of selection, select the GT corresponding to the minimum value in M box As its target, other GT box rounding;

[0134] 5.4.2.11) Return the divided sample label label cls ,label reg .

[0135] 6) Define the generalized focal loss function GFL and calculate the loss value between the predicted box and the corresponding sample.

[0136] 6.1) For classification prediction, QFL is used to calculate the loss of classification quality prediction, which is calculated as shown in the following formula.

[0137] loss cls =loss QFL

[0138] loss QFL (σ)=-|y-σ| β ((1-y)log(1-σ)+ylog(σ))

[0139] Where y is the true quality label, which is a continuous value between [0, 1]. For positive samples, its value is the IOU value between the predicted box and the true target box, and for negative samples, its value is 0. σ is the classification prediction score of the current network, that is, the value of the classification prediction value after activation by the Sigmoid function. β is the weight coefficient, which is 2 in this paper.

[0140] 6.2) For bounding box regression prediction, regression loss is calculated only for positive samples. This paper uses DFL and GIOU to calculate the loss separately. The loss structure of the bounding box regression branch is shown below.

[0141] loss reg =loss DFL +loss GIOU

[0142] 6.2.1) First, we use DFL to calculate the loss of the probability distribution of the bounding box regression output. We learn the distribution of bounding box probabilities and optimize the probability values ​​closest to the target location on both sides, allowing the model to quickly focus on the distribution of the area near the target location. The DFL calculation method is as follows:

[0143] loss DFL (S i ,S i+1 )=-((y i+1 -y)log(S i )+(yy i )log(S i+1 ))

[0144] Among them, y represents the subscript of the target position, y i Indicates the position subscript on the left side of the target position, y i+1 Indicates the right position subscript, y i The probability of the position, y i+1 The probability of the position.

[0145] 6.2.2) After decoding the output of the bounding box regression prediction, the position information of the target box is obtained. The predicted target box and the real target box are compared using the GIOU loss function to calculate the loss. The calculation process is as follows:

[0146]

[0147]

[0148] loss GIOU =1-GIoU

[0149] Among them, A and B are the predicted box and the real box respectively, and C is the minimum bounding rectangle containing A and B.

[0150] 6.3) Finally, the results of the three loss calculations, QFL, DFL, and GIOU, are added together to obtain the final function, which is specifically constructed as follows:

[0151] loss = W1loss QFL +W2loss DFL +W3loss GIOU

[0152] Where W1, W2, and W3 represent the weights of different losses respectively

[0153] 7) Carry out vehicle inspection

[0154] 7.1) Resize the images in the test sample set to 320×320 and input them into the trained deep learning network for forward propagation to obtain the predicted category and predicted box coordinates of each image.

[0155] 7.2) Use non-maximum suppression to filter out overlapping prediction boxes and use the final prediction box to mark the location and category of the target on the image:

[0156] 7.2.1) Set the screening threshold N.

[0157] 7.2.2) Use all prediction boxes to form the initial prediction box set B.

[0158] 7.2.3) Create the final prediction box set D and initialize it to an empty set.

[0159] 7.2.4) Multiply the IOU value of the predicted box and the true box and the predicted category probability as the confidence level, and sort the predicted boxes in set B in descending order according to the confidence level.

[0160] 7.2.5) Select the first predicted box, add it to set D, and then delete it from set B.

[0161] 7.2.6) Traverse all predicted boxes in set B and calculate their IOU values ​​with the final predicted boxes in set D. If the IOU value is greater than the threshold N, delete it from set B.

[0162] 7.2.7) Repeat steps 7.2.5) to 7.2.6) until set B is empty.

[0163] 7.2.8) Output the elements in the set D as the final prediction box, and use the final prediction box to mark the location and category of the target in the image to complete the target detection.

[0164] Example 1

[0165] The detection effect of the present invention can be further illustrated by the following experiments:

[0166] The hardware environment of the experiment is:

[0167] TU102[GeForce RTX 2080Ti]

[0168] 32 Intel(R)Xeon(R)Silver 4110 CPU@2.10GHz

[0169] The software environment configuration is as follows:

[0170] Ubuntu 20.04, CUDA 11.4, OpenCV 4.5.3, Python 3.8.12, Pytorch

[0171] 1.10.1.

[0172] Performance comparison with other lightweight detection algorithms on the BDD vehicle dataset:

[0173] As can be seen from the table below, when the input image size is 320×320, in terms of speed, the algorithm of the present invention can achieve an inference speed of 448 frames per second, which is about 5% faster than the inference speed of YOLOv4-Tiny and about 1.5 times faster than the inference speed of NanoDet-m. In terms of accuracy, the algorithm of the present invention achieves a mAP of 37.1%, which is 6.8% higher than YOLOv4-Tiny and 12% higher than NanoDet-m. In terms of model parameter count, SFVD has 4.06M parameters, more than 4 times that of NanoDet-m, but compared to YOLOv4-Tiny, its parameter count is reduced by nearly 30%, and the total parameter count remains within an acceptable range. For YOLOv4-Tiny with an input image size of 416×416, the algorithm of the present invention still maintains a leading position in all indicators.

[0174]

[0175] In order to compare the detection results more intuitively, the present invention visualizes the detection results of various algorithms, as shown in the attached figure. Figure 7 , where from top to bottom correspond to the detection results of the algorithm of the present invention, YOLOv4-Tiny-416, and NanoDet-m. As can be seen from the figure, the algorithm of the present invention is more accurate in detecting vehicle targets. The vehicle detection algorithm designed by the present invention is fast and accurate, and has better performance than existing lightweight algorithms. It is more suitable for handling vehicle detection tasks in driving scenarios. Figure 7 .

Claims

1. A lightweight vehicle detection method based on deep learning, characterized by: The specific steps include: Step 1. Input the vehicle video image data to be detected; Step 2. Preprocess the image input in step 1 by random flipping, brightness enhancement, contrast enhancement, saturation enhancement, normalization, scaling, and stretching. Step 3. Input the image preprocessed in step 2 into the feature extraction unit of the lightweight backbone network ResCSPNet-Tiny, perform multiple downsampling feature extractions, and output multi-scale features. : The first feature extraction unit is used to sequentially slice the input image and perform a standard convolutional layer to obtain a feature map P1; A second feature extraction unit is used to pass the feature map P1 output by the first feature extraction unit through a shallow residual module and a downsampling module to obtain a feature map P2; A third feature extraction unit is used to obtain a feature map P3 by passing the feature map P2 output by the second feature extraction unit through a shallow residual module and a downsampling module; a fourth feature extraction unit, configured to obtain a feature map P4 by passing the feature map P3 output by the third feature extraction unit through a cross-level local module and a downsampling module; a fifth feature extraction unit, configured to obtain a feature map P5 by subjecting the feature map P4 output by the fourth feature extraction unit to a cross-level local module and a downsampling module; Step 4. The multi-scale features obtained in step 3 are aggregated using a lightweight path aggregation network LPANet. Fuse and output the fused multi-scale features ; Specifically: the multi-scale feature maps P3, P4, and P5 obtained in step 3 are respectively passed through the convolution layer to transform the channel dimension; Construct a top-down feature fusion path, perform upsampling of the multi-scale feature maps P4 and P5 through bilinear interpolation, and perform fusion of features of different scales through addition to obtain multi-scale feature maps H3, H4, and H5. Construct a bottom-up feature fusion path, complete the downsampling of the multi-scale feature maps H3 and H4 through bilinear interpolation operations, and complete the fusion of different scale features through addition operations to obtain multi-scale feature maps C3, C4, and C5; Step 5. Use the multi-scale prediction sub-network to predict the multi-scale features obtained in step 4 Further processing to obtain the final multi-scale prediction value , that is, the multi-scale feature maps C3, C4, and C5 are sequentially passed through the convolution layer, batch normalization layer, activation function layer, and convolution layer to obtain the final multi-scale prediction values ​​O3, O4, and O5; Step 6. Multi-scale prediction value obtained in step 5 Decode and get the predicted target category score and location information ; Step 7. Perform positive and negative sample label division and calculate the labels of positive and negative samples; Then, the loss is calculated through GFL and GIOU, and the model parameters are iteratively updated using the back-propagation algorithm to complete the training of the algorithm model. Step 8. When the algorithm model is actually used, directly use the target category score obtained in step 6 and location information After NMS processing, the vehicle detection results are output.

2. The method of lightweight vehicle detection based on deep learning according to claim 1, characterized in that: The step 7 is specifically as follows: 1). Customize the sample set of deep learning network training hyperparameters; 2) Input the batch of images in the training sample set in step 7 (1) into the constructed network for forward propagation to obtain the predicted multi-scale features. ; 3). Multi-scale features obtained in step 2) of decoding step 7 Prediction information of different scale features in: 3.1) Multi-scale features Divide in the channel dimension, and the dimension is Quality prediction value and dimensions are Bounding box regression prediction value ; 3.2) For a Multi-scale features of size , put this Map the feature points back to the original image to get the preset anchor points coordinates of 3.3) For quality prediction After the channel dimension is activated by the Sigmoid function, the classification prediction score of the corresponding category of each anchor position is obtained ; 3.4) For bounding box regression prediction , whose value represents the current anchor point Distance from the four sides of the prediction box , using a length of One-dimensional vector of To represent; the distance vector of each edge After the Softmax function is performed, the value of each position represents the probability of being at the current position, and the actual distance is , calculated using the expected method; 4) Perform positive and negative sample label division: 4.1) Initialization of target labels for quality prediction , the target label for bounding box regression ; 4.2) The regression prediction value obtained by step 3.1) of step 7 Get the distance vector , and then compare it with Decode and get the target frame predicted by the algorithm ; 4.3) Select all center points fall into within and its corresponding , recorded as , ,Will of Set to correspond of ; 4.4) Calculate all and of , recorded as ;calculate The price ; 4.5) For of Value and Multiplying together, we get , and order ; 4.6) For , calculate its mass and The cross entropy loss is recorded as ; 4.7) Calculation The cost matrix ; 4.8) According to the settings Value, for each Select Center front indivual , recorded as ,calculate of The sum of , and rounded down, we get each The number of anchor boxes that should be assigned ; 4.9) According to each Quantity to be allocated , for each Select Center front Small As positive samples, the rest are regarded as negative samples. Filling 4.10) If there is a By multiple Select the case, select The minimum value corresponds to As its goal, other rounding; 4.11) Return the divided sample labels ; 5) Define the generalized focal loss function GFL and calculate the loss value between the predicted box and the corresponding sample: 5.1) For multi-scale prediction values The classification prediction value in is used to calculate the loss of classification quality prediction using QFL, which is calculated as shown in formula (1.1); （1.1） in, is the true quality label, whose value is a continuous value between [0,1]. For positive samples, its value is the IOU value between the predicted box and the true target box, and for negative samples, its value is 0; is the classification prediction score of the current network, that is, the value of the classification prediction value after the function is activated. is the weight coefficient; 5.2) For multi-scale features The bounding box regression prediction value in is used to calculate the regression loss only for positive samples. The loss composition of the bounding box regression branch is shown in formula (1.2): （1.2） 5.2.1) Use DFL to calculate the loss of the probability distribution of the bounding box regression prediction value. Learn the distribution of the bounding box probability and optimize the probability values ​​on both sides of the closest target position, so that the model can quickly focus on the distribution of the area near the target position. The calculation method of DFL is shown in formula (1.3): （1.3） in, The subscript representing the target position, Indicates the position subscript to the left of the target position, Indicates the right position subscript, for The probability of the position, for Probability of location; 5.2.2) After decoding the bounding box regression prediction value, the position information of the target box is obtained. The GIOU loss function is used to directly calculate the loss between the predicted target box information and the actual target box information. The calculation process is shown in formula (1.4): （1.4） in are the predicted box and the true box respectively, To include The minimum bounding rectangle of ; 5.3) Finally, the QFL, DFL, and GIOU loss calculation results are added together to obtain the final loss function, which is specifically composed as shown in formula (1.5): （1.5） in They represent the weights of different losses.

3. The lightweight vehicle detection method based on deep learning according to claim 1, characterized in that: The step 8 is specifically as follows: 1) After resizing the images in the actual usage scenario, i.e., the test sample set, to a custom size, the images are sequentially fed into the trained deep learning network for forward propagation to obtain the predicted category and predicted box coordinates for each image; 2) Use the non-maximum suppression method to filter out overlapping prediction boxes and use the final prediction box to mark the location and category of the target on the image: 2.1) Set the screening threshold N; 2.2) Use all prediction boxes to form the initial prediction box set B; 2.3) Create the final prediction box set D and initialize it to an empty set; 2.4) Multiply the IOU value of the predicted box and the true box and the predicted category probability as the confidence level, and sort the predicted boxes in set B in descending order according to the confidence level; 2.5) Select the first predicted box, add it to set D, and then delete it from set B; 2.6) Traverse all the predicted boxes in set B and calculate their IOU values ​​with the final predicted boxes in set D. If the IOU value is greater than the threshold N, it is deleted from set B; 2.7) Repeat steps 2.5) to 2.6) until set B is empty; 2.8) Output the elements in the set D as the final prediction box, and use the final prediction box to mark the location and category of the target in the image to complete the target detection.

Images