A vehicle target detection method based on a convolutional neural network

Abstract

The application relates to a vehicle target detection method based on a convolutional neural network, which comprises the following steps: on the basis of a YOLOv3 algorithm, firstly, a backbone network DarkNet53 is used for feature extraction to obtain basic features layer by layer; features of a last layer of the DarkNet53 are input into a Soft-SPP module for feature fusion; the fused features are further extracted by a 1*1 convolutional layer and a 3*3 convolutional layer, and the features are input into a coordinate attention module for weight adjustment between channels; finally, the network outputs three feature maps of different scales through up-sampling, basic anchor frames obtained through preprocessing are used for respectively predicting large, medium and small targets, and two-dimensional surrounding frames and category information of the targets are obtained. Compared with a YOLOv3 model, the convolutional neural network model has better detection effect, and obvious improvement is achieved in subjective vision and objective evaluation indexes, the missing detection and false detection phenomena of vehicle target detection are effectively improved, and the detection speed still meets the requirement of real-time detection.

Description

Technical Field

[0001] This invention relates to the fields of computer vision, object detection, and autonomous driving, specifically a vehicle object detection method based on convolutional neural networks. Background Technology

[0002] Object detection, also known as object recognition, is a classic problem in computer vision. Its purpose is to obtain the two-dimensional bounding box and category of an object of interest from an input RGB image. Accuracy and real-time performance are crucial characteristics of the entire system. Detecting vehicles in real-world scenarios to obtain relevant information is helpful for traffic flow statistics and also assists parking lot scheduling systems and autonomous driving systems.

[0003] Traditional vehicle detection algorithms primarily target static backgrounds. When the target's movement speed is too slow, target extraction becomes difficult and computationally intensive. Furthermore, accuracy drops when the target's size or shape changes. Convolutional neural networks (CNNs), on the other hand, possess strong generalization capabilities and can automatically learn deep semantic features of targets. After training, they can also accurately locate targets, making their application in target detection highly valuable. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a vehicle target detection method based on convolutional neural networks, which achieves high-precision target detection of input images and overcomes the shortcomings of the current YOLOv3 algorithm in small target detection. It can be widely used in urban intelligent transportation and parking lot systems.

[0005] The technical solution adopted by this invention to achieve the above objectives is: a vehicle target detection method based on convolutional neural networks, comprising the following steps:

[0006] Step 1: Obtain RGB images to construct a dataset for training the convolutional neural network. Preprocess the dataset to obtain N anchor boxes.

[0007] Step 2: Extract features from the RGB images in the dataset using the DarkNet53 network;

[0008] Step 3: Using the Soft-SPP module based on feature fusion, the features are input into different pooling layers with different pooling kernels. After pooling in the pooling layer using the SoftPool method, the features from different feature channels are concatenated.

[0009] Step 4: The concatenated first-size feature map is upsampled through a Convolutional Set convolutional layer to obtain a second-size feature map. The second-size feature map is then upsampled through a Convolutional Set convolutional layer to obtain a third-size feature map. The first-size, second-size, and third-size feature maps are concatenated with the first-size, second-size, and third-size feature maps in the DarkNet53 network, respectively, and then input into the first coordinate attention module, the second coordinate attention module, and the third coordinate attention module. Each coordinate attention module assigns different weights to each feature channel and multiplies the input feature map with the weights according to the feature channel to obtain the output feature.

[0010] Step 5: Based on the feature maps of the first, second, and third dimensions output by the first, second, and third coordinate attention modules respectively, use the N / 3 anchor boxes obtained from the preprocessed data as references for prediction, and obtain the predicted boxes on the feature maps of the first, second, and third dimensions respectively. Complete the training of the convolutional neural network composed of DarkNet53 network, Soft-SPP module, Convolutional Set convolutional layer and coordinate attention module to obtain the trained convolutional neural network.

[0011] Step 6: Input the image containing the vehicle target into the trained convolutional neural network to predict the bounding box and target category.

[0012] The process of preprocessing the dataset to obtain N anchor boxes includes the following steps:

[0013] Obtain the label data from the KITTI dataset, use the K-means algorithm to cluster the bounding box sizes in the labels, and obtain the sizes of N anchor boxes to predict targets of different sizes;

[0014] The label data includes the target, its ground truth bounding box location, and its category.

[0015] The feature fusion-based Soft-SPP module uses SoftPool instead of MaxPool for pooling:

[0016]

[0017] Among them, a i For a pixel within the neighborhood R, w i Let be the weight of the i-th pixel, and j represent the pixel index within that neighborhood.

[0018] The output value of pooling is calculated using the weights w. i With the corresponding pixel ai We get the weighted summation as follows:

[0019]

[0020] represents the output value after pooling, and * represents the dot product.

[0021] During training, both the distance between the two corner points of the bounding box and the aspect ratio of the bounding box are considered, and the KIoU function, which measures the closeness between the predicted box and the ground truth box, is used:

[0022]

[0023]

[0024]

[0025] The bounding box loss function is:

[0026] L box =1-KIoU

[0027] Where h and w represent the length and width of the predicted bounding box, respectively. gt and w gt ρ represents the length and width of the ground truth bounding box, respectively; k1 and k2 represent the Euclidean distance between the corresponding points of the predicted bounding box and the ground truth bounding box; k1 and k2 represent the top left and bottom right corners of the predicted bounding box, respectively. and These represent the top-left and bottom-right corners of the true bounding box, respectively; 'c' represents the smallest bounding box A. c The diagonal length of the predicted bounding box; v is used to measure the similarity of the aspect ratio, making the shapes of the predicted bounding box and the ground truth bounding box more similar; α is the weight function, and IoU represents the ratio of the intersection area of ​​the predicted bounding box and the union area of ​​the ground truth bounding box.

[0028] A vehicle target detection system based on a convolutional neural network includes:

[0029] The dataset construction module is used to acquire RGB images to build a dataset for training the convolutional neural network. The dataset is preprocessed to obtain N anchor boxes.

[0030] The feature extraction module is used to extract features from RGB images in the dataset using the DarkNet53 network;

[0031] The Soft-SPP module is used to input features into pooling layers with different pooling kernels, and then concatenate the features from different feature channels after pooling in the pooling layers using the SoftPool method.

[0032] The coordinate attention module is used to upsample the concatenated first-size features through a Convolutional Set layer to obtain a second-size feature map, and then upsample the second-size feature map through a Convolutional Set layer to obtain a third-size feature map. The first, second, and third-size feature maps are then concatenated with the first, second, and third-size feature maps from the DarkNet53 network, respectively, and input into the first, second, and third coordinate attention modules. Each coordinate attention module assigns different weights to each feature channel, multiplies the input feature map with the weights according to the feature channel, and obtains the output feature.

[0033] The convolutional neural network module is used to predict based on the feature maps of the first, second, and third dimensions output by the first, second, and third coordinate attention modules, respectively. It uses N / 3 anchor boxes obtained from preprocessed data as references to obtain prediction boxes on the feature maps of the first, second, and third dimensions, respectively. The module completes the training of the convolutional neural network, which is composed of DarkNet53 network, Soft-SPP module, Convolutional Set convolutional layer and coordinate attention module, to obtain the trained convolutional neural network.

[0034] The object detection module is used to input images containing vehicle targets into a trained convolutional neural network to predict bounding boxes and object categories.

[0035] A vehicle target detection device based on a convolutional neural network includes a memory and a processor; the memory is used to store a computer program; the processor is used to implement the vehicle target detection method based on a convolutional neural network when the computer program is executed.

[0036] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the vehicle target detection method based on a convolutional neural network.

[0037] The present invention has the following beneficial effects and advantages:

[0038] This invention can effectively handle the vehicle target detection problem. The network model can accurately learn the salient features of the image, especially improving the learning of small targets, and finally providing the localization and classification information of the vehicle target. Attached Figure Description

[0039] Figure 1 This is a structural diagram of the vehicle detection neural network in this invention;

[0040] Figure 2 This is a diagram of the DarkNet53 backbone network structure;

[0041] Figure 3 This is a structural diagram of the Soft-SPP module in this invention;

[0042] Figure 4 This is a structural diagram of the coordinate attention module in this invention;

[0043] Figure 5 This is a schematic diagram of the KIoU Loss loss function in this invention. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0045] A vehicle target detection method based on convolutional neural networks includes: firstly, based on the YOLOv3 algorithm, feature extraction is performed using its backbone network DarkNet53 to obtain layer-by-layer basic features; the features from the last layer of DarkNet53 are input into a Soft-SPP module for feature fusion; the fused features are further extracted by 1×1 and 3×3 convolutional layers, and then input into a coordinate attention module for inter-channel weight adjustment; finally, the network outputs feature maps of three different scales through upsampling, and uses the preprocessed basic anchor boxes to predict targets at large, medium, and small scales respectively, obtaining the target's two-dimensional bounding box and category information. The convolutional neural network model proposed in this invention achieves better detection results than the YOLOv3 model, showing significant improvements in both subjective visual perception and objective evaluation metrics, effectively reducing false negatives and false positives in vehicle target detection, while still meeting the requirements for real-time detection.

[0046] YOLOv3 is a classic two-dimensional object detection algorithm. It uses the classic deep network Darknet-53 as its backbone, combines features of ResNet, and borrows from the multi-scale detection method of Feature Pyramid Networks (FPN). It employs feature maps at three different scales to detect large, medium, and small objects respectively. However, YOLOv3 still has shortcomings in small object detection. Therefore, reasonable improvements are needed to make it more suitable for vehicle detection, which will improve the accuracy of localization and recognition.

[0047] The KITTI dataset, jointly created by the Karlsruhe Institute of Technology in Germany and Toyota Research Institute of America, contains real-world image data collected from urban, rural, and highway scenes. Each image contains up to 15 vehicles and 30 pedestrians, with varying degrees of occlusion and truncation. It is currently the largest international dataset for evaluating computer vision algorithms for autonomous driving scenarios.

[0048] like Figure 1 As shown, a vehicle target detection method based on convolutional neural networks addresses the issue of small targets containing only a few pixels and having inconspicuous features. Therefore, a Soft-SPP structure is designed on top of the YOLOv3 neural network to fuse multiple receptive fields. The SoftPool operation is used to preserve details to the greatest extent and effectively extract features from small targets. A coordinate attention mechanism is inserted before the last feature map layer to capture long-range dependencies with precise location information while adjusting the weights assigned to each channel feature. Finally, the KIoU Loss loss function, based on keypoints and aspect ratio, is used as the bounding box loss function, making bounding box regression more accurate. The specific steps include:

[0049] Step 1: Use 7481 training images from the KITTI dataset as experimental data, and preprocess the original label information of the dataset, retaining only the labels of three categories: Car, Van, and Truck. The processed data is divided into training and test sets in a 7:3 ratio.

[0050] Step 2: Read the label data from the KITTI dataset, use the K-means algorithm to cluster the bounding box sizes in the labels, and obtain 9 basic anchor box sizes for target prediction in the last layer of feature map.

[0051] The K-means algorithm classifies given samples based on Euclidean distance. If two samples are close together, they are considered to belong to the same class. Ultimately, the sample set is divided into k clusters, with points within each cluster as closely connected as possible, while maximizing the distance between clusters. Average overlap (AvgIoU) is used as a measure of similarity between data points.

[0052]

[0053] Where B represents the clustering sample, i.e., the width and height of the ground truth bounding box in the dataset; C represents the cluster center; n k The number of samples belonging to the k-th cluster; n is the total number of samples; k represents the number of clusters; I IOU (B,C) represents the intersection-union ratio of the cluster center box and the cluster box. Since YOLOv3 has three prediction scales of different sizes, we selected k=9 pairs of samples in the KITTI dataset for cluster analysis and obtained 9 basic anchor box sizes [78×151, 126×244, 52×95, 40×57, 28×75, 27×39, 19×52, 15×37, 10×27].

[0054] Step 3: Modify the YOLOv3 neural network by adding a Soft-SPP module to the last layer of the backbone feature extraction network and inserting a coordinate attention module before the feature map of the last layer.

[0055] The backbone of YoloV3 is the DarkNet53 network, used to extract features downsampled by 8x, 16x, and 32x respectively. The network extensively uses 1x1 and 3x3 convolutions, with 1x1 primarily used for channel expansion and reduction. The overall convolutional network employs a structure of convolutional layers + batch normalization layers + activation function layers. For example... Figure 2 As shown.

[0056] like Figure 3 As shown, the Soft-SPP module consists of four branches. To ensure the largest pooling kernel is as close as possible to the size of the feature map to be pooled, the largest pooling kernel is designed to be 13×13. The other two window sizes decrease by 4 respectively, set to 9×9 and 5×5. The features of the previous level are soft-pooled by the three different pooling kernel sizes and then concatenated with the unpooled features of the previous level, i.e., superimposed. To ensure that the output size of each branch is consistent, different degrees of padding are performed during soft pooling. The specific calculation of soft pooling is as follows:

[0057] For a local region R in feature map a, the following formula is used to calculate each pixel a. i Calculate a weight:

[0058]

[0059] Among them, w i The value represents the weight of the corresponding pixel, where j represents the element in the neighborhood R of that pixel.

[0060] Finally, the output value of the pooling is calculated using the weights w. iWith the corresponding pixel a i We get the weighted summation as follows:

[0061]

[0062] represents the output value after pooling, and * represents the dot product.

[0063] like Figure 4 As shown, the coordinate attention module includes two steps: coordinate information embedding and coordinate attention generation. Specifically, for the input feature map X, a pooling kernel of size (H,1) is used to encode the features of each channel along the horizontal coordinate, and correspondingly, a pooling kernel of size (1,W) is used along the vertical coordinate direction. The feature map generated after pooling is first concatenated along the dimension, and then transformed using a shared 1×1 convolution. The above process is represented as follows:

[0064] f=δ(F1([z h ,z w ]))

[0065] Among them, z h This represents the feature map after pooling along the horizontal direction, z. w This represents the feature map after pooling along the vertical direction; δ is a non-linear activation function. This is an intermediate feature map encoded in both the horizontal and vertical directions. r is the downsampling rate that controls the module size. C represents the number of channels in the input feature. The superscript indicates the feature map size.

[0066] Next, f is split into two separate feature maps along the spatial dimensions (horizontal and vertical). and Finally, two 1×1 convolutions F are used. h and F w feature map f h and f w Transforming the input X to have the same number of channels yields the following result:

[0067] g h =σ(F h (f h ))

[0068] g w =σ(F w (f w ))

[0069] Finally, regarding g h and g w The input features are multiplied by g and then extended as attention weights. h and gw Output after weighting.

[0070] Step 4: Finally, within the YOLOv3 network framework, the weighted features are processed through a series of 1x1 and 3x3 convolutional layers in a Convolutional Set, and the features are fused in a manner similar to a feature pyramid network. The final output features are 13×13, 26×26, and 52×52, which predict target detection boxes and categories at different scales.

[0071] Step 5: During training, KIoU Loss is used as the loss function for the bounding box. KIoU Loss, based on GIoU, considers both the distance between the two corner points of the bounding box and the aspect ratio of the bounding box.

[0072]

[0073]

[0074]

[0075] Where h and w represent the length and width of the predicted bounding box, respectively. gt and w gt ρ represents the length and width of the ground truth bounding box, ρ represents the Euclidean distance between the two points, and k1 and k2 represent the top left and bottom right corners of the predicted bounding box, respectively. and These represent the top-left and bottom-right corners of the true bounding box, respectively; 'c' represents the smallest bounding box A. c The diagonal length of the bounding box; v is used to measure the similarity of aspect ratio, making the shapes of the predicted box and the ground truth box more similar; α is the weight function. To make the bounding box regression more accurate and converge faster, α can be multiplied by a coefficient to amplify it to 1.2 times in the initial training stage, so that the bounding box first has a shape similar to the ground truth box, and then the two boxes gradually approach and overlap through the penalty term of key points. Figure 5 As shown.

[0076] This invention focuses on vehicle detection in real-world scenarios. It selects 7481 2D images from the KITTI dataset as training data for the vehicle detection model in this paper. The original label information of the dataset is preprocessed, retaining only the labels of the three categories: Car, Van, and Truck. The processed data is divided into training set and test set in a 7:3 ratio.

[0077] The Mosaic data augmentation method is used to read four images from the training set at a time. For each image, conventional augmentation methods such as flipping, scaling, and color gamut adjustment are randomly selected. The four processed images are then cropped and stitched together according to the top left, bottom left, top right, and bottom right positions to form an image containing rich information, thereby increasing the amount of data during training.

[0078] The operating system used in this experiment was Ubuntu 16.04. The server hardware configuration was as follows: Intel(R) Xeon(R) CPU E5-2678v3 @ 2.50GHz, 32GB of RAM, and an Nvidia RTX2080Ti GPU with 11GB of VRAM. The experiment used the PyTorch 1.5.0 framework for model building, training, and testing, with Python version 3.7, CUDA version 10.1, and CuDNN version 7.5.

[0079] The experimental results of this invention are shown in Table 1:

[0080] Table 1

[0081]

[0082] During the training phase, the number of images processed per batch was set to 8 based on the available GPU memory, and the input image size in each batch was fixed at 512×512. The momentum was configured to 0.937, the weight decay to 0.0005, and the initial learning rate to 10. -3 Experiments showed that after the program ran for 60 generations, the loss value oscillated and stopped decreasing. This was because the learning rate was set back to 0.1 in the 60th generation to allow the loss value to continue to decrease slightly, achieving a better fitting effect.

Claims

1. A vehicle target detection method based on convolutional neural networks, characterized in that, Includes the following steps: Step 1: Obtain RGB images to construct a dataset for training the convolutional neural network. Preprocess the dataset to obtain N anchor boxes. Step 2: Extract features from the RGB images in the dataset using the DarkNet53 network; Step 3: Using the Soft-SPP module based on feature fusion, the features are input into different pooling layers with different pooling kernels. After pooling in the pooling layer using the SoftPool method, the features from different feature channels are concatenated. Step 4: The concatenated first-size feature map is upsampled through a Convolutional Set convolutional layer to obtain a second-size feature map, and the second-size feature map is upsampled through a Convolutional Set convolutional layer to obtain a third-size feature map; The feature maps of the first, second, and third dimensions are concatenated with the feature maps of the first, second, and third dimensions in the DarkNet53 network, respectively, and then input into the first, second, and third coordinate attention modules, respectively. Each coordinate attention module assigns different weights to each feature channel and multiplies the input feature map with the weights according to the feature channel to obtain the output feature. Step 5: Based on the feature maps of the first, second, and third dimensions output by the first, second, and third coordinate attention modules respectively, use the N / 3 anchor boxes obtained from the preprocessed data as references for prediction, and obtain the predicted boxes on the feature maps of the first, second, and third dimensions respectively. Complete the training of the convolutional neural network composed of DarkNet53 network, Soft-SPP module, Convolutional Set convolutional layer and coordinate attention module to obtain the trained convolutional neural network. Step 6: Input the image containing the vehicle target into the trained convolutional neural network to predict the bounding box and target category; During training, both the distance between the two corner points of the bounding box and the aspect ratio of the bounding box are considered, and the KIoU function, which measures the closeness between the predicted box and the ground truth box, is used: ； ； ； The bounding box loss function is: ； in, and These represent the length and width of the prediction box, respectively. and These represent the length and width of the actual bounding box, respectively. This represents the Euclidean distance between corresponding points in the predicted bounding box and the ground truth bounding box. and These represent the top left and bottom right corners of the prediction box, respectively. and These represent the top left and bottom right corners of the actual bounding box, respectively. Represents the smallest bounding box The length of the diagonal; It is used to measure the similarity of aspect ratio, making the shapes of the predicted boxes and the ground truth boxes tend to be similar; It is a weighting function. This represents the ratio of the intersection area to the union area of ​​the predicted bounding box and the ground truth bounding box.

2. The vehicle target detection method based on convolutional neural networks according to claim 1, characterized in that, The process of preprocessing the dataset to obtain N anchor boxes includes the following steps: Obtain the label data from the KITTI dataset, use the K-means algorithm to cluster the bounding box sizes in the labels, and obtain the sizes of N anchor boxes to predict targets of different sizes; The label data includes the target, its ground truth bounding box location, and its category.

3. The vehicle target detection method based on convolutional neural networks according to claim 1, characterized in that: The feature fusion-based Soft-SPP module uses SoftPool instead of MaxPool for pooling: ； in, For the neighborhood Pixels within, Let the weight of the i-th pixel be... This represents the pixel number within that neighborhood; The output value of pooling is calculated using the weights. and corresponding pixel We get the weighted summation as follows: ； This represents the output value after pooling. This indicates dot product.

4. A vehicle target detection system based on a convolutional neural network, characterized in that, include: The dataset construction module is used to acquire RGB images to build a dataset for training the convolutional neural network. The dataset is preprocessed to obtain N anchor boxes. The feature extraction module is used to extract features from RGB images in the dataset using the DarkNet53 network; The Soft-SPP module is used to input features into pooling layers with different pooling kernels, and then concatenate the features from different feature channels after pooling in the pooling layers using the SoftPool method. The coordinate attention module is used to upsample the concatenated first-size features through a Convolutional Set convolutional layer to obtain a second-size feature map, and then upsample the second-size feature map through a Convolutional Set convolutional layer to obtain a third-size feature map. The feature maps of the first, second, and third dimensions are concatenated with the feature maps of the first, second, and third dimensions in the DarkNet53 network, respectively, and then input into the first, second, and third coordinate attention modules, respectively. Each coordinate attention module assigns different weights to each feature channel and multiplies the input feature map with the weights according to the feature channel to obtain the output feature. The convolutional neural network module is used to predict based on the feature maps of the first, second, and third dimensions output by the first, second, and third coordinate attention modules, respectively. It uses N / 3 anchor boxes obtained from preprocessed data as references to obtain prediction boxes on the feature maps of the first, second, and third dimensions, respectively. The module completes the training of the convolutional neural network, which is composed of DarkNet53 network, Soft-SPP module, Convolutional Set convolutional layer and coordinate attention module, to obtain the trained convolutional neural network. The object detection module is used to input images containing vehicle targets into a trained convolutional neural network to predict bounding boxes and object categories.

5. A vehicle target detection device based on a convolutional neural network, characterized in that, It includes a memory and a processor; the memory is used to store a computer program; the processor is used to implement, when executing the computer program, a vehicle target detection method based on a convolutional neural network as described in any one of claims 1-3.

6. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements a vehicle target detection method based on a convolutional neural network as described in any one of claims 1-3.

Images