An image target detection method based on deep learning

Abstract

The application discloses a kind of based on deep learning's image target detection method, its steps are as follows: S1, construct image dataset, S2, construct target detection network, its main network adopts CSPDarknet53 structure, neck network adopts SPP structure, head network takes three different scale feature map mode;S3, determine the loss function LLIoU of target detection network, S4, the LLIoU determined by step S3 is used to train target detection network until the loss function LLIoU shows convergence, and target detection network optimization is completed;The method optimizes network loss function gradient and network convergence speed, and the target detection network with high gradient and high precision is constructed, to achieve the purpose of improving the perception accuracy of pedestrians, animals and vehicles in different scenes, and has good application and popularization prospect in subsequent field of unmanned driving.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned driving, in particular to an image target detection method based on deep learning. BACKGROUND

[0002] In recent years, the unmanned driving technology has developed rapidly, which integrates environment perception, decision planning and control, and relies on artificial intelligence technology to enable vehicles to safely travel on the road without drivers. The perception module fuses sensor information such as cameras and laser radars to perceive the environment around the vehicle in real time. The decision module outputs the best decision planning according to the perception information and vehicle state information. The control module controls the vehicle to travel on the road along the planning trajectory at a specified speed according to the decision information. The perception module is the cornerstone of the entire technology chain and is the focus of research on unmanned driving technology.

[0003] There are currently three directions in the research of perception: 1) target detection algorithm based on visual camera, 2) target detection algorithm based on radar sensor, and 3) target detection algorithm based on the fusion of visual camera and radar sensor. These three target detection algorithms have very high requirements for the selection of loss function, and their accuracy is directly related to whether the loss function converges or not.

[0004] The selection of loss function in the existing image target detection method based on deep learning has achieved certain results in the field of unmanned driving, but as the data volume increases, the fluctuation of the target detection network in the training process increases, the gradient of the loss function becomes smaller, and the overall accuracy of the algorithm deteriorates. Therefore, how to optimize the loss function to solve these problems is of great significance to target detection. SUMMARY

[0005] The purpose of the present application is to provide an image target detection method based on deep learning which solves the above technical problems.

[0006] To this end, the technical solution of the present application is as follows:

[0007] An image target detection method based on deep learning, the steps are as follows:

[0008] S1, constructing an image data set, comprising:

[0009] 1) collecting different scene images containing detection targets; the detection targets are selected from at least one of vehicles, pedestrians and animals, and the number of detection targets is ≥1;

[0010] 2) labeling each detection target in each image with a minimum bounding rectangle label box, and obtaining the class name of the detection target in each label box, the height and width of each rectangular label box, and the position coordinates of the center point;

[0011] S2, construct a target detection network composed of a backbone network, a neck network and a head network connected in sequence; the backbone network adopts a CSPDarknet53 structure, the neck network adopts an SPP structure, and the head network adopts three different scale feature map modes;

[0012] S3, determine a loss function LLIoU of the target detection network, and the expression is:

[0013]

[0014] In the formula, A is the area of the image prediction frame output by the target detection network, B is the area of the image real frame, C is the minimum circumscribed rectangle area of the image prediction frame and the real frame, and β1 and β2 are variable gradient parameters;

[0015] S4, adopt the loss function LLIoU determined in step S3 to train the target detection network. The target detection network is trained until the loss function LLIoU converges, and the optimization of the target detection network is completed.

[0016] Further, in step S1, in step S3, the value range of β1 is 1-2, and the value range of β2 is 1-5.

[0017] Further, in the training process of step S4, the initial learning rate of the target detection network is 0.01, Batchsize is 64, and Epoch is 300. The average precision of the network is taken as the standard, and the weight with the highest average precision saved in the training period is taken as the training result.

[0018] Compared with the prior art, the image target detection method based on deep learning not only optimizes the gradient of the network loss function and the network convergence speed, but also further improves the detection precision of the target detection network. A target detection network with high gradient and high precision is constructed to achieve the purpose of improving the perception accuracy of pedestrians, animals and vehicles in different scenes, and has good application and promotion prospects in the field of autonomous driving. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 is a flowchart of the image target detection method based on deep learning of the application;

[0020] Fig. 2(a) is a B-spline fitting simulation diagram of the minimum value of CIoU, DIoU, GIoU and LIoU corresponding to different IoU thresholds when the variable gradient parameter β1=β2=1 in the embodiment of the application;

[0021] Fig. 2(b) is a gradient diagram after derivation of the B-spline fitting curve of Fig. 3(a); ​

[0022] FIG. 3(a) is a B-spline fitting simulation diagram of the minimum values of CIoU, DIoU, GIoU and LIoU corresponding to different IoU thresholds when the variable gradient parameters β1=1.1 and β2=2 in the embodiment of the present application;

[0023] FIG. 3(b) is a gradient diagram after derivation of the B-spline fitting curve in FIG. 3(a);

[0024] Figure 4 FIG. 4 is a comparison of the visualization results on the open source dataset in the embodiment of the present application. DETAILED DESCRIPTION

[0025] The present application will be further described below in conjunction with the drawings and specific embodiments, but the following embodiments are by no means any limitation on the present application.

[0026] Referring to Figure 1 The specific implementation steps of the deep learning-based image target detection method are as follows:

[0027] S1, constructing an image dataset, the specific implementation steps are as follows:

[0028] S101, collecting different scene images containing detection targets; the detection targets are selected from at least one of a vehicle, a pedestrian and an animal, and the number of detection targets is ≥1; the vehicle includes but is not limited to an airplane, a train, a car, a bus, a bicycle, etc.

[0029] S102, processing each image obtained by step S101 to obtain image information of each image; specifically, the image processing method is as follows: first, all detection targets are labeled in the image using a rectangular label box, and the rectangular label box is the minimum circumscribed rectangular box of the detection target; then, the class name of the detection target in each label box, the height and width of each rectangular label box, and the position coordinates of the center point, i.e. the image information, are obtained.

[0030] S103, all images obtained by step S101 and the image information of each image processed by step S102 constitute an image dataset.

[0031] S2, constructing a target detection network:

[0032] The target detection network is constructed based on a YOLOv5 network and is composed of a backbone network, a neck network and a head network connected in sequence. Specifically, the backbone network uses a CSPDarknet53 structure, that is, adopts a Cross Stage Partial Network (CSP) structure to improve the feature extraction capability while keeping the network depth and parameter quantity unchanged. The neck network uses an SPP (Spatial Pyramid Pooling) structure to pool the features at different scales, thereby improving the expression capability of the features. The head network adopts three different scale feature map modes to generate a group of prediction boxes on each feature map, each prediction box including the position and category information of the target, thereby enhancing the detection capability of the network for targets of different sizes.

[0033] S3, determining an optimization loss function of the target detection network, the expression of which is:

[0034]

[0035] In the formula, A is the area of the image prediction box output by the target detection network, B is the area of the image real box, C is the minimum circumscribed rectangle area of the prediction box and the real box of the image, β1 and β2 are variable gradient parameters, the values of which are empirical values and increase with the increase of the data volume; wherein the value range of β1 is 1-2, and the value range of β2 is 1-5.

[0036] Wherein, LIoU is a new position evaluation index constructed by considering the gradient influence of the loss function, the expression of which is:

[0037]

[0038] As the four commonly used position evaluation indexes IoU, CIoU, DIoU and GIoU, the expressions are: CIoU = IoU - Term CIoU , DIoU = IoU - Term DI oU,

[0039] Wherein, Term CIoU and TermDIoU are the penalty terms of CIoU and DIoU, and the calculation methods are: In the formula, ρ(A, B) is the center point distance between A and B, c is the diagonal distance of the minimum enclosing box, v is the error of the aspect ratio between A and B, and α is an adaptive weight coefficient; w g , h g are the width and height of the real box, respectively.p , h p are the width and height of the prediction box, respectively, and the subscript g and p are the abbreviations of ground truth (i.e. real) and prediction (i.e. predicted), respectively;

[0040] In the target detection network, the smaller the position evaluation index value is, the greater the corresponding loss function is, and the more accurate the output result of the corresponding training detection network is; therefore, the new position evaluation index should be constructed to maximize the proportion of the intersection in the loss function, that is, the calculation result is the smallest compared with other evaluation indexes under the same conditions. The new position evaluation index is tested to be the smallest in the calculation result under various possible position relationships between the image prediction box and the real box based on the target detection network, as follows:

[0041] 1) When the positions of the two have no intersection relationship:

[0042] 2) When the positions of the two have an intersection relationship: LIoU≤GIoU≤IoU,

[0043] 3) When the positions of the two have a containing relationship:

[0044] At the same time, it has been verified that the loss function expressed by 1-LIoU as the loss function can make the loss function infinitely close to 0 and 1 in the case of accurate prediction and poor prediction of the network, so as to meet the correction when training the entire network.

[0045] S4, using the loss function determined by step S3 training the target detection network until the loss function is convergent, that is, the target detection network is optimized;

[0046] The python version used in the network training step is 3.6, the pytorch version is 1.7.1, the cuda version is 10.1, and the graphics card is two 2080ti; during the training process, the initial learning rate of the target detection network is 0.01, the Batchsize is 64, and the Epoch is 300, that is, the network has a training period of 300 Epochs, and the average precision of the network is taken as the standard, and the weight with the highest average precision saved in the training period is taken as the training result of the network.

[0047] In order to prove the loss function optimization method of the image target detection method based on deep learning of the present application, the feasibility of the present method is further verified by simulation test.

[0048] First, the values of different loss functions in various position cases are simulated and the gradients are calculated using Matlab. Specifically, a circular area with a center at (10, 10) and a radius of 3 is selected, and 100 random aspect ratio rectangular frames are placed at the center as the real frame; 100 target points are randomly placed in the area, and 100 random aspect ratio rectangular frames are placed at each target point as the predicted frame. The lowest value of the aspect ratio and the center distance between the predicted frame and the real frame are adjusted to simulate as many relative position cases of the predicted frame and the real frame as possible. A total of four repetitions are performed, so the total number of experimental samples is 4x100x100x100 = 4000000.

[0049] Next, the values of IoU, CIoU, DIoU, GIoU and LIoU of the predicted frame and the real frame in each sample are calculated for comparative analysis.

[0050] As shown in FIG. 2(a) and FIG. 3(a) are the result comparison charts when different variable gradient parameters are taken in different positions. As shown in FIG. 2(a), the variable gradient parameters β1 and β2 take the value of 1; as shown in FIG. 3(a), the variable gradient parameters β1 and β2 take the value of 1.1 and 2; as can be clearly seen from the figures, by using the loss function optimization method of the present application, the minimum value of LIoU can be taken as the lowest value of all kinds of IoU, and the value of LIoU will be lower when the value of the variable gradient parameter β is increased to a certain extent. That is, when the loss function is used to calculate the new position evaluation index (i.e. ), The value of LIoU has a greater rising space, and the higher the threshold value of IoU, the more obvious the gradient advantage of LIoU (see the corresponding gradient chart, i.e. FIG. 2(b) and FIG. 3(b)), so that LIoU has a stronger function representation ability in any position case; in addition, based on the fact that the convergence speed of the target detection network is proportional to the gradient of the loss function, the method of the present application not only optimizes the gradient of the network loss function, but also improves the network convergence speed.

[0051] The specific detection accuracy results obtained by comparing IoU, DIoU and LIoU are shown in Table 1.

[0052] Among them,

[0053] Table 1:

[0054]

[0055] From the detection accuracy results shown in Table 1, The three loss functions are evaluated on the YOLOv5s network using the PASCAL VOC dataset, and the precision is improved by 2.61% and 3.48%, respectively. In particular, as evidenced by the results of the simulation experiment described above, the method has a more remarkable precision improvement, with an improvement of 9.03% and 9.98% from the experimental results of the high threshold value (AP 75 to AP 95 ).

[0056] As Figure 4 shown is a visualization result obtained by implementing the method of the present application based on the existing open-source dataset PASCAL VOC in the target detection field, and as a comparison, IoU and LIoU are used as a comparison group when training the target detection network. From Figure 4 the visualization results shown, in the first group of processing results, the method of the present application achieves effective and accurate detection of the partial aircraft appearing in the image, while the detection of the comparison group has a missed detection problem; in the second group of processing results, the method of the present application achieves accurate detection of the image boundary, while the comparison group has unnecessary frame selection on the upper side of the image output frame of the van; in the third group of processing results, the method of the present application achieves accurate detection of the image boundary, while the comparison group has unnecessary frame selection on the left side of the image output frame of the left sheep and / or the lower side of the image output frame of the right sheep; in the fourth group of processing results, the method of the present application achieves accurate detection of the image boundary, while the comparison group does not completely frame the front wheel of the image output frame of the bicycle; in the fifth group of processing results, the method of the present application achieves effective and accurate detection of the image boundary, while the comparison group has a missed detection problem for the bicycle located on the rear side of the two overlapping bicycles, which is detected as one detection target together with the bicycle on the front side.

[0057] In summary, the method of the present application has higher precision and is less likely to have a missed detection problem.

Claims

1. A deep learning-based image target detection method, characterized in that, The steps are as follows: S1. Construct an image dataset, including: 1) Collecting different scene images containing detection targets; the detection targets are selected from at least one of vehicles, pedestrians, and animals, and the number of detection targets is ≥1; 2) Labeling each detection target in each image with a minimum bounding rectangle, and obtaining the category name of the detection target in each label, the height and width of each rectangle, and the position coordinates of the center point; S2. Construct an object detection network, which consists of a backbone network, a neck network, and a head network connected in sequence. The backbone network adopts the CSPDarknet53 structure, the neck network adopts the SPP structure, and the head network adopts three different scale feature map methods. S3. Determine the loss function LLIoU of the object detection network, its expression is: In the formula, A is the area of ​​the image prediction box output by the object detection network, B is the area of ​​the image ground truth box, C is the area of ​​the minimum bounding rectangle of the image prediction box and the ground truth box; β1 and β2 are variable gradient parameters. S4. Use the method determined in step S3. Train the object detection network until the loss function is reached. The convergence indicates that the target detection network optimization is complete.

2. The image target detection method based on deep learning according to claim 1, characterized in that, In step S1 and step S3, the value range of β1 is 1 to 2, and the value range of β2 is 1 to 5.

3. The image target detection method based on deep learning according to claim 1, characterized in that, During the training process in step S4, the initial learning rate of the object detection network is 0.01, the batch size is 64, and the epoch is 300. The average accuracy of the network is used as the standard, and the weight with the highest average accuracy stored in the network during this training cycle is taken as the training result.

Images