Small target detection method for automatic driving scene

Abstract

The invention relates to the technical field of target detection of deep learning, in particular to an automatic driving scene-oriented small target detection method, which innovatively adopts an improved structural design, better fuses high-level semantic information and low-level feature information through a bidirectional feature fusion module, and adopts a learnable anchor strategy, so that the detection accuracy is improved. Through the randomly initialized anchor size and size, the anchor size and size are learned according to the error loss in the training stage, compared with a traditional artificially designed anchor super parameter, the method has higher generalization, and experiments prove that the performance index APs of small target detection is also improved while better performance can be achieved by adopting a smaller number of anchors, so that the method is suitable for large-scale popularization and application. And the calculation amount of the algorithm is reduced due to fewer anchors, and the detection speed is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning target detection, and in particular to a small target detection method for autonomous driving scenarios. Background Art

[0002] At present, deep learning small target detection methods can be divided into single-stage and dual-stage detection methods. Single-stage detection methods include Yolo series, SSD, RetinaNet, etc., which extract image features through the network and directly predict the category and location information of the target on the feature map. The dual-stage detection method is mainly based on the Faster R-CNN series. First, the image features are extracted through the neural network, and then the Region Proposal Network (RPN) is used to preliminarily extract the potential target location information. Finally, the final prediction result is obtained based on the preliminary predicted suggestion box. The single-stage method has a faster detection speed. Compared with the dual-stage detection method, it omits the step of generating potential target regions and is an end-to-end detection method.

[0003] In real life, due to the inconsistent size distribution of targets in different scenes, detection methods such as single-stage SSD and RetinaNet that use the anchor mechanism set the size and dimensions of the anchor through clustering or manual experience, resulting in the lack of generalization of target detection methods. If the distribution of target sizes between data sets is very different, it may lead to a decrease in the performance of the detection method. In addition, in self-driving scenarios, traffic signs are mostly small targets that occupy fewer image pixels, but most algorithms have low performance in detecting small targets. Summary of the invention

[0004] In order to reduce the problems of poor generalization and low small target detection performance caused by manually setting anchors, the present invention proposes a small target detection method for autonomous driving scenarios based on a single-stage detection method, and provides a learnable anchor strategy that can meet high accuracy and real-time requirements.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] The small target detection method for autonomous driving scenarios specifically includes the following steps:

[0007] Step 0: Initialization of the detection method: including network initialization and image preprocessing. A learnable strategy is used for the size and size of the anchor. In the initial stage, it is randomly set and the size and size of the anchor is learned through the network. In the image preprocessing part, during the training stage, it mainly includes random scaling, color jittering, and normalization of the image.

[0008] Step 1: The input is image data, and the features are extracted from the preprocessed image to obtain the feature maps C3, C4 and C5 after 8, 16 and 32 times downsampling respectively;

[0009] Step 2: Using the improved feature fusion module, the features of C3, C4 and C5 are bidirectionally fused to obtain F3, F4 and F5. The F5 feature map is downsampled twice to further extract image features to obtain F6 and F7.

[0010] Step 3: Through the shared decoupled prediction branch, prediction is performed on the F3, F4, F5, F6 and F7 feature maps to obtain the category and location information prediction of the target;

[0011] Step 4: Positive and negative sample allocation: By calculating the intersection over union (IoU) of the prediction and the anchor, when IoU is greater than 0.5, the prediction is a positive sample; when IoU is less than 0.4, the prediction is a negative sample; when IoU is an intermediate value, the prediction is to ignore the sample and no loss is calculated;

[0012] Step 5: Calculate the loss: By calculating the loss of the predicted target and the true value, the loss includes category loss and coordinate loss. The category loss is calculated using Focal loss, and the calculation formula is shown in (1-1). The coordinate loss is calculated using Generalized IoU, and the calculation formula is shown in (1-2).

[0013]

[0014] Where α and γ are balance factors, p t is the category prediction probability, A C is the area of ​​the smallest rectangular box that encloses the target box and the prediction box, and U is the overlapping area of ​​the target box and the prediction box;

[0015] Step 6: Update network parameters: Based on the error, learn and adjust the network weight parameters, including the size and size of the anchor, through gradient back propagation.

[0016] Furthermore, in the step Step0, the image is randomly scaled to a size ranging from 640, 672, 704, 736, 768, and 800, and the maximum side length after scaling does not exceed 1333 pixels.

[0017] Furthermore, the Step 1 uses ResNet50 as the backbone network to extract features from the preprocessed image.

[0018] Furthermore, the improved feature fusion module of Step 2 includes a dimensionality reduction layer, a dimensionality increase layer, and up- and down-sampling modules; the dimensionality increase layer and the dimensionality reduction layer adopt 1×1 convolution to change the dimension size of the feature channel, the up- and down-sampling modules adopt a residual structure, the up-sampling module adopts a transposed convolution with a step size of 2 to achieve the expansion of the feature map, and the down-sampling module mainly uses 3×3 convolution to achieve a 2-fold scaling of the feature map.

[0019] Furthermore, the decoupling category and position information prediction branches in Step 3 are respectively connected in series using three convolutional layers.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] The present invention innovatively adopts an improved structural design, and through a bidirectional feature fusion module, high-level semantic information and low-level feature information are better integrated. At the same time, a learnable anchor strategy is adopted. Through randomly initialized anchor sizes, the anchor sizes are learned according to error loss during the training phase. Compared with traditional manually designed anchor hyperparameters, this method is more generalizable. Experiments have proved that using fewer anchors can achieve better performance, and the performance indicator APs of small target detection is also improved. Fewer anchors also reduce the computational complexity of the algorithm and improve the detection speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:

[0023] Figure 1 This is a training flow chart of the algorithm of the present invention.

[0024] Figure 2 This is a structural diagram of the feature fusion module of the present invention.

[0025] Figure 3 The detection results of the baseline network and the algorithm are shown in Figure 2. DETAILED DESCRIPTION

[0026] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation mode, structure, characteristics and effects of the present invention are described in detail below in combination with the accompanying drawings and preferred embodiments.

[0027] The present invention proposes a small target detection method for autonomous driving scenarios, which is an improved algorithm based on the learnable anchor strategy (hereinafter referred to as LAtinaNet algorithm). It is a single-stage detection method. After feature extraction and feature fusion of the input image, the decoupled prediction branch obtains the category and position information of the target, performs initialization processing before image processing, learns the size and dimension of the anchor through the network, and calculates the error loss between the predicted target and the real target during the training phase, and adjusts the anchor through gradient back propagation. The specific detection method is as follows Figure 1 As shown, the following steps are included:

[0028] Step 0: Initialization of the detection method. Including network parameter initialization and image preprocessing. Among them, a learnable strategy is used for the size and size of the anchor, which is randomly set in the initial stage, and the size and size of the anchor are learned through the network. The image preprocessing part, in the training stage, mainly includes random scaling, color jittering, and normalization of the image. Among them, the random scaling of the image size has a scaling range of (640, 672, 704, 736, 768, 800), and the maximum side length after scaling does not exceed 1333 pixels.

[0029] Step 1: The input is image data. The present invention adopts ResNet50 as the backbone network to extract features of the preprocessed image and obtain feature maps C3, C4 and C5 after 8, 16 and 32 times downsampling respectively.

[0030] Step 2: Use the improved feature fusion module (such as Figure 2 As shown in FIG. 1 , the features of C3, C4 and C5 are bidirectionally fused to obtain F3, F4 and F5. Among them, the F5 feature map is downsampled twice to further extract image features to obtain F6 and F7.

[0031] Step 3: Through the shared decoupled prediction branch, prediction is performed on the F3, F4, F5, F6 and F7 feature maps to obtain the target category prediction and coordinate prediction. The decoupled category and coordinate prediction branches are connected in series using three convolutional layers respectively.

[0032] Step 4: Assignment of positive and negative samples. By calculating the intersection over union (IoU) between the prediction and the anchor, when IoU is greater than 0.5, the prediction is a positive sample; when IoU is less than 0.4, the prediction is a negative sample; when IoU is an intermediate value, the prediction is ignored and no loss is calculated.

[0033] Step 5: Calculate the loss. By calculating the loss between the predicted target and the true value, the loss includes category loss and coordinate loss. The category loss is calculated using Focal loss, and the calculation formula is shown in (1-1). The coordinate loss is calculated using Generalized IoU, and the calculation formula is shown in (1-2).

[0034]

[0035] Where α and γ are balance factors, p t is the category prediction probability, A C is the area of ​​the smallest rectangular box that encloses the target box and the prediction box, and U is the overlapping area of ​​the target box and the prediction box.

[0036] Step 6: Update network parameters. Based on the error, learn and adjust the network weight parameters, including the size and size of the anchor, through gradient back propagation.

[0037] Step 7: After one round of parameter iteration, determine whether the preset number of training rounds has been reached until the training is completed. This method trains for a total of 30 epochs.

[0038] Example: Detection experiment of a small target detection method (LAtinaNet) for autonomous driving scenarios in the present invention

[0039] 1) Comparison of test results

[0040] This experiment is trained and tested on the TT100K traffic sign dataset. The dataset contains about 100,000 images in total, of which 10,592 images are annotated, and the annotated images include 232 traffic sign categories. Figure 3 As shown in the figure, the left column is the detection results of the baseline network, the middle column is the detection results after the improved network structure, and the right column is the LAtinaNet detection structure after the learnable anchor strategy. It can be seen that the baseline network has false detections, the method with the improved network structure has no false detections but has missed detections, and LAtinaNet can correctly detect traffic signs in the image without false alarms or missed detections.

[0041] At the same time, the test was conducted on the VisDrone dataset to further verify the improvement of this method in small target detection performance. The specific detection results are shown in Table 1. Compared with the other five methods, LAtinaNet has the smallest network parameter amount while ensuring detection performance, and also ensures the accuracy of small target detection.

[0042] Table 1 Performance comparison of five detection methods

[0043] Method AP <![CDATA[AP 50 ]]> <![CDATA[AP 75 ]]> APs <![CDATA[AP M ]]> <![CDATA[AP L ]]> Params(M) Cascade R-CNN 23.2 39.9 23.4 16.5 36.8 39.4 273.2 Libra RCNN 24.3 41.2 24.9 16.8 34.0 36.8 185.4 HawkNet 25.6 44.3 25.8 19.9 36.0 39.1 130.9 VFNet 25.9 42.1 27.0 16.8 37.3 41.4 296.2 LAtinaNet 26.6 46.1 24.3 18.7 34.9 38.3 45.9

[0044] Table 2 shows the ablation experiment of different anchor parameters, where 'LA' represents the learnable anchor strategy, 'anchorscale' represents the number of anchor sizes used, and 'aspect ratio' represents the number of anchor aspect ratios used. As can be seen from Table 2, the method of using 1 size and 1 aspect ratio (a total of 1 anchor) improves the small target detection performance index APs by 2.1% compared with the traditional method of using 3 sizes and 3 aspect ratios (a total of 9 anchors) while ensuring performance. Using fewer anchors reduces the amount of model calculation and speeds up the algorithm reasoning.

[0045] Table 2 Ablation experiments with different anchor parameters

[0046]

[0047]

[0048] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A small target detection method for autonomous driving scenarios, characterized by: The specific steps include: Step 0: Initialization of the detection method: including network initialization and image preprocessing. A learnable strategy is used for the size and size of the anchor. In the initial stage, it is randomly set and the size and size of the anchor is learned through the network. In the image preprocessing part, during the training stage, it mainly includes random scaling, color jittering, and normalization of the image. Step 1: The input is image data, and the features are extracted from the preprocessed image to obtain the feature maps C3, C4 and C5 after 8, 16 and 32 times downsampling respectively; Step 2: Using the improved feature fusion module, the features of C3, C4 and C5 are bidirectionally fused to obtain F3, F4 and F5. The F5 feature map is downsampled twice to further extract image features to obtain F6 and F7. Step 3: Through the shared decoupled prediction branch, prediction is performed on the F3, F4, F5, F6 and F7 feature maps to obtain the category and location information prediction of the target; Step 4: Positive and negative sample allocation: By calculating the intersection over union (IoU) of the prediction and the anchor, when IoU is greater than 0.5, the prediction is a positive sample; when IoU is less than 0.4, the prediction is a negative sample; when IoU is an intermediate value, the prediction is to ignore the sample and no loss is calculated; Step 5: Calculate the loss: Calculate the loss of the predicted target and the true value. The loss includes category loss and coordinate loss. The category loss is calculated using Focal loss. The calculation formula is shown in (1-1). The coordinate loss is calculated using GeneralizedIoU. The calculation formula is shown in (1-2). Where α and γ are balance factors, p t is the category prediction probability, A C is the area of ​​the smallest rectangular box that encloses the target box and the prediction box, and U is the overlapping area of ​​the target box and the prediction box; Step 6: Update network parameters: Based on the error, learn and adjust the network weight parameters, including the size and size of the anchor, through gradient back propagation.

2. The small target detection method for autonomous driving scenarios according to claim 1, characterized in that: In the step Step 0, the image is randomly scaled to a size ranging from 640, 672, 704, 736, 768, and 800, and the maximum side length after scaling does not exceed 1333 pixels.

3. The small target detection method for autonomous driving scenarios according to claim 1, characterized in that: In Step 1, ResNet50 is used as the backbone network to extract features from the preprocessed image.

4. The small target detection method for autonomous driving scenarios according to claim 1, characterized in that: The improved feature fusion module of Step 2 includes a dimensionality reduction layer, a dimensionality increase layer, and up- and down-sampling modules; the dimensionality increase layer and the dimensionality reduction layer adopt 1×1 convolution to change the dimension size of the feature channel, the up- and down-sampling modules adopt a residual structure, the up-sampling module adopts a transposed convolution with a step size of 2 to achieve the expansion of the feature map, and the down-sampling module mainly uses 3×3 convolution to achieve a 2-fold scaling of the feature map.

5. The small target detection method for autonomous driving scenarios according to claim 1, characterized in that: The Step 3 decouples the category and location information prediction branches by connecting three convolutional layers in series.

Images