A Method for Autonomous Driving Scene Mining Based on Semi-Supervised Transformer Detection

Abstract

The present invention provides an autonomous driving scenario mining method based on semi-supervised transformer detection, including: using an existing dataset as labeled data, setting hyperparameters of the labeled data, and training two detection models respectively; training a small model using the existing dataset; for unlabeled data collected by a camera, obtaining detection results by setting reasonable parameters; taking the intersection of the detection results of the two detection models to obtain the true values of the unlabeled data, which are used as pseudo-labeled data; performing data augmentation processing on the pseudo-labeled data to expand the pseudo-labeled dataset; using the labeled data and the pseudo-labeled dataset as the total data, setting the loss weight of the pseudo-labeled data, and retraining the student detection model; using the small model and the student detection model to perform inference detection on the existing unlabeled data and newly collected unlabeled data to mine the required long-tail scenario data; screening the long-tail scenario data as data to be labeled to expand the long-tail data scenario library.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous driving, and more specifically, to an autonomous driving scenario mining method based on semi-supervised transformer detection. Background Art

[0002] In the autonomous driving scenario, there are many types of targets that need to be detected by the deep learning perception model. In order to achieve better detection accuracy, in the full-supervised learning framework, not only a reasonable algorithm model needs to be constructed, but also an extremely large dataset is required. Currently, in the conventional open-source datasets, due to natural collection, the distribution quantity of targets such as trucks and buses is relatively scarce compared to cars and pedestrians, and rainy and cloudy weather is also relatively rare, that is, there is a problem of data imbalance in the dataset, which is generally called the long-tail problem.

[0003] The long-tail problem will lead to poor detection effects of the target detection model for these scenarios. It is necessary to mine long-tail data from the newly collected data, and then label and expand the existing dataset to ensure a relatively balanced data distribution.

[0004] The cost of obtaining a large amount of labeled data is very high, but we can easily obtain a vast amount of unlabeled data. Currently, mining long-tail scenarios in the vast amount of data generally relies on manual screening. Due to the extremely large amount of data, which can reach the T-level scale, the cost of manual screening increases significantly in the context of big data. How to quickly and effectively improve the performance of the model by using these vast amounts of data, and screen out the required long-tail data, has become an extremely important part of algorithm optimization. Summary of the Invention

[0005] The purpose of the present invention is to provide an autonomous driving scenario mining method based on semi-supervised transformer detection.

[0006] The present invention aims to solve the problems existing in the existing autonomous driving scenario mining methods.

[0007] Compared with the prior art, the technical solution of the present invention and its beneficial effects are as follows:

[0008] A method for mining autonomous driving scenarios based on semi-supervised Transformer detection, comprising: using an existing dataset as labeled data, setting hyperparameters of the labeled data, and training 2 YOLOv5 detection models with Swin Transformer as the backbone respectively; training a small model with the existing dataset; for the unlabeled data collected by the camera, obtaining detection results by setting reasonable parameters; taking the intersection of the detection results of the 2 detection models to obtain the ground truth of the unlabeled data as pseudo-labeled data; performing data augmentation on the pseudo-labeled data to expand the pseudo-labeled dataset; using the labeled data and the pseudo-labeled dataset as the total data, setting the loss weight of the pseudo-labeled data, and retraining the YOLOv5 detection model with Swin Transformer as the backbone, which is called the student model; using the small model and the student detection model to perform inference detection in the existing unlabeled data and newly collected unlabeled data to mine the required long-tail scenario data; screening the long-tail scenario data as the data to be labeled to expand the long-tail data scenario library.

[0009] As a further improvement, for the unlabeled data collected by the camera, obtaining detection results by setting reasonable parameters, including: for the unlabeled data collected by the camera, obtaining detection results by setting reasonable NMS thresholds and confidence thresholds.

[0010] As a further improvement, the data augmentation process includes rotation, translation, and flipping.

[0011] As a further improvement, the version of the detection model is YOLOv5.

[0012] The beneficial effects of the present invention are as follows:

[0013] Semi-supervised learning can make full use of unlabeled data to improve the performance of the model when the labeled data is limited. After comparison, the YOLOv5 detection network with Swin Transformer as the backbone is selected for semi-supervised learning to obtain the benchmark large model; the long-tail data is obtained by comparing the inference of the benchmark large model and the existing deployed small model in the unlabeled data, and finally the local long-tail data scenario library is expanded;

[0014] Through this solution, the present invention can make full use of the vast amount of unlabeled data, mine and expand the local long-tail database, and thus improve the detection performance of the perception model. Description of the Drawings

[0015] Figure 1 It is the first schematic diagram of a method for mining autonomous driving scenarios based on semi-supervised Transformer detection provided by an embodiment of the present invention.

[0016] Figure 2 It is the second schematic diagram of an autonomous driving scenario mining method based on semi-supervised transformer detection provided by an embodiment of the present invention. Detailed implementation manners

[0017] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention. Therefore, the detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed present invention, but merely represents selected embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0018] In the description of the present invention, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the present invention, "a plurality of" means two or more unless otherwise specifically defined.

[0019] Refer to Figure 1 and Figure 2As shown in the figure, a method for mining autonomous driving scenarios based on semi-supervised transformer detection includes: using an existing dataset as labeled data, setting hyperparameters of the labeled data, and training two yolov5 detection models with swin transformer as the backbone, namely teacher1 and teacher2 models; training a small model for field deployment using the existing dataset; obtaining detection results for the unlabeled data collected by the camera by setting reasonable parameters; taking the intersection of the detection results of the two detection models to obtain the ground truth of the unlabeled data as pseudo-labeled data; performing data augmentation on the pseudo-labeled data to expand the pseudo-labeled dataset; using the labeled data and the pseudo-labeled dataset as the total data, setting the loss weight of the pseudo-labeled data, and retraining the yolov5 detection model with swin transformer as the backbone, which is called the student model; using the small model and the student detection model to perform inference detection in the existing unlabeled data and the newly collected unlabeled data to mine the required long-tail scenario data; screening the long-tail scenario data as the data to be labeled to expand the long-tail data scenario library. Through this solution, we can make full use of the vast amount of unlabeled data, mine and expand the local long-tail database, and then improve the detection performance of the perception model.

[0020] Obtaining detection results for the unlabeled data collected by the camera by setting reasonable parameters includes: obtaining detection results for the unlabeled data collected by the camera by setting reasonable nms thresholds and confidence thresholds.

[0021] Data augmentation processing includes rotation, translation, and flipping.

[0022] Semi-supervised learning can make full use of unlabeled data to improve the performance of the model when the labeled data is limited. After comparison, we choose the yolov5 detection network with swin transformer as the backbone to perform semi-supervised learning to obtain a benchmark large model. Using the benchmark large model and the existing deployed small model to perform inference comparison in the unlabeled data to obtain long-tail data, and finally expand the local long-tail data scenario library.

[0023] The above embodiments are only used to explain and illustrate the technical solutions of the present invention rather than to limit it. Those skilled in the art should understand that any modification and equivalent replacement without departing from the spirit and scope of the present invention shall fall within the protection scope of the claims of the present invention.

Claims

1. A method for autonomous driving scenario mining based on semi-supervised transformer detection, characterized in that, Including: Using existing datasets as labeled data, setting hyperparameters of the labeled data, and training 2 detection models respectively; Training a small model using existing datasets; For the unlabeled data collected by the camera, obtaining detection results by setting reasonable parameters; Taking the intersection of the detection results of the 2 detection models to obtain the ground truth of the unlabeled data, which is used as pseudo-labeled data; Performing data augmentation on the pseudo-labeled data to expand the pseudo-labeled dataset; Taking the labeled data and the pseudo-labeled dataset as the total data, setting the loss weight of the pseudo-labeled data, and retraining the yolov5 detection model with swin transformer as the backbone, which is called the student model; Using the small model and the student detection model to perform inference detection on the existing unlabeled data and the newly collected unlabeled data to mine the required long-tail scenario data; Screening the long-tail scenario data as the data to be labeled to expand the long-tail data scenario library.

2. The semi-supervised transformer detection-based autonomous driving scenario mining method according to claim 1, characterized in that For the unlabeled data collected by the camera, obtaining detection results by setting reasonable parameters, including: For the unlabeled data collected by the camera, obtaining detection results by setting reasonable nms thresholds and confidence thresholds.

3. A method for autonomous driving scenario mining based on semi-supervised transformer detection according to claim 1, characterized in that, The data augmentation process includes rotation, translation, and flipping.

Images