A laser radar target detection method based on a transformer

Abstract

This invention provides a LiDAR target detection method based on Transformer, relating to the fields of intelligent driving, computer technology, and deep learning. The method includes the following steps: acquiring LiDAR point cloud data and camera data at the same timestamp, and preprocessing the data using a preprocessing module; labeling the target to be detected in the point cloud data based on the camera data, and dividing the processed point cloud data into a test set and a training set; inputting the training set point cloud data into a multi-scale Transformer model, using the labeled data as the target, and performing unified regression training on the model; inputting the test set point cloud data into the trained multi-scale Transformer model for target recognition. This invention effectively improves the reliability and efficiency of intelligent driving perception software and can optionally reduce the cost of intelligent driving hardware.

Description

Technical Field

[0001] This invention relates to a LiDAR target detection method based on Transformer, belonging to the fields of intelligent driving, computer technology, and deep learning technology. Background Technology

[0002] Perception is crucial in intelligent driving technology, characterized by a wealth of sensing devices, diverse sensing data modalities, massive data volumes, and complex task outputs. Deep learning techniques have been widely applied in areas such as computer vision, natural language processing, and recommendation systems. The successful application of these machine learning techniques relies on a foundation of massive amounts of data. However, in real-world applications, a large number of sensing devices inevitably increases terminal costs, making intelligent driving technology more difficult to implement. Using a single sensing device to implement intelligent driving technology is becoming a trend. LiDAR, a star sensing device, has established itself in intelligent driving perception devices due to its advantages of high sensing accuracy, high recognition accuracy, more direct data, less burden on intelligent driving system chips and algorithms, and greater safety and reliability.

[0003] Deep learning is a machine learning algorithm based on learning representations of data. The advantage of deep learning is that it replaces manual feature acquisition with efficient algorithms for unsupervised or semi-supervised feature learning and hierarchical feature extraction. Several deep learning frameworks, such as deep neural networks, convolutional neural networks, deep belief networks, and recurrent neural networks, have been applied in computer vision, speech recognition, natural language processing, audio recognition, and bioinformatics with excellent results. Currently, there is a lack of a deep learning-based technique that provides a perception model for target detection in the field of autonomous driving, effectively improving the reliability and efficiency of autonomous driving perception software and optionally reducing the cost of autonomous driving hardware. Summary of the Invention

[0004] The purpose of this invention is to provide a Transformer-based LiDAR target detection method that effectively improves the reliability and efficiency of intelligent driving perception software and can optionally reduce the cost of intelligent driving hardware.

[0005] To achieve the above objectives, the present invention employs the following technical solution:

[0006] Step 1: Collect LiDAR point cloud data and camera data at the same timestamp, and preprocess the data through the preprocessing module; based on the camera data, mark the target to be detected in the point cloud data, and divide the processed point cloud data into test set and training set;

[0007] Step 2: Input the training set point cloud data into the multi-scale transformer model, use the label data as the target, and perform unified regression training on the model;

[0008] The multi-scale Transformer model includes an input layer, a hidden layer, a fully connected layer, and an output layer. The hidden layer includes a convolutional layer and a pooling layer, and the output layer has three layers.

[0009] Step 3: Input the test set point cloud data into the trained multi-scale transformer model for target recognition.

[0010] Preferably, the backbone network of the multi-scale transformer model is constructed using TransUnit as the basic module.

[0011] Preferably, the TransUnit consists of two Transformers and a one-dimensional convolutional block, and uses skip connections, which are shallow inputs added to the middle layers of the deep network.

[0012] Preferably, the backbone network includes 27 TransUnits, and three branches branch off at the 11th, 19th and 27th TransUnits for detecting targets of different sizes.

[0013] Preferably, the multi-scale Transformer model detects large objects using small-sized feature maps and small objects using large-sized feature maps.

[0014] Preferably, the preprocessing module uses a multilayer perceptron to preprocess the point cloud data and uses the output features as input to a multiscale transformer model.

[0015] The advantages of this invention are: it effectively improves the reliability and efficiency of intelligent driving perception software and can selectively reduce the cost of intelligent driving hardware. Unlike traditional camera recognition methods, this invention reduces the number of intelligent driving terminal hardware devices and lowers the cost of intelligent driving hardware. Attached Figure Description

[0016] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0017] Figure 1 This is a schematic diagram of the jump-layer connection structure of the present invention.

[0018] Figure 2 It is a multi-scale Transformer model.

[0019] Figure 3This is the point cloud preprocessing module. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Step 1: Collect LiDAR point cloud and camera data at the same timestamp, and preprocess the data through the preprocessing module; based on the camera data, label the target to be detected in the point cloud data, and divide the processed point cloud data into test set and training set; the preprocessing module uses multilayer perceptron to preprocess the point cloud data, and uses the output features as input to the multiscale transformer model.

[0022] Step 2: Input the training set point cloud data into the multi-scale transformer model, use the label data as the target, and perform unified regression training of the model.

[0023] The multi-scale Transformer model comprises an input layer, hidden layers, fully connected layers, and an output layer. The hidden layers include convolutional layers and pooling layers, and there are three output layers. The backbone network of the multi-scale Transformer model is constructed using TransUnits as basic modules. Each TransUnit consists of two Transformers and a one-dimensional convolutional block, and uses skip connections, which are shallow inputs added to intermediate layers of the deep network. The backbone network includes 27 TransUnits, and three branches branch off at TransUnits 11, 19, and 27 for detecting targets of different sizes.

[0024] Step 3: Input the test set point cloud data into the trained multi-scale transformer model for target recognition.

[0025] The multi-scale Transformer model detects large objects using small-sized feature maps and small objects using large-sized feature maps.

[0026] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for Transform-based LiDAR target detection, characterized in that, The method comprises the following steps: Step 1: Collect laser radar point cloud and camera data with the same time stamp, and pre-process the data through a pre-processing module; according to the camera data, mark the target to be detected in the point cloud data, and divide the processed point cloud data into a test set and a training set; Step 2: input the training set point cloud data into a multi-scale transformer model, take the label data as the target, and perform unified regression training of the model; The multi-scale Transformer model comprises an input layer, a hidden layer, a full connection layer and an output layer, the hidden layer comprises a convolution layer and a pooling layer, and the output layer has three; Step 3: input the test set point cloud data into the trained multi-scale transformer model to perform target identification; The main network of the multi-scale transformer model is composed of TransUnit as a basic module; The TransUnit is composed of two Transformers and a one-dimensional convolution block, and a skip connection is used, wherein the skip connection is to add the input of a shallow layer to an intermediate layer of a deep network; The main network comprises 27 TransUnit, and three branches are divided at the 11th, 19th and 27th TransUnit for detecting targets of different sizes.

2. The Transform er-based lidar target detection method of claim 1, wherein, The multi-scale Transformer model detects large-size objects through small-size feature maps and detects small-size objects through large-size feature maps.

3. The Transformer-based lidar target detection method of claim 1, wherein, The pre-processing module uses a multi-layer perception to pre-process the point cloud data, and takes the output features as the input of the multi-scale transformer model.

Images