An automatic driving method based on conditional imitation learning

Abstract

The present application relates to a kind of automatic driving control method based on conditional imitation learning, belong to the field of automatic driving.The method is obtained by the laser radar, long focus camera and three front view cameras carried on the automatic driving vehicle environment perception data.Utilize the privilege information in Carla simulator to traffic light detection model is trained, whether the automatic driving vehicle needs to stop is determined by the perception of long focus camera.At the same time, using the perception information of laser radar and three front view cameras, automatic driving network is trained, and different network output branch is switched by traffic light signal and high-level steering command, vehicle control is realized.The experimental results show that, compared with prior art, the automatic driving method of the present application has the best vehicle control performance at urban intersection, and has the best driving score in Carla simulator after testing.

Description

Technical Field

[0001] This application relates to the fields of deep learning and autonomous driving, specifically a method for achieving end-to-end autonomous driving within a conditional imitation learning framework using imitation learning methods. Background Technology

[0002] In recent years, with the rapid development of artificial intelligence, deep learning technology has achieved tremendous success in the field of end-to-end autonomous driving. Given the high stability and logical consistency of computer operations, it is foreseeable that once autonomous driving technology is implemented, it will significantly reduce the number of traffic accidents currently caused by driver error. Furthermore, the accompanying vehicle-to-infrastructure interconnection technology will greatly alleviate traffic congestion through vehicle-to-cloud interaction. Therefore, major industrialized nations have introduced policies to facilitate the development of autonomous driving.

[0003] Currently, deep learning technology is widely used in the field of autonomous driving, and its invention patents are abundant, including patents related to object detection, object recognition, image semantic segmentation, road planning, route selection, safety decision-making, autonomous driving control, and autonomous navigation. The application of these patents has continuously improved the performance and safety of autonomous vehicles, laying a solid foundation for the development of autonomous driving technology.

[0004] Patent document CN114170488A discloses an autonomous driving method based on imitation learning and reinforcement learning. By combining conditional imitation learning and reinforcement learning, it reduces the model's dependence on the training dataset and solves the problem of low exploration efficiency in reinforcement learning caused by random initialization. By collecting real vehicle data and applying it to the model training, the trained model is made closer to real driving scenarios. This invention mainly addresses the problem of combining imitation learning and reinforcement learning methods. Patent document CN114358128A discloses an end-to-end method for training an autonomous driving strategy. This method also uses a combination of imitation learning and reinforcement learning. Supervised imitation learning is used for model pre-training to give the network a good initial weight before reinforcement learning is performed. Patent document CN115158336A proposes a two-stage imitation learning method for autonomous driving. This method decouples autonomous driving-related navigation modules, perception modules, control modules, etc., and trains the various modules required by the autonomous driving method separately through two stages. This not only reduces redundancy between end-to-end modules but also enhances the interpretability of the autonomous driving method.

[0005] However, the aforementioned algorithms in the field of autonomous driving do not focus on the safe driving problem in urban road environments that must be solved for autonomous vehicle applications. In particular, how should autonomous vehicles ensure that they can perceive changes in information about surrounding vehicles and traffic lights at densely populated intersections in urban roads, and how can they use neural networks to plan the best actions in different scenarios at intersections? Therefore, this patent proposes an autonomous driving method based on a conditional imitation learning architecture, which uses two high-level commands to design eight corresponding control branches to cope with the complex and diverse scenarios in urban roads. Summary of the Invention

[0006] To address the problem of insufficient model performance caused by autonomous vehicles running red lights in intersection environments, this invention proposes a deep learning autonomous driving algorithm that combines a conditional imitation learning architecture, a ResNet encoder, an MLP network, and an attention unit, as well as a traffic light detection algorithm for driving environments.

[0007] In the field of autonomous driving, because the loss function is often only related to the vehicle's control output, updating the network using the loss function tends to highlight features in the environment related to vehicle control. However, if only one perception module is used for autonomous driving perception, the network often ignores traffic light perception because traffic light signals occupy very few image pixels. This leads to vehicles failing to accurately recognize traffic lights at intersections, resulting in vehicles stopping indefinitely or running red lights. Therefore, to address the challenges faced by autonomous driving algorithms in urban intersection environments, we improve the conditional imitation learning architecture. Specifically, the high-level commands of the conditional imitation learning architecture consist of traffic light commands (including red light and others) and steering commands (including lane following, left turn, right turn, and straight ahead), giving the autonomous driving network eight control branches to better handle autonomous driving problems in intersections.

[0008] During the training of the autonomous driving algorithm, traffic light commands are provided by the Carla simulator. During the testing of the autonomous driving algorithm, traffic light commands are replaced by our trained traffic light detection model. Therefore, our autonomous driving framework consists of both the autonomous driving algorithm and the traffic light detection algorithm.

[0009] Our proposed traffic light detection network utilizes a ResNet encoder to extract RGB telephoto image features from three consecutive time steps, obtaining feature J. We then employ a self-attention mechanism to focus the network on regions within the image features relevant to the traffic light output, resulting in feature Z.

[0010]

[0011]

[0012] Among them, I tel The image was taken with a telephoto camera. These are the weight matrices for query, key, and value in the self-attention mechanism, d m Set it to 128.

[0013] The final step in the traffic light detection network is to use the sigmoid function to output whether the current traffic light signal requires the car to stop, and to use a manually set threshold to customize and adjust the safety redundancy.

[0014] D = ω(sigmoid(Z))

[0015] Here, ω is a manually set stopping threshold. When the output of the traffic light detection network is greater than 0.65, it is determined that a red light has been detected and a stop is required; otherwise, normal driving continues. D equals 0 or 1. We use a binary cross-entropy loss function to train the traffic light signal detection network.

[0016] Furthermore, during the traffic light signal detection network development process, we implemented various weather conditions within the urban environment provided by Carla to augment the data, improving the robustness of the network and preventing overfitting. Simultaneously, we implemented different traffic light signals and vehicle control behaviors within the same urban environment to ensure the network learned the correct mapping relationships.

[0017] The specific details of the traffic light detection network are as follows: First, ResNet-18 is used as the feature extraction module to extract the features J of the telephoto camera image (where the input size of the telephoto camera image is 300*400) at three consecutive time steps. Next, a 128-dimensional representation Z is generated through a self-attention module, and the output D is obtained using a classification layer with a sigmoid activation function.

[0018] Our proposed autonomous driving network incorporates LiDAR into the perception input to generate depth information L, and installs three cameras (Ii, Id, Ii, Id) on the autonomous vehicle. f I f l,I fr The left and right front-view angles were shifted 45° to the left and right respectively to perceive a wider range of complex environments, addressing the challenges of complex urban intersections. During the preprocessing of the perception, the 3D spatial coordinates of the laser point cloud were mapped to the 2D pixel coordinates of the image through a 3D-2D coordinate transformation to generate a depth map. A 170° forward field of view was selected as the perceived depth information. Simultaneously, to eliminate edge distortion of the image and ensure consistent perception range across the membrane sensor, the IL at the same time step t was... f I fl I frThree images are stitched together to cover a 170° field of view in front. Then, we use a feature extraction network to extract high-dimensional features from the image and depth map separately, and fuse them through an MLP network to obtain K. At the same time, we augment the input steering and velocity information into the high-dimensional features to obtain H.

[0019] Because human drivers pay more attention to environmental changes to the left or right of the vehicle when turning left or right, we utilize an attention mechanism. An attention map generated from steering and speed features H is multiplied by a V map generated from environmental perception features K, allowing the model to focus on different areas of the environment under different steering patterns.

[0020]

[0021] Where P is the perceptual feature generated through the attention mechanism. These are the weight matrices for query, key, and value in the attention mechanism, d n Set it to 64.

[0022] Finally, the autonomous driving network generates control commands by inputting the perceived features P into different control branches based on the traffic light command D and the turn command S.

[0023] a = M i (Branch(D, S), P), i∈(1, 2, 3…8)

[0024] Among them, M i The selected control branch is a = (s, r), which controls the steering and throttle outputs.

[0025] The overall learning objective of our proposed autonomous driving algorithm is:

[0026]

[0027] The loss function for the autonomous driving task is set as follows:

[0028] l(a,a label )=l <s,s label >, <r,r label >)=λ||ss label || 2 +||rr label || 2

[0029] Similar to the traffic light detection dataset, our autonomous driving dataset also features data augmentation with various weather conditions in the Carla environment to improve the robustness of environmental perception.

[0030] The specific details of the autonomous driving network are as follows: ResNet-18 and ResNet-34 are used to extract depth map and camera image features from three consecutive time steps (the depth map input size is 64×490, and the camera image input size is 300×640). Then, a two-layer MLP network is used to fuse multimodal features to obtain K (where the MLP network contains 1024 and 256 units respectively), generating a high-dimensional environment representation. Next, the velocity and steering information from the three consecutive time steps are combined and passed through a linear layer with 128 units to increase the dimensionality to obtain H. Then, K and H are input into the attention layer, where the query and key generated by H are used to generate attention weights, which are multiplied by the value generated by K to obtain a 64-dimensional perceptual representation P. Finally, the perceptual representation is input into the control branch to produce control commands a = (s, r).

[0031] Based on the methods and details described above, we constructed a traffic light detection network and an autonomous driving network, respectively, and trained them on the Town01 Carla simulator. Since our autonomous driving algorithm employs multimodal perception using cameras and radar, we compared our method with Late Fusion, Transfuser, and LAV. We selected a road with intersections in Town02 of the Carla simulator for testing.

[0032] Table 1 shows the performance of each model after three tests on the specified road section of Town02, where RC represents the route completion rate, DS represents the driving score, and Red represents red light violations.

[0033]

[0034] Table 1 shows the performance of different models at intersections. The data in Table 1 shows that the present invention has the highest path completion rate and driving score, and achieves safe control output with no red light violations.

[0035] We then conducted ablation experiments on the proposed autonomous driving model. (No attention) means we removed the attention mechanism from the autonomous driving model; (No traffic lights) means we removed the traffic light detection signal for high-level commands; and (Base) means we removed both of the above. Keeping the rest of the model framework unchanged, we retrained an end-to-end autonomous driving model and compared it with an autonomous driving algorithm that includes a traffic light detection model.

[0036] Table 2 Ablation Experiment

[0037]

[0038] As shown in Table 2, the RC (Responsiveness Control) performance of the model including traffic light detection is not significantly different from that of the model without traffic light detection. However, the DS (Discretionary Disorder) performance of the model without traffic light detection drops significantly, and traffic light violations also occur in this case. This indicates that our designed traffic light detection model has a significant effect on improving the safety performance of intersections. Furthermore, after removing the attention mechanism from the model, both RC and DS performance decrease to some extent, indicating that the attention mechanism we introduced has a positive effect on improving the performance of autonomous driving.

[0039] In addition, we also tested a traffic light detection network.

[0040] Table 3 Traffic Light Detection Network Test

[0041]

[0042] The reason for red light detection errors in the traffic light detection network test is that when the traffic light is too far from the car, the pixel size of the traffic light in the input telephoto image is too small, causing the model to fail to detect the red light. Other situations that lead to detection errors occur when vehicles are queuing at a traffic light, in which case the label of the traffic light in the dataset is 0, but the input telephoto camera can detect that the current traffic light is red. Therefore, the model outputs 1, resulting in a false positive detection. Attached Figure Description

[0043] Figure 1 This is a diagram of the traffic light detection network structure proposed in this invention.

[0044] Figure 2 This is the network structure diagram of the autonomous driving system proposed in this invention.

[0045] Figure 3 This is a summary drawing of the present invention, which summarizes the autonomous driving network structure proposed in this invention.

Claims

1. A deep learning-based autonomous driving control algorithm, characterized in that... Includes the following steps: Collect simulated autonomous driving datasets and traffic light perception datasets using the Carla simulator; A traffic light detection network was trained based on a traffic light perception dataset to detect whether there is a red light ahead of an autonomous vehicle. The autonomous driving network based on conditional imitation learning is trained using an autonomous driving dataset. Privilege information of traffic lights and steering commands in the Carla simulator are used as high-level commands to generate control commands for the accelerator and steering of the autonomous vehicle during training. During testing, a trained traffic light detection network was used to replace the privileged information of the traffic lights.

2. The deep learning-based autonomous driving control algorithm according to claim 1, characterized in that, The process of installing specified sensors on an expert model using Carla simulation software and collecting data along a predetermined route includes: Install a LiDAR, three RGB cameras, and an IMU sensor on the expert model to collect autonomous driving datasets and traffic light perception datasets. Three RGB cameras were placed at the front, left front, and right front of the car, respectively, with the left front and right front cameras offset by 45 degrees. The RGB images generated from the three perspectives were stitched together and preprocessed to form a front-view RGB image with a wider field of view, and the data was saved. An RGB telephoto camera was placed directly in front of the vehicle to generate an RGB image of the front of the vehicle, and the data was saved. The raw point cloud generated by the lidar is transformed into a depth map in image coordinates through 3D-2D coordinate transformation and the data is saved. When collecting autonomous driving datasets, the privileged information of the Carla simulator is used to collect traffic light information related to the vehicle in the environment, advanced steering information in path planning, and throttle and steering control actions output by the expert model at the current moment, and the data is saved. When collecting traffic light perception datasets, the privileged information of the traffic lights is also used to collect traffic light information related to vehicles in the environment and save the data.

3. The deep learning-based autonomous driving control algorithm according to claim 2, characterized in that, Advanced steering information is provided in path planning, consisting of four instructions: left, right, forward, and lane following, which controls the driving direction of the autonomous vehicle.

4. The deep learning-based autonomous driving control algorithm according to claim 1, characterized in that, The network uses an RGB telephoto camera as input to sense traffic light signals in the environment and outputs whether a red light is currently present. The network features include: Use a residual network to extract RGB telephoto image features from three consecutive time steps in the past; The high-dimensional features from the three time steps are concatenated and input into a linear network to generate environmental perception features for the traffic light detection network. The self-attention mechanism is used to make the network pay more attention to environmental features related to traffic lights; Based on this, the possibility of a red light existing in the current environment is output through the Sigmoid activation function; Based on a manually set threshold, if the network output is greater than 0.65, it is determined that a red light is detected in the current environment, and the network output is set to 1; if the network output is less than or equal to 0.65, it is determined that no red light is detected in the current environment, and the network output is set to 0.

5. The deep learning-based autonomous driving control algorithm according to claim 1, characterized in that, By utilizing both LiDAR and RGB images to jointly perceive environmental features, and using traffic light signals and turn commands as high-level commands, the environmental features are input into different control branches to generate different control outputs, including: By using two different residual networks to extract preprocessed LiDAR depth maps and stitched RGB images from three consecutive time steps, the two perception features are fused through a two-layer fully connected neural network to obtain high-dimensional environmental features.

6. The deep learning-based autonomous driving control algorithm according to claim 5, characterized in that, A linear layer containing 128 units is used to upscale the steering and speed information of the autonomous vehicle in the past time step. Then, an attention mechanism is used to generate an attention map containing steering and speed information, which is multiplied with the value vector generated by the environment representation to obtain the autonomous driving network's representation of the environment. Based on traffic light signals and turn commands as high-level commands, eight control branches are set up. In different driving situations, environmental representations are input into different control branches, and different control actions are output to cope with different driving environments.

7. The deep learning-based autonomous driving control algorithm according to claim 5, characterized in that, The loss functions of deep learning models include: The mean squared error loss function is used to calculate and update the throttle and steering separately; Since the steering command changes relatively little, a fixed scaling factor is also added to the steering loss function.

8. The deep learning-based autonomous driving control algorithm according to claim 4, characterized in that, The loss functions of deep learning models include: The error between the network output and the true label is calculated using the binary cross-entropy loss function, and this error is used to update the network model.

Images