Vision-based deep imitation reinforcement learning driving strategy training method

Abstract

The invention discloses a vision-based deep imitation reinforcement learning driving strategy training method. The method comprises the steps of constructing an imitation learning network; training the imitation learning network; performing network splitting on the trained imitation learning network to obtain a sensing module; constructing a DDPG network to obtain a control module; completing theconstruction of a deep imitation reinforcement learning model through the sensing module and the control module; and training the deep imitation reinforcement learning model. An imitation learning network comprises five convolution layers and four full connection layers, the convolution layers are used for extracting features, and the full connection layers are used for predicting a steering angle, an accelerator and a brake opening degree; in addition, a reward function is set in the training process of the deep imitation reinforcement learning model, and comfort and safety of curve driving are guaranteed.

Description

technical field [0001] The invention relates to the technical field of automatic driving, in particular to a vision-based deep imitation reinforcement learning driving strategy training method. Background technique [0002] The rise of autonomous driving technology provides new solutions to existing traffic problems. Autonomous driving technology can effectively improve the driving efficiency of road motor vehicles, thereby alleviating traffic pressure. And by using the efficient and precise execution of the machine, traffic accidents are reduced and the driving safety index is improved. At the same time, the development of science and technology has promoted the rise of traffic intelligence. From computing power, traffic big data to popular deep learning, they have jointly promoted the rapid development of autonomous driving technology. [0003] In various tasks of autonomous driving, sensors such as radar, lidar, ultrasonic sensors and infrared cameras have been widely u...

AI Technical Summary

Problems solved by technology

[0006] The performance of the end-to-end model based on the end-to-end visual lane keeping method disclosed in Chinese patent document CN109446919A depends on the quantity and quality of the collected driving data. To obtain an excellent driving strategy, it is necessary to collect data of various driving scenarios, which consumes a lot of time. time

Secondly, because it is unrealistic to collect driving data for all scenarios, the model cannot handle unknown scenarios, and it is difficult to improve its performance in one step

[0007] Chinese patent document CN108897313A discloses a hierarchical end-to-end vehicle automatic driving system construction method. The first two layers of neural network models need to rely on a large amount of label data for training, and the second two layers of reinforcement learning model algorithms are still traditional reinforcement learning training methods. , requires a lot of exploration, and the training is very time-consuming

Due to its complex network structure, each network model needs to be trained separately, which further increases the complexity of training

[0008] In 2019, Berkeley proposed a model-free deep reinforcement learning algorithm in the article "Model-free Deep Reinforcement Learning for Urban Autonomous Driving" to simulate urban driving. The biggest problem with this model is that the deep reinforcement learning part is still a traditional learning method. In the huge exploration space of city simulation, it leads to low learning efficiency and limits its scope of use

[0009] The NVIDIA team proposed the DAVE2 network model for automatic driving in the article "End to end learning for self-driving cars". Its DAVE2 network model and the model proposed in the patent document CN109446919A have the same shortcomings and require a large amount of labeled data Train the network, and because it is difficult to collect full-scene driving data, it cannot handle other unknown scenarios

Images