Automatic driving decision-making method and system based on partial observable migration reinforcement learning

Abstract

The invention discloses an automatic driving decision-making method and system based on partial observable migration reinforcement learning, and employs a scheme reuse method related to a scene, and achieves auxiliary solving of a driving problem under a strange road condition through a conventional scheme in a migration driving scheme database. In order to achieve good riding experience, reinforcement learning is used for solving a decision problem in the field of automatic driving. The system comprises a scene unit, a sensing unit, a decision-making unit, an action planning unit and a control unit. A new environment model is added to a virtual environment database to cope with increasingly complex driving scenarios; a convolution layer is added to the neural network to identify obstaclesaround the vehicle; the important historical information is memorized by adding a long-short-term memory unit into the neural network; a Q value is estimated more accurately by using a weighted depthdouble-Q network algorithm based on Boltzmann soft maximization; the probability that each driving scheme is selected is solved by using a maximum entropy Mellowmax algorithm.

Description

technical field [0001] The invention relates to an automatic driving decision-making method and system based on partially observable transfer reinforcement learning, which is applicable to partially observable driving environments and belongs to the technical field of automobile automatic driving. Background technique [0002] Autonomous driving needs to solve three problems: localization, path planning and selection of driving behavior. At present, the problem of "where am I" can be solved using a variety of sensor fusion technologies, and the problem of "how to go" can be solved using algorithms such as Dijkstra, A*, and dynamic programming. However, the choice of driving behavior, such as how to drive on city roads or how to ensure safety while driving at night, is still a research hotspot today. [0003] We regard the driving process of the car as a partially observable Markov decision process (Partially Observable Markov Decision Process, POMDP). As a result, the vehi...

AI Technical Summary

Problems solved by technology

However, due to the difference in the dynamical system between the target task and the source task, the prior knowledge obtained using a single source task cannot always perfectly solve the subproblems in the target task

Taking autonomous driving as an example, because of the differences in the number of lanes, traffic density, and speed limit between highways and urban roads, the performance of driving strategies based on urban roads may be unsatisfactory on highways.

Images