Vehicle reinforcement learning motion planning method based on driving risk analysis

Abstract

The invention discloses a vehicle reinforcement learning motion planning method based on driving risk analysis. The method comprises the following steps: acquiring boundary information on two sides of a lane where a vehicle is located, wherein the boundary information consists of horizontal and vertical coordinates (Xr, Xl) of boundary points on two sides of the lane and vertical distances (dl, dr) from the vehicle to boundaries on the left and right sides; a high-precision map and a radar are used for obtaining a state information array which is composed of the coordinate position (X, Y) of a vehicle, the coordinate position (Xobs, Yobs) of an obstacle and the relative speed delta v; the distance delta v is equal to [delta v1, delta v2,..., delta vn], and n is the number of detected obstacles; the sum is combined into state vector relative state information, the state vector relative state information is input into a vehicle motion planning model phi based on deep reinforcement learning, so that a vehicle motion vector action = [a, theta] is output, a belongs to [-amax, amax] is an acceleration action, amax is the maximum acceleration, theta belongs to [-theta max, theta max] is a steering wheel turning action, when theta belongs to [0, theta max], theta is turned leftwards, and when theta belongs to [-theta max, 0], theta is turned rightwards; after the vehicle runs for t duration according to the obtained action, whether the vehicle arrives at the destination or not is judged, if yes, the working state is ended, and if not, the step 1 is executed for vehicle control of the next time step length.

Description

technical field [0001] The invention belongs to the field of unmanned vehicle motion planning, and more particularly relates to a vehicle reinforcement learning motion planning method based on driving risk analysis. Background technique [0002] The motion planning module is an important technical link of unmanned driving. In the unmanned driving technology chain, it is connected to the environment perception module and the execution control module. It is the intelligent command center of the unmanned vehicle. In the past, the research on unmanned motion planning mainly used rule-based methods, supervised learning methods and optimization methods. The rule-based method can only model parameters for a simple specific driving environment based on certain assumptions, such as following a car and changing lanes. , obstacle avoidance and other specific scenarios, but it is not suitable for complex and changeable urban driving scenarios; the supervised learning method needs to col...

AI Technical Summary

Problems solved by technology

At present, the research on motion planning based on deep reinforcement learning mostly adopts end-to-end training strategies. The black-box characteristics of the neural network make the model output action lack of interpretability, and the model mostly uses sparse rewards for strategy optimization, making it difficult for the model to converge and requires a lot of effort. A lot of time for trial and error training

Images