Partially observed intersection autonomous merging method based on particle attention deep q-learning

Abstract

The invention discloses a partly observed intersection autonomous merging method based on particle attention depth Q learning, which focuses on intersection scenes, vehicle merging tasks, and partial observation conditions where the observation field of view is blocked by buildings and other vehicles. The deep Q-learning algorithm optimizes the driving behavior of merging vehicles on a given route. Use low-dimensional physical information as the observation representation of the vehicle; use particle-based representation to deal with partial observation problems caused by occlusion; optimize the state representation by introducing an attention mechanism, so that the model can only accept vehicle information that is not occluded and have input Arrangement invariance; use the deep Q-learning algorithm to output the current optimal driving action according to the acquired social vehicle information; add sampling data under various traffic densities to the experience playback pool, combined with priority experience playback technology, to make autonomous merging behavior It can adapt to the changing traffic density in the real environment.

Description

technical field [0001] The present invention relates to a partial observation intersection autonomous merging method based on particle attention depth Q learning, using particle-based representation to deal with partial observations caused by the occlusion of the field of view during driving, and using an attention-based mechanism to optimize the state representation The deep Q-learning algorithm optimizes driving behavior and belongs to the technical field of automobile automatic driving. Background technique [0002] Autonomous driving needs to solve three problems: localization, path planning and selection of driving behavior. The problem of the first type of positioning can be processed using various sensor fusion technologies and increasingly mature computer vision technology; the problem of the second type of path planning can be processed using Dijkstra, A* or some other dynamic programming methods; the present invention handles It is the third type of problem, and c...

AI Technical Summary

Problems solved by technology

There are two problems designed here: one is what to use as an observation, and the other is how to better restore the real environmental state from the observation or let the self-vehicle realize that the observation is insufficient, and it is necessary to be alert to the possible existence of Risk (visual blind zone)

The problem of the RNN-based method is mainly in the instability of its convergence performance, there are more difficulties in optimization, and the meaning of the hidden state is unknown and lacks interpretation.

Images