Micro-decision-making method for self-driving vehicles based on reinforcement learning

Abstract

The invention discloses a microcosmic decision-making method for an automatic driving vehicle based on reinforcement learning. The method adopts the A3C algorithm of reinforcement learning, the driving behavior is output by the Actor network, the flexibility is strong, and the complexity of the judgment logic is not affected by the size of the state space and the behavior space. The method employs a two-stage training solution process. In the first stage of training, a microcosmic decision-making model for automatic driving is obtained for all road sections to ensure driving safety. In the second stage, the overall model of the first stage is deployed to each road segment, and each road segment trains a single-segment model on this basis, which is portable. Meanwhile, the continuous training in the second stage enables the method to adapt to the influence of various real-time factors. Finally, the distributed communication architecture based on the real Internet of Vehicles system structure is described, which can complete the distributed calculation in the solution process. Therefore, the method can adapt to different road characteristics and dynamic driving environments, and has wide applicability and robustness. sex.

Description

technical field [0001] The invention relates to the technical field of automatic driving, in particular to a microscopic decision-making method for automatic driving vehicles based on reinforcement learning. Background technique [0002] Automated driving technology is one of the core technologies in intelligent transportation. Automated driving decisions are usually divided into two categories. One is the macroscopic path planning problem, that is, after the departure and destination of the vehicle are determined, the driving distance and congestion situation are comprehensively considered. Factors, how to choose the optimal driving route, this kind of problem has a relatively mature solution, another kind of problem is how to drive the vehicle on a certain micro road after the macro driving route is determined. [0003] In the prior art, the micro-decision-making models of self-driving vehicles are divided into the following categories: [0004] Finite state machine model...

AI Technical Summary

Problems solved by technology

[0009] Both the finite state machine model and the decision tree model ignore the uncertainty of the environment and cannot adapt well to the dynamic changes of the environment. When more behavior patterns are defined, the state space and behavior space are large, and the judgment logic is complex. The feasibility is not high, and it is difficult to show good decision-making performance in urban roads with rich structural features

[0010] Both the finite state machine model and the decision tree model in the prior art ignore the uncertainty of the environment, and cannot adapt to the dynamic changes of the environment well, and when more behavior patterns are defined, the state space and behavior space are relatively large , the judgment logic is complex, the feasibility is not high, it is difficult to show better decision-making performance in urban roads with rich structural characteristics, and no effective solution has been proposed so far

Images