Automatic driving vehicle microscopic decision-making method based on reinforcement learning

Abstract

The invention discloses an automatic driving vehicle microcosmic decision-making method based on reinforcement learning. According to the method, a reinforcement learning A3C algorithm is adopted, driving behaviors are output by an Actor network, the flexibility is high, and the complexity of logic judgment is not affected by state space and behavior space. According to the method, a two-stage training solving process is adopted. In the first stage, an automatic driving microcosmic decision model suitable for all road sections is obtained through training so as to guarantee driving safety. Inthe second stage, the overall model in the first stage is deployed to each road section, and each road section trains a single-road-section model on the basis of the overall model and has transportability. Meanwhile, the continuous training of the second stage enables the method to adapt to the influence of various real-time factors. Finally, distributed communication architecture based on a realInternet of Vehicles system structure is elaborated, and distributed calculation in the solving process can be completed, so that the method can adapt to different road features and dynamic driving environments, and has wide applicability and robustness.

Description

technical field [0001] The invention relates to the technical field of automatic driving, in particular to a microscopic decision-making method for automatic driving vehicles based on reinforcement learning. Background technique [0002] Automated driving technology is one of the core technologies in intelligent transportation. Automated driving decisions are usually divided into two categories. One is the macroscopic path planning problem, that is, after the departure and destination of the vehicle are determined, the driving distance and congestion situation are comprehensively considered. Factors, how to choose the optimal driving route, this kind of problem has a relatively mature solution, another kind of problem is how to drive the vehicle on a certain micro road after the macro driving route is determined. [0003] In the prior art, the micro-decision-making models of self-driving vehicles are divided into the following categories: [0004] Finite state machine model...

AI Technical Summary

Problems solved by technology

[0009] Both the finite state machine model and the decision tree model ignore the uncertainty of the environment and cannot adapt well to the dynamic changes of the environment. When more behavior patterns are defined, the state space and behavior space are large, and the judgment logic is complex. The feasibility is not high, and it is difficult to show good decision-making performance in urban roads with rich structural features

[0010] Both the finite state machine model and the decision tree model in the prior art ignore the uncertainty of the environment, and cannot adapt to the dynamic changes of the environment well, and when more behavior patterns are defined, the state space and behavior space are relatively large , the judgment logic is complex, the feasibility is not high, it is difficult to show better decision-making performance in urban roads with rich structural characteristics, and no effective solution has been proposed so far

Images