Micro-decision-making method for self-driving vehicles based on reinforcement learning
A reinforcement learning and automatic driving technology, applied in neural learning methods, biological neural network models, control devices, etc., can solve the problem of difficult urban roads to show good decision-making performance, not well adapted to environmental dynamic changes, state space and Large behavior space, etc., to achieve the effect of strong universality and portability, easy deployment, and strong feasibility
A reinforcement learning and automatic driving technology, applied in neural learning methods, biological neural network models, control devices, etc., can solve the problem of difficult urban roads to show good decision-making performance, not well adapted to environmental dynamic changes, state space and Large behavior space, etc., to achieve the effect of strong universality and portability, easy deployment, and strong feasibility
CN111845773BActive Publication Date: 2021-10-26BEIJING UNIV OF POSTS & TELECOMM
Image
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0064] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.
[0065] Such as Figure 1 to Figure 4 As shown, the self-driving vehicle micro-decision-making method includes the following steps:
[0066] Step 1, reinforcement learning modeling, modeling and representation of automatic driving decision-making scheme:
[0067] In step 1.1, the driving process of the vehicle is defined as a Markov decision process. The autonomous vehicle is regarded as an agent, and the driving environment of the vehicle is regarded as a reinforcement learning environment. The agent vehicle makes driving decisions and Driving behavior, adjust the driving decision based on the driving results, divide the driving time into multiple time slots, each agent vehicle makes a driving decision at the beginning of the time slot, and determine the driving behavior of each agent vehicle in the time slot;
[0068] Step 1.2, use...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses a microcosmic decision-making method for an automatic driving vehicle based on reinforcement learning. The method adopts the A3C algorithm of reinforcement learning, the driving behavior is output by the Actor network, the flexibility is strong, and the complexity of the judgment logic is not affected by the size of the state space and the behavior space. The method employs a two-stage training solution process. In the first stage of training, a microcosmic decision-making model for automatic driving is obtained for all road sections to ensure driving safety. In the second stage, the overall model of the first stage is deployed to each road segment, and each road segment trains a single-segment model on this basis, which is portable. Meanwhile, the continuous training in the second stage enables the method to adapt to the influence of various real-time factors. Finally, the distributed communication architecture based on the real Internet of Vehicles system structure is described, which can complete the distributed calculation in the solution process. Therefore, the method can adapt to different road characteristics and dynamic driving environments, and has wide applicability and robustness. sex.
Description
technical field [0001] The invention relates to the technical field of automatic driving, in particular to a microscopic decision-making method for automatic driving vehicles based on reinforcement learning. Background technique [0002] Automated driving technology is one of the core technologies in intelligent transportation. Automated driving decisions are usually divided into two categories. One is the macroscopic path planning problem, that is, after the departure and destination of the vehicle are determined, the driving distance and congestion situation are comprehensively considered. Factors, how to choose the optimal driving route, this kind of problem has a relatively mature solution, another kind of problem is how to drive the vehicle on a certain micro road after the macro driving route is determined. [0003] In the prior art, the micro-decision-making models of self-driving vehicles are divided into the following categories: [0004] Finite state machine model...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
26 Oct 2021
Publication
CN111845773B
- IPC
- B60W50/00; B60W60/00; G06N3/04; G06N3/08
- CPC
- B60W50/00; B60W60/001; G06N3/08; B60W2050/0028; G06N3/047; G06N3/045
- Inventors
- ιδΎ; εζ°