Expressway unmanned vehicle formation method based on multi-agent reinforcement learning

Abstract

The invention provides an expressway unmanned vehicle formation method based on multi-agent reinforcement learning, the method regards a vehicle formation problem as a multi-agent cooperation problem, and each vehicle has an independent decision-making capability. The method can achieve flexible formation on the premise of safe and rapid driving, namely, safe obstacle avoidance is achieved when the traffic flow is large while the formation does not need to be kept, and the formation is recovered when the traffic flow is small; an end-to-end mode of directly mapping image input to vehicle control quantity is large in training difficulty due to large action search space, so that a lane changing strategy is learned only by using a multi-agent reinforcement learning method, the accurate control quantity is calculated in combination with an S-T graph trajectory optimization method, so that control constraints are increased, the kinematics principle is respected, safety guarantee is achieved, and human driving habits are met.

Description

technical field [0001] The invention belongs to the technical field of intelligent vehicles, and in particular relates to a formation method for unmanned vehicles on expressways based on multi-agent reinforcement learning. Background technique [0002] Autonomous Vehicles (Autonomous Vehicles) have been researched for decades. They can replace humans to complete tedious operations in complex scenarios such as high-density, long-cycle, and large-flow, and have high social and economic value. Expressway has the characteristics of clear topology, known traffic rules, clear restrictions, and relatively closed. It is a typical scenario for the implementation of autonomous driving. Among them, the formation of intelligent logistics vehicles is a key problem worthy of research, which plays an important role in reducing fuel consumption, improving fleet operating efficiency, and reducing traffic congestion. However, there are still many problems for formation tasks on high-speed st...

AI Technical Summary

Problems solved by technology

First of all, the dynamic vehicle movement state on the expressway is complex, and it is difficult for the vehicle formation to coordinate; secondly, the vehicle perception is limited, and the stability of the formation system is poor; third, the fixed formation mode makes the system less flexible and has a greater impact on surrounding vehicles

[0003] The formation method based on traditional control requires complex controller design, and the system-level control method has high requirements for the stability of a single vehicle. If a vehicle fails during formation driving, the control program needs to be changed manually. In high-speed road scenarios, fixed control modes will also lose system flexibility and adaptability to environmental changes

Reinforcement learning is machine learning. With the development of artificial intelligence and machine learning, reinforcement learning has gradually been applied to autonomous driving tasks, but it is usually aimed at single-vehicle intelligence, and the advantages of reinforcement learning in the field of multi-agents have not been fully exploited.

Images