Reinforcement Learning Agent for Autonomous Vehicle Trajectory Planning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional motion planning for autonomous driving vehicles is inadequate in open spaces without reference lanes, as it fails to accurately plan trajectories that avoid collisions and are smooth, due to the lack of consideration for vehicle-specific features and obstacles.
Innovation Solution
An online agent using reinforcement learning is employed to plan trajectories for autonomous driving vehicles in open spaces by perceiving the environment, determining target functions based on obstacles and map information, and iteratively performing quadratic programming optimizations to generate smooth and collision-free paths.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If conventional motion planning is used based on curvature and speed, then the planning process is simple, but the trajectory accuracy and smoothness deteriorate because vehicle-specific features are not considered
Solution Approach 1:
The patent applies local quality by creating vehicle-specific motion planning models that are tailored to each vehicle type's unique characteristics (dimensions, mass, inertia, etc.). Instead of using a universal planning approach, the system generates customized planning algorithms for each vehicle, improving trajectory accuracy while maintaining manageable complexity through automated model generation.
Solution Approach 2:
The patent utilizes parameter changes by incorporating vehicle-specific parameters (mass, dimensions, moment of inertia, wheelbase) into the motion planning process. These parameters are used to generate vehicle-specific models that adapt the planning algorithm to each vehicle's physical characteristics, thereby improving trajectory precision without requiring overly complex manual tuning.
2Adaptability or versatility
If trajectories are planned based on pre-labeled reference lanes, then the planning is straightforward, but the applicability to open space scenarios deteriorates
Solution Approach 1:
The patent applies dynamics by transitioning from static reference lane-based planning to dynamic motion planning that adapts to real-time environmental conditions. The system generates trajectories based on current vehicle state, obstacle positions, and environmental constraints, enabling operation in open spaces without pre-labeled lanes while maintaining planning effectiveness through automated model generation.
Solution Approach 2:
The patent achieves universality by creating a motion planning system that can handle multiple scenario types (open spaces, roads with lanes, parking lots) through a unified vehicle-specific modeling approach. The same planning framework adapts to different environments by incorporating relevant constraints and parameters, eliminating the need for separate planning systems for each scenario type.
3Manufacturing precision
If same motion planning is applied to all vehicle types, then the system is simple to implement, but the trajectory smoothness and accuracy deteriorate under certain circumstances
Solution Approach 1:
The patent applies local quality by generating customized motion planning models for each vehicle type based on its specific physical characteristics. Each vehicle receives a tailored planning algorithm that accounts for its unique dimensions, mass, and dynamic properties, thereby improving trajectory precision while the automated generation process keeps implementation complexity manageable.
Solution Approach 2:
The patent uses preliminary action by pre-generating vehicle-specific motion planning models during an offline training phase. These pre-computed models incorporate each vehicle's specific parameters and are ready for deployment, eliminating the need for complex real-time calculations and reducing implementation complexity while maintaining high planning precision.
Data Source
AI summary
In one embodiment, a system uses an actor-critic reinforcement learning model to generate a trajectory for an autonomous driving vehicle (ADV) in an open space. The system perceives an environment surrounding an ADV. The system applies a RL algorithm to an initial state of a planning trajectory based on the perceived environment to determine a plurality of controls for the ADV to advance to a plurality of trajectory states based on map and vehicle control information for the ADV. The system determines a reward prediction by the RL algorithm for each of the plurality of controls in view of a target destination state. The system generates a first trajectory from the trajectory states by maximizing the reward predictions to control the ADV autonomously according to the first trajectory.