Offline RL Trajectory Planning for Open-Space Autonomous Vehicles
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional motion planning for autonomous driving vehicles is inadequate as it fails to accurately plan trajectories in open spaces without reference lanes, leading to potential collisions and inefficiencies, as it does not consider the unique features and constraints of different vehicle types.
Innovation Solution
An offline reinforcement learning agent is employed to generate trajectories for autonomous driving vehicles in open spaces by using quadratic programming optimizations and actor-critic reinforcement learning models, which perceive the environment, determine target functions, and iteratively optimize trajectories to avoid obstacles and reach destinations safely and efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional motion planning operations are used that estimate difficulty from curvature and speed, then the planning process is simple and fast, but the trajectory planning accuracy is insufficient for different vehicle types and open space scenarios
Solution Approach 1:
The patent transforms the motion planning problem from traditional curvature and speed estimation into a reinforcement learning parameter optimization problem. The system learns optimal trajectories by adjusting parameters such as position, velocity, acceleration, and steering angle through iterative training with reward functions that evaluate planning accuracy, thereby achieving higher precision for different vehicle types without manually encoding complex planning rules
Solution Approach 2:
The patent replaces conventional deterministic motion planning algorithms with a data-driven reinforcement learning approach. Instead of using fixed mathematical models based on curvature and speed, the system employs neural networks trained through interaction with simulated environments to learn adaptive motion planning policies that generalize across different vehicle types and open space scenarios
2Adaptability or versatility
If trajectories are planned based on pre-labeled reference lanes in HD maps, then the navigation follows predefined paths, but the system cannot handle open space scenarios without reference lanes
Solution Approach 1:
The reinforcement learning agent is designed to perform multiple functions: it can plan trajectories both in conventional scenarios with reference lanes and in open space scenarios without reference lanes. The same learned policy generalizes across different environment types, making the system universally applicable to various navigation contexts while maintaining reliability through continuous evaluation against safety constraints
Solution Approach 2:
The system performs preliminary learning and training in simulated environments that include diverse scenarios such as open spaces, intersections, and crowded areas. This preliminary action in simulation allows the agent to acquire robust motion planning skills before deployment, ensuring it can handle open space scenarios without reference lanes while maintaining collision avoidance capabilities
3Measurement precision
If the same motion planning and control is applied to all vehicle types, then the system is simple to implement, but the trajectories are not accurate and smooth for specific vehicle dynamics
Solution Approach 1:
The reinforcement learning approach learns vehicle-specific parameters and dynamics characteristics during training. By adjusting the simulation environment to match different vehicle types (e.g., cars, trucks, buses) and training the agent with vehicle-specific parameters such as mass, dimensions, and dynamic constraints, the system generates smooth and accurate trajectories tailored to each vehicle type without requiring manual tuning of planning algorithms
Data Source
AI summary
In one embodiment, a system generates a plurality of driving scenarios to train a reinforcement learning (RL) agent and replays each of the driving scenarios to train the RL agent by: applying a RL algorithm to an initial state of a driving scenario to determine a number of control actions from a number of discretized control/action options for the ADV to advance to a number of trajectory states which are based on a number of discretized trajectory state options, determining a reward prediction by the RL algorithm for each of the controls/actions, determining a judgment score for the trajectory states, and updating the RL agent based on the judgment score.