Navigation Trajectory Using Reinforcement Learning for Ego Vehicle
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing in-vehicle trajectory planning systems face challenges in efficiently determining navigation trajectories for ego vehicles due to large state spaces and the need for numerous parameters and datasets, especially when considering visual information and granular actions like steering and acceleration.
Innovation Solution
A navigation network system utilizing reinforcement learning, where decider modules generate trajectory decisions, a grader module provides rewards, and a resolver module selects the final trajectory based on a three-dimensional state space, incorporating current and future vehicle locations, and external server computers for faster training and decision-making.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a single trajectory planning system uses all available visual information and granular output actions, then the navigation accuracy is improved, but the training time and computational resources increase significantly
Solution Approach 1:
The patent segments the trajectory planning system into multiple decider modules, each responsible for generating specific trajectory decisions (e.g., lane changes, turns, straight movement). This segmentation allows the system to process navigation tasks in discrete, manageable units rather than treating all visual information and actions as a single complex problem, thereby reducing training time while maintaining navigation accuracy.
Solution Approach 2:
The patent transforms the planning horizon into a temporal dimension by generating multiple future two-dimensional slices at different time steps. This converts a complex spatial planning problem into a sequence of simpler temporal decisions, allowing the neural network to learn from incremental time-step transitions rather than processing the entire trajectory as a single high-dimensional output, thus reducing computational complexity and training time.
2Measurement precision
If multiple parameters are tuned to train a neural network for single trajectory determination, then the decision-making accuracy is improved, but the system complexity increases
Solution Approach 1:
The patent divides the trajectory planning into multiple independent decider modules, each handling specific decision types. This segmentation reduces the number of parameters each individual module needs to learn compared to a single comprehensive module, simplifying the training process while collectively achieving high decision-making accuracy through the ensemble of specialized modules.
Solution Approach 2:
The patent generates multiple candidate trajectory decisions in advance through different decider modules before selecting the final trajectory. This preliminary generation of options allows the system to evaluate multiple possibilities and choose the optimal one, improving decision-making accuracy without requiring a single complex module to learn all possible outcomes simultaneously.
3Measurement precision
If granular output actions like steering angle and gas pedal acceleration are used, then the control precision is improved, but the state space size increases
Solution Approach 1:
The patent segments granular control actions into discrete trajectory decisions (lane changes, turns, straight movement) handled by separate decider modules. Each module learns a simplified state space for its specific decision type rather than the full granular control space, reducing the effective state space size while maintaining control precision through the combination of specialized modules.
Solution Approach 2:
The patent transforms granular continuous control actions into discrete trajectory decisions organized in a temporal sequence of two-dimensional slices. This dimensional transformation converts a high-dimensional continuous control problem into a series of lower-dimensional discrete decisions at each time step, reducing the effective state space while preserving control precision through temporal decomposition.
Data Source
AI summary
An ego vehicle includes decider modules and a grader module coupled to a resolver module. The decider modules generate trajectory decisions at a current time, generate a current two-dimensional slice of a flat space around the ego vehicle, generate future two-dimensional slices of the flat space by projecting the current two-dimensional slice of the flat space forward in time, and generate a three-dimensional state space by stacking the current two-dimensional slice and the future two-dimensional slices. The grader module generates rewards for the trajectory decisions based on a recent behavior of an ego vehicle. The resolver module selects a final trajectory decision for the ego vehicle from the trajectory decisions based on the three-dimensional state space and the rewards. The current two-dimensional slice includes a current ego vehicle location and current neighboring vehicle locations. The future two-dimensional slices include future ego vehicle locations and future neighboring vehicle locations.