Vehicle Behavior Planning Using LLM-Guided Scenario Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing rule-based planners for autonomous vehicles are limited to predefined scenarios and struggle with complex situations, while Large Language Models (LLMs) provide unreliable and slow planning results, posing safety risks.
Innovation Solution
A combination of a rule-based planner with a text-based neural network, specifically an LLM, where prompts are generated based on a scene representation to select planning scenarios, allowing the rule-based planner to handle complex scenarios reliably.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a rule-based planner is used for behavioral planning, then the planning is robust and interpretable for simple scenarios, but it cannot generalize to complex scenarios that deviate from predefined planning scenarios
Solution Approach 1:
The patent introduces an LLM as an intermediary component that bridges the gap between simple rule-based planning and complex scenario handling. The LLM receives scene representations, generates natural language descriptions of complex situations, and selects appropriate planning scenarios from the predefined set, allowing the rule-based planner to handle complex scenarios while maintaining reliability through the structured scenario selection process
Solution Approach 2:
The planning system is segmented into distinct functional components: scene representation generation, LLM-based scenario selection, and rule-based trajectory generation. This segmentation allows each component to specialize in its strength while working together to solve the overall planning problem, with the LLM handling scenario understanding and the rule-based planner handling reliable trajectory generation
2Adaptability or versatility
If an LLM is used for behavioral planning, then the generalization capability is excellent, but the planning results are slow and may produce unsafe or unrealistic trajectories
Solution Approach 1:
The patent extracts the slow and unreliable trajectory generation function from the LLM and assigns it to the fast rule-based planner. The LLM is retained only for its strength in understanding complex scenarios and selecting appropriate planning scenarios, thereby eliminating the productivity bottleneck while preserving the generalization capability
Solution Approach 2:
The system dynamically assigns different tasks to different planning components based on their strengths. The LLM is used for high-level scenario understanding and selection, while the rule-based planner handles low-level trajectory generation. This dynamic task allocation optimizes both speed and accuracy for different aspects of the planning process
3Loss of information
If an LLM is used for behavioral planning, then the understanding of described situations is excellent, but it produces unreliable quantitative trajectory data
Solution Approach 1:
The LLM serves as an intermediary that translates complex scene understanding into structured scenario selections, which then guide the rule-based planner to generate precise trajectories. The LLM's natural language processing capabilities are used for qualitative scene understanding, while quantitative trajectory precision is handled by the mathematically rigorous rule-based planner
Solution Approach 2:
The patent substitutes the LLM's unreliable quantitative trajectory generation with the rule-based planner's deterministic trajectory generation. The LLM's scene understanding is preserved and used to select appropriate scenarios, while the trajectory generation is performed by the rule-based system that guarantees precision and safety
Data Source
AI summary
A concept for planning a behavior of a vehicle in a traffic scene expands planning competencies of a rule-based planner so that the rule-based planner can also be used for behavioral planning in more complex scenarios. The rule-based planner performs behavioral planning based on a scene representation of the traffic scene that replicates at least one planning scenario from a planner-specific set of predefined planning scenarios. A text-based neural network is used for selection of the at least one planning scenario. Based on the scene representation, at least one text query describing the traffic scene is generated for the neural network. The at least one query requests a behavioral recommendation that is supported by the planner-specific set of predefined planning scenarios. Based on the at least one text query, the neural network generates a text-based behavioral recommendation, which is assigned to at least one of the predefined planning scenarios.