LVLM-Guided MPPI Path Planning for Complex Road Scenarios

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing navigation techniques for autonomous vehicles face challenges in balancing various costs, such as track, collision, and goal costs, while correctly understanding road situations and adhering to traffic rules, particularly in complex and dynamic environments.

Innovation Solution

Integrating a Large Vision Language Model (LVLM) with a Model Predictive Path Integral (MPPI) controller to enhance path planning by providing structured driving instructions, which are processed by the MPPI to calculate optimal trajectories based on real-time environmental data and traffic scenarios.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If rule-based systems are used to analyze road situations and infer decisions for MPPI, then navigation decisions can be made based on explicit traffic rules, but the system requires significant effort to create rules for all road cases and maintaining consistency is challenging

Engineering Contradiction:

Improvenavigation decision reliabilityVSAvoidrule creation and maintenance complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent replaces the mechanical rule-based system with a neural network model that learns traffic rules and road situation understanding from data. The neural network processes perception data and map data to generate navigation decisions, substituting the manual rule-creation approach with an automated learning-based approach that scales better to diverse road cases.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Adaptability or versatility

If data-driven methods are used to learn navigation rules from data, then the system can adapt to various road scenarios, but the immense amount of data required and difficulty in testing and interpreting decisions are challenges

Engineering Contradiction:

Improveroad scenario adaptabilityVSAvoiddata requirement and interpretability

Core Design Contradiction:

Adaptability or versatilityVSLoss of information

Solution Approach 1:

The patent segments the navigation system into multiple components: a neural network for learning from data, a perception system for extracting road features, a map system for providing contextual information, and an MPPI controller for trajectory optimization. This segmentation allows the system to process information in manageable stages and reduces the need for a single massive dataset by using structured data flow through specialized modules.

Inventive Principle:

Principle #1Segmentation

3Productivity

If MPPI controller optimizes trajectories by balancing track costs, collision costs, and goal costs, then optimal paths can be calculated, but correctly understanding road situations while considering all traffic restrictions remains challenging

Engineering Contradiction:

Improvetrajectory optimization efficiencyVSAvoidroad situation understanding difficulty

Core Design Contradiction:

ProductivityVSDifficulty of detecting and measuring

Solution Approach 1:

The patent introduces a neural network as an intermediary between the perception/map systems and the MPPI controller. The neural network processes raw perception and map data to extract meaningful road situation features and traffic restriction information, then feeds this processed information to the MPPI controller. This intermediary layer simplifies the MPPI controller's task by providing pre-processed, semantically enriched inputs that encode road situation understanding.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS20250236314A1Model predictive path integral controller guided by large vision language model for intelligent autonomous vehicle path planning

Publication Date: 2025.07.24 CONSTRUCTOR AUTONOMOUS AG

AI summary

Systems and methods for autonomous-vehicle navigation using a large vision language model (LVLM) to understand road situations and construct a guide for the autonomous vehicle's planning system and Model Predictive Path Integral (MPPI) controller. The LVLM analyzes image data from driving scenarios and generates driving suggestions. The LVLM is pre-trained on a dataset of image-pairs and fine-tuned with specific driving scenarios to optimize performance.