Language Model Control Pipeline for Precise Vehicle and Robot Decisions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing artificial intelligence systems for autonomous driving and robotic control suffer from poor decision-making precision due to decoupled task modules, where errors in upstream modules lead to incorrect downstream decisions.
Innovation Solution
An end-to-end automatic control system is constructed through interaction between a first and a second language model, where the first language model obtains scenario information and generates a query text, and the second language model provides a reply text for decision-making, integrating sensing requirements in a unified manner.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If task modules are decoupled with clear upstream-downstream dependency relationships, then interpretability and ease of development and maintenance are improved, but decision-making precision deteriorates due to error propagation from upstream to downstream modules
Solution Approach 1:
The system segments task processing into multiple specialized language models (first language model for scenario understanding, second language model for decision-making) that operate in a coordinated pipeline. Each model handles specific aspects of the task, improving both interpretability through modular specialization and precision through focused processing at each stage.
Solution Approach 2:
The patent introduces intermediate processing steps between upstream sensor data and downstream decisions, including scenario information extraction and query text generation. These intermediary layers filter and structure information to prevent error propagation while maintaining modular architecture benefits.
2Device complexity
If traditional fixed-format interfaces are used for module communication, then system structure is simple and easy to implement, but the ability to represent diverse sensing requirements and improve decision precision is limited
Solution Approach 1:
The system changes the parameter format of inter-module communication from fixed structured interfaces to flexible natural language queries and responses. This allows the system to convey rich, unstructured sensing requirements and scenario information that better support precise decision-making while maintaining manageable complexity through standardized language model interfaces.
Data Source
AI summary
A data processing method, and includes: obtaining scenario information based on sensor data and a first prompt text by using a first language model, where a target object is a vehicle or a robot; generating a second prompt text based on the scenario information and a current execution objective of the target object by using a second language model, where the second prompt text indicates a query for information that needs to be known by the target object when the target object completes the execution objective and that is not included in the scenario information; obtaining, based on the sensor data and the second prompt text by using the first language model, a reply text corresponding to the second prompt text; and obtaining a control instruction of the target object based on the reply text, the scenario information, and the current execution objective by using the second language model.