Language Model Routing for Resource-Limited Vehicle Query Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current large language models require significant computational power, memory, and resources, making them incompatible with the processing capabilities of automotive computing infrastructure, leading to inefficiencies in query processing.
Innovation Solution
A method involving a smaller first language model to evaluate queries, select an appropriate second language model based on query complexity, and process queries using that model, either locally or through external servers, reducing the need for extensive local resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If large language models are used to accurately process complex queries, then query processing accuracy is improved, but computational power requirements and processing time increase significantly
Solution Approach 1:
The patent segments the language model processing task into multiple smaller models with different capabilities. Instead of using one large model for all queries, the system divides models into tiers (e.g., small, medium, large) and assigns queries to appropriate models based on complexity evaluation, thereby reducing overall computational requirements while maintaining accuracy for complex queries.
Solution Approach 2:
The patent implements dynamic model selection where the system adapts its resource usage based on query characteristics. A first language model evaluates query complexity and dynamically selects from multiple second language models of varying sizes, allowing the system to optimize between accuracy and computational power in real-time based on actual needs.
2Speed
If large language models are deployed locally in vehicles, then query processing speed is improved, but memory and computational resources required exceed automotive infrastructure capabilities
Solution Approach 1:
The patent segments the language model infrastructure into multiple smaller models that can be distributed across different locations (local vehicle systems and remote servers). This allows the system to process queries faster by using smaller local models for simple tasks while offloading complex queries to remote infrastructure, avoiding the need to deploy a single large model locally.
Solution Approach 2:
The patent introduces an intermediary evaluation model (first language model) that assesses query complexity and mediates between local processing capabilities and remote server resources. This intermediary selects appropriate second language models from multiple options, enabling the system to balance local processing speed with available memory and computational resources by routing queries appropriately.
3Power
If smaller language models are used to reduce resource requirements, then computational power and memory needs are reduced, but query processing accuracy deteriorates
Solution Approach 1:
The patent implements dynamic model selection where the system adapts its resource usage based on query characteristics. A first language model evaluates query complexity and dynamically selects from multiple second language models of varying sizes, allowing the system to optimize between accuracy and computational power in real-time based on actual needs.
Solution Approach 2:
The patent changes the parameter of model size by providing multiple second language models with different sizes and capabilities. Instead of using a fixed small model, the system varies the model size parameter based on query complexity, using smaller models for simple queries and larger models for complex queries, thereby maintaining accuracy while reducing overall computational requirements.
4Adaptability or versatility
If multiple language models are maintained for different query types, then query processing versatility is improved, but system complexity increases
Solution Approach 1:
The patent applies preliminary action by using a first language model to evaluate and classify queries before they are processed by second language models. This pre-evaluation step organizes queries into complexity categories, allowing the system to efficiently select appropriate models without requiring complex real-time decision-making logic during query processing.
Solution Approach 2:
The patent introduces an intermediary evaluation model (first language model) that assesses query complexity and mediates between local processing capabilities and remote server resources. This intermediary selects appropriate second language models from multiple options, enabling the system to balance local processing speed with available memory and computational resources by routing queries appropriately.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A computer-implemented method is disclosed. The method comprising receiving an input query from a user; evaluating the input query based, at least in part, on a first language model; selecting a second language model from a plurality of second language models based, at least in part, on evaluating the input query; processing the input query based, at least in part, on the second language model; and performing an operation based, at least in part, on the processing of the input query.