Hierarchical Edge Language Models for Unreliable Network Queries
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Providing large language models in areas with limited and unreliable internet connections leads to prolonged downtime, significant delays, and high operational costs due to reliance on cloud-based solutions.
Innovation Solution
A hierarchical edge architecture with diverse compute capabilities, including small, medium, and large language models, dynamically selects the appropriate model based on network connectivity and context to process queries locally, reducing dependency on cloud resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If cloud-based large language models are used, then response accuracy and capabilities are improved, but operational costs and network dependency increase
Solution Approach 1:
The system segments language model capabilities into multiple tiers (small, medium, large models) deployed at different levels of the edge hierarchy. This allows the system to use only the necessary model size for each specific task, avoiding the cost and resource overhead of always deploying large cloud-based models while maintaining accuracy when needed.
Solution Approach 2:
The patent introduces a hierarchical dimension to model deployment, organizing language models across multiple levels (edge devices, gateway servers, cloud) rather than relying solely on a single cloud-based dimension. This multi-dimensional architecture enables flexible model selection based on task requirements and network conditions.
2Measurement precision
If cloud-based large language models are used, then response accuracy is improved, but downtime during network outages increases
Solution Approach 1:
The system segments the language model ecosystem into multiple independent components deployed at different locations (edge devices, gateway servers, cloud). This segmentation allows local models to continue operating independently during network outages, ensuring system reliability while maintaining the option to use more accurate cloud models when available.
Solution Approach 2:
The system performs preliminary deployment of language models across the edge hierarchy before network outages occur. By pre-positioning models at edge devices and gateway servers, the system ensures that language processing capabilities are already available locally, eliminating downtime during network disruptions.
3Adaptability or versatility
If cloud-based large language models are used, then model capabilities are improved, but data transmission time over weak links increases
Solution Approach 1:
The system segments language model deployment across multiple tiers, allowing simpler tasks to be handled by smaller local models that require no data transmission, while reserving complex tasks for larger models at gateway or cloud levels. This segmentation eliminates unnecessary data transmission time for routine operations while maintaining access to advanced capabilities when needed.
Solution Approach 2:
The patent implements local quality by deploying appropriate language model capabilities directly at edge devices and gateway servers based on local requirements. This ensures that data processing occurs locally without network transmission delays, while still providing access to more powerful remote models for complex tasks that require higher capabilities.
4Measurement precision
If a single large language model is deployed, then response quality is improved, but device complexity and resource requirements increase
Solution Approach 1:
The system segments language model deployment into a hierarchical structure with multiple model sizes and capabilities distributed across different devices and locations. This segmentation reduces the complexity burden on individual devices while maintaining overall system response quality through coordinated multi-model operations.
Solution Approach 2:
The hierarchical edge architecture creates a universal system where multiple language models with different capabilities work together to handle diverse tasks. Rather than requiring each device to host a single large model, the system enables any device to participate in language processing using appropriately-sized models, reducing individual device complexity while maintaining versatile capabilities across the system.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present disclosure relates to systems and methods for using language models in locations with limited network connectivity. The systems and methods include a hierarchical edge architecture with a plurality of language models with diverse compute capabilities. The systems and methods dynamically select a language model from the plurality of language models to use to respond to a query received by a user in response to determining a level of network connectivity available at a user device.