Edge NLP Retrieval Architecture for Resource-Constrained Queries
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing large language models (LLMs) are not feasible for space missions due to size, weight, and power constraints, and require substantial computational resources, limiting their deployment in contested and non-internet connected environments.
Innovation Solution
Deployment of retrieval-augmented generation (RAG) models using edge computing systems and methods that leverage state-of-the-art language model techniques to efficiently query and retrieve relevant information from a corpus of instruction manuals and procedural documentation using natural language queries in resource-constrained environments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If large language models are deployed for natural language processing, then processing capability is improved, but device size and power consumption increase
Solution Approach 1:
The patent segments the large language model into smaller specialized models (e.g., BERT, GPT-2) that can be distributed across multiple edge devices. Each device runs a portion of the processing workload, enabling complex NLP tasks while maintaining individual device size constraints. The system divides the computational burden through functional segmentation of the AI processing architecture.
Solution Approach 2:
The patent transitions from single-device centralized processing to multi-device distributed processing architecture. By adding the spatial dimension of networked edge devices, the system achieves large-scale NLP capabilities without increasing individual device size. The computational power is aggregated across the network rather than concentrated in one large device.
2Productivity
If large language models are deployed for natural language processing, then processing capability is improved, but power consumption increases
Solution Approach 1:
The patent segments the computational workload across multiple edge devices, so each device consumes less power individually while the collective system achieves high NLP capability. The processing tasks are divided and distributed, preventing any single device from exceeding power constraints.
Solution Approach 2:
The patent uses smaller specialized language models that are copies or distilled versions of larger models. These condensed models retain essential NLP capabilities while requiring significantly less computational power and energy consumption, making them suitable for resource-constrained edge devices.
3Measurement precision
If retrieval-augmented generation models are used, then query accuracy is improved, but computational complexity increases
Solution Approach 1:
The patent segments the RAG system into distinct functional modules: embedding generation, similarity search, and response synthesis. Each module is implemented as a separate computational component that can be optimized independently, reducing overall system complexity while maintaining high query accuracy through coordinated operation of specialized sub-systems.
Solution Approach 2:
The patent introduces embedding vectors as an intermediary representation between the query and the document corpus. This intermediate step transforms the complex full-text matching problem into a simpler vector similarity search, significantly reducing computational complexity while preserving or improving query accuracy through semantic understanding.
4Reliability
If edge computing systems are deployed in extreme environments, then operational reliability is improved, but resource constraints increase
Solution Approach 1:
The patent deploys smaller specialized language models that are condensed copies of larger models. These compact models require fewer computational resources (less memory, fewer parameters, lower processing power) while maintaining sufficient accuracy for edge device operation in resource-constrained extreme environments.
Solution Approach 2:
The patent optimizes each edge device's computational resources locally by deploying only the specific model components and data needed for local processing. Each device is configured with appropriate resource allocation for its specific function, avoiding the waste of deploying full-scale models across all devices and maximizing resource efficiency in extreme environments.
Data Source
AI summary
Exemplary system and methods for processing a natural language query in an edge computing system are disclosed. A processor of the computing system receives a natural language textual input as a query from a user interface and receives one or more containers of documentation over a communication channel. The processor generates a query embedding vector from the textual input. The processor extracts text from the received container and generates text chunks of specified length from the extracted data. Text embeddings are generated from the text chunks and stored in memory for a specified period. The query embeddings are compared with the text embeddings to determine relevant context information. The processor passes the relevant context information and the query through a trained neural network to generate a response. The response generated by the trained neural network is formatted and output to a user interface.