LLM Query Response via Custom Corpus Retrieval
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current large language models (LLMs) face challenges such as high computational costs for specialization, potential for inaccurate responses due to overfitting, and inability to fit on devices with limited memory, especially when data updates occur.
Innovation Solution
The approach involves using a custom corpus of documents to condition LLM responses, allowing external applications to select relevant documents based on user queries without retraining the LLM, enabling tailored responses without fine-tuning and accommodating data updates without additional training.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If LLM parameters are fine-tuned on a new set of training data to specialize the model, then the LLM can generate more accurate responses for specific domains or users, but the computational cost increases significantly and the model size remains too large for local deployment on memory-constrained devices
Solution Approach 1:
The patent extracts the customization functionality from the LLM parameters themselves and relocates it to external retrieval components. Instead of modifying the LLM to incorporate domain-specific knowledge through fine-tuning, the system extracts this knowledge into separate document collections that can be independently managed and retrieved based on query context, thereby avoiding the computational overhead of retraining while maintaining response accuracy.
Solution Approach 2:
The patent introduces an intermediary retrieval system that acts as a mediator between the user query and the LLM. This intermediary component fetches relevant documents from external sources based on the query context and injects them into the LLM prompt, allowing the model to generate accurate responses without requiring parameter modifications. This intermediary layer decouples the customization process from the model training process.
2Reliability
If the LLM is retrained when the training data is updated to maintain accurate responses, then the LLM can adapt to new information, but the computational cost and time required for retraining increase
Solution Approach 1:
The patent implements a dynamic retrieval system where the set of documents injected into the LLM prompt can be flexibly updated without retraining the model. When new information becomes available, the system simply adds or modifies documents in the external corpus and adjusts the retrieval logic to incorporate these updates, allowing the LLM to adapt to new data dynamically while maintaining the same trained parameters.
Solution Approach 2:
The patent performs preliminary organization and indexing of domain-specific documents outside the LLM training process. By pre-processing and structuring the knowledge base in advance, the system enables rapid updates through simple document additions or modifications rather than requiring full retraining cycles, thus reducing the time and computational resources needed for updates.
3Adaptability or versatility
If the LLM is made extremely large with billions of parameters to improve general performance, then the model can handle diverse tasks better, but it cannot fit locally on user devices with constrained memory space
Solution Approach 1:
The patent moves the knowledge storage from the model parameters dimension to an external document dimension. Instead of increasing model size to accommodate more domain-specific knowledge, the system stores this knowledge in separate document collections that are retrieved and injected into the LLM prompt as needed. This dimensional shift allows the model to maintain its original size while gaining access to extensive domain-specific information.
Solution Approach 2:
The patent creates a universal framework where a single LLM model can serve multiple specialized domains by dynamically injecting different document sets based on the query type. The same base model can handle various domains (medical, legal, technical, etc.) by retrieving and incorporating relevant documents from corresponding external corpora, eliminating the need for multiple specialized models or a single oversized model.
Data Source
AI summary
At least utilizing a custom corpus of documents to condition a large language model (LLM) when generating a response to a user query. In some implementations, a user query associated with a client device is received. An API query for an external application is generated by an LLM based on the user query. The external application has access to a custom corpus of documents comprising a plurality of documents. The external application is queried using the API query. Data representative of one or more documents in the custom corpus of documents is received from the external application in response to the API query. The LLM generates a response to the query that is conditioned on the data representing one or more of the documents in the custom corpus of documents received from the external application. The response to the user query is caused to be rendered on the client device.