LLM Plan Generation with Contextual Knowledge Retrieval
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current natural language processing technologies, particularly Large Language Models (LLMs), face challenges in efficiently generating plans of action for knowledge worker tasks within organizational contexts, often requiring extensive training data and struggling with ambiguous or insufficiently relevant requests.
Innovation Solution
An automated general knowledge worker system that enables human users to interact using natural language, leveraging a Large Language Model (LLM) to generate plans of action by mapping requests to previous fulfilled requests or generating new plans based on contextual data from a knowledge base.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If LLMs are trained using vast amounts of text scraped from the Internet, then they contain knowledge about syntax, semantics, and ontology data, but the training process becomes time-consuming and resource-intensive
Solution Approach 1:
The system performs preliminary action by pre-training LLMs on vast amounts of text data before deployment. The models are trained in advance on internet-scraped text to acquire knowledge about syntax, semantics, and ontology, so that when deployed in the automated knowledge worker system, they can immediately generate plans without requiring additional training time during operation.
2Productivity
If the automated general knowledge worker attempts to map requests to previously fulfilled requests using search techniques, then it can reuse existing plans, but the mapping process becomes complex and may fail for ambiguous requests
Solution Approach 1:
The system uses an intermediary approach by introducing a hybrid architecture that combines traditional search techniques with LLM-based generation. The search component attempts to map new requests to previously fulfilled requests using text searching with relevancy determination, while the LLM component generates plans for ambiguous or novel requests, thereby reducing mapping complexity and improving overall task efficiency.
Solution Approach 2:
The system applies dynamics by implementing a dynamic request processing architecture that adapts its approach based on request characteristics. The system first attempts deterministic mapping using search techniques, and when mapping confidence is low or requests are ambiguous, it dynamically transitions to using LLM-based plan generation, allowing the system to handle both routine and novel requests effectively.
3Adaptability or versatility
If the system uses LLM to generate new plans of action based on contextual data, then it can handle ambiguous requests, but the generation process requires extensive computational resources
Solution Approach 1:
The system applies partial action by using LLMs selectively rather than for all requests. The architecture first attempts to handle requests through lighter-weight search and mapping operations, and only invokes the computationally intensive LLM generation process when necessary—for ambiguous, novel, or complex requests—thereby reducing overall computational energy consumption while maintaining adaptability.
4Measurement precision
If the automated general knowledge worker requests additional information from users for ambiguous mappings, then it can improve plan accuracy, but the interaction process becomes longer and more complex
Solution Approach 1:
The system implements feedback by having users provide additional information when request mappings are ambiguous or insufficient. The system detects mapping ambiguity, feeds this information back to the user through a query interface, and uses the user's response to refine the plan generation process, thereby improving plan accuracy while minimizing interaction time through targeted information requests.
Data Source
AI summary
An automated general knowledge worker may be used to perform knowledge worker tasks. The automated general knowledge worker may receive a text-based request to perform a knowledge worker task for an organization from a computing device. The automated general knowledge worker may retrieve one or more existing plans of action and contextual data that are relevant to the text-based request from a knowledge base. Further, the automated general knowledge worker may generate a text-based prompt that at least includes text information included in the text-based request, the one or more existing plans of action that are relevant to the text-based request, and the contextual data that are relevant to the text-based request. Subsequently, the automated general knowledge worker may send the text-based prompt to a large language model (LLM) to prompt the LLM to generate a new plan of action for performing the knowledge worker task.