LLM Context Provisioning with Iterative Tool-Based Code Retrieval
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large language models (LLMs) face limitations in generating correct and contextually relevant compilable computer code due to constraints on the amount of context they can process, leading to errors in syntax and formatting, especially when dealing with proprietary or complex code types.
Innovation Solution
An iterative tools-based approach where a host program provides a list of tools to a LLM, iteratively invoking these tools to gather context, and a language server identifies relevant code snippets to enhance context for code generation, using techniques like bidirectional encoders and generative adversarial networks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the amount of context provided to the LLM is increased to improve code generation accuracy, then the reliability of code generation improves, but the inference costs increase
Solution Approach 1:
The patent extracts only the most relevant context information needed for code generation rather than providing all available context. The system identifies and extracts specific code snippets, file paths, and contextual information that are directly relevant to the current coding task, filtering out unnecessary information to reduce inference costs while maintaining generation accuracy.
Solution Approach 2:
The patent segments the context into multiple hierarchical levels: current file context, imported module contexts, and broader project context. This segmentation allows the system to provide comprehensive context when needed while enabling selective provision of context segments, reducing overall inference costs by only processing necessary segments.
2Manufacturing precision
If comprehensive context is provided to the LLM to ensure correct syntax and formatting, then the manufacturing precision of code output improves, but the device complexity increases
Solution Approach 1:
The patent introduces an intermediary component that acts as a bridge between the codebase and the LLM. This intermediary analyzes the codebase, identifies relevant context, and prepares it in a structured format for the LLM. It also processes the LLM's output to ensure syntax and formatting correctness, reducing the complexity burden on the LLM itself while maintaining high manufacturing precision.
Solution Approach 2:
The system performs preliminary analysis and preparation of context information before it reaches the LLM. This includes pre-processing code snippets, organizing file structures, and preparing contextual data in advance, so that the LLM receives ready-to-use, structured information that reduces the complexity of real-time processing during code generation.
3Productivity
If more context snippets are gathered to improve code generation relevance, then the productivity of accurate code output increases, but the loss of time in context gathering worsens
Solution Approach 1:
The patent applies partial action by gathering only the necessary amount of context information required for the current coding task rather than comprehensively analyzing the entire codebase. The system determines the minimum viable context needed and gathers only that, achieving high productivity while minimizing time loss.
Solution Approach 2:
The system performs preliminary indexing and organization of codebase information in advance, creating a structured representation that enables rapid context retrieval during code generation. This pre-processing allows the system to quickly gather relevant context snippets without time-consuming analysis during the actual coding task.
Data Source
AI summary
In an example embodiment, an iterative tools-based solution is utilized wherein a listing of possible tools for context gathering may be provided as input to an LLM along with a user prompt, by a host program. The LLM then generates text that invokes one or more of the tools when executed by the host program. When the host program executes the generated text from the LLM, context is gathered in accordance with the tool or tools selected by the LLM. This context can then be fed back into the LLM. This process can continuously iterate until the LLM has enough context to generate a response to the initial user prompt itself.