LLM Context Provisioning With Iterative Tool Selection for Code Reliability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large language models (LLMs) face limitations in generating correct and contextually relevant compilable computer code due to constraints on input context and the difficulty in identifying relevant context without overwhelming inference costs, especially when dealing with proprietary or complex code types.
Innovation Solution
An iterative tools-based solution and a language server-based solution are employed to gather and utilize context for LLMs, where the iterative approach involves invoking tools specified in the user prompt and iteratively refining context, while the language server identifies relevant code snippets and metadata to enhance context for code generation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If more context is provided to the LLM, then code generation reliability is improved, but inference costs increase
Solution Approach 1:
The patent extracts only the most relevant context information needed for code generation, separating essential context from unnecessary information. The system identifies and retrieves specific code snippets, file metadata, and contextual information that are directly relevant to the generation task, excluding redundant data to minimize inference costs while maintaining reliability.
Solution Approach 2:
The context provided to the LLM is segmented into distinct, manageable components such as current file content, imported files, related code snippets, and metadata. This segmentation allows the system to provide targeted context information in a structured manner, reducing the overall volume of context needed while improving the quality and relevance of information provided.
2Adaptability or versatility
If relevant context is identified without hard-coded rules, then system adaptability is improved, but device complexity increases
Solution Approach 1:
The system employs the LLM itself to identify relevant context by having it select from a list of available tools and functions. Rather than using external hard-coded rules, the LLM autonomously determines which context information is relevant based on the generation request, making the system adaptable to different scenarios without requiring complex rule-based configuration.
Solution Approach 2:
The patent introduces an intermediary layer that provides the LLM with a list of available tools and functions, along with their descriptions and example syntax. This intermediary structure guides the LLM in selecting relevant context without requiring hard-coded rules, bridging the gap between system capabilities and LLM decision-making while maintaining adaptability.
3Manufacturing precision
If iterative context gathering is performed, then code accuracy is improved, but time consumption increases
Solution Approach 1:
The system performs preliminary actions by providing the LLM with a pre-formulated list of available tools and functions with their descriptions and example syntax before the LLM begins generating code. This preliminary preparation enables the LLM to efficiently select relevant context in subsequent iterations without requiring extensive exploration, reducing overall time consumption while maintaining accuracy.
Solution Approach 2:
The iterative process incorporates feedback mechanisms where the LLM's generated code and tool selections are evaluated, and context information is refined in subsequent iterations. The system uses feedback from previous iterations to improve context selection, gradually enhancing code accuracy while minimizing the number of iterations needed and thus reducing time consumption.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
In an example embodiment, an iterative tools-based solution is utilized wherein a listing of possible tools for context gathering may be provided as input to an LLM along with a user prompt, by a host program. The LLM then generates text that invokes one or more of the tools when executed by the host program. When the host program executes the generated text from the LLM, context is gathered in accordance with the tool or tools selected by the LLM. This context can then be fed back into the LLM. This process can continuously iterate until the LLM has enough context to generate a response to the initial user prompt itself.