AST Code Elision for Token-Limited and Valid LLM Edits
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for submitting source code to large language models (LLMs) face challenges in fitting within token limits while maintaining syntactic validity, often resulting in inefficient resource usage and inaccurate edits due to incomplete or irrelevant code segments.
Innovation Solution
A system that dynamically compacts source code using an abstract syntax tree (AST) to identify and select the most relevant code segments, generating a compacted document that maintains syntactic validity and fits within token limits, allowing for accurate and efficient LLM interactions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the entire source code file is submitted to the LLM, then the LLM has complete context for accurate edits, but the prompt exceeds the maximum token limit
Solution Approach 1:
The source code file is segmented into an abstract syntax tree (AST) structure, dividing the code into hierarchical nodes representing different syntactic elements. This segmentation allows selective extraction of relevant code segments while maintaining syntactic validity, resolving the contradiction between providing complete context and staying within token limits.
Solution Approach 2:
The system extracts only the necessary code segments from the full source code file by traversing the AST and identifying nodes that are relevant to the user's request. This extraction process removes redundant code while preserving the syntactic structure needed for accurate LLM processing, thereby reducing prompt length without sacrificing edit accuracy.
2Length of stationary object
If code segments are randomly selected to fit token limits, then the prompt length is reduced, but the syntactic validity and contextual relevance are compromised
Solution Approach 1:
The system performs preliminary processing by constructing the AST before selecting code segments. This preliminary action organizes the code into a structured format that makes it easy to identify syntactically valid segments, ensuring that any subset of code extracted for the prompt maintains proper syntax while fitting within token limits.
Solution Approach 2:
The system changes the parameter of code representation from raw text to structured AST nodes. This parameter change enables precise control over which code segments are included in the prompt, allowing the system to maintain syntactic validity by selecting complete nodes from the AST rather than arbitrary text fragments.
3Length of stationary object
If irrelevant code segments are included in the prompt, then the token limit is reduced, but the LLM's editing accuracy deteriorates due to lack of contextual relevance
Solution Approach 1:
The system uses the user's request as feedback to guide the selection of code segments. By analyzing the user's intent and comparing it against the AST structure, the system identifies and extracts only those code segments that are relevant to the requested operation, ensuring high contextual relevance while maintaining efficient use of the token limit.
Solution Approach 2:
The system applies local quality by making different parts of the code structure serve different functions: the AST provides the structural framework for validity, while the selectively extracted nodes provide the contextual relevance needed for accurate editing. This local differentiation ensures that each included code segment serves a specific purpose related to the user's request.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Examples enable dynamic eliding of code using structural information for syntactic validity. An identified code segment from an original document and a request for a large language model (LLM) to perform an action associated with the identified code segment is received. A compacted abstract syntax tree (AST) including removable nodes is generated based on the original document. The removable nodes are scored for relevance to the identified code segment. Code segments corresponding to the most relevant removable nodes are added to a compacted document without exceeding a configurable token limit for prompts to the LLM. A modified prompt including the identified code segment and the most relevant code segments is provided to the LLM. The edits received from the LLM in response to the modified prompt are mapped into the original document to create a syntactically valid edited version of the original source code while minimizing resource usage.