Code Token Substitution for Long-Context ML Code Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning models, particularly large-language models, are limited by the maximum request length, which restricts the amount of program code that can be processed in a single query, leading to inefficiencies in processing longer sections of code and reducing the quality and speed of operations such as translation or optimization.
Innovation Solution
The method involves replacing elements of the original code with shorter substitute elements and processing the modified code using a machine learning model, allowing longer sections of code to be processed effectively, and includes training the model to adapt to different replacement strategies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Length of moving object
If the maximum request length is used to process program code, then the model can process code within the length limit, but longer sections of code cannot be processed in a single query
Solution Approach 1:
The patent segments long code elements (such as variable names, function names, or code blocks) into smaller substitute tokens that fit within the model's request length limits. For example, a long variable name like 'rbg_VSSWiperCtrl_stLSSCar' is replaced with shorter substitute tokens, allowing the overall code structure to be processed in fewer queries while maintaining the ability to represent the full code when needed.
2Quantity of substance
If code elements are broken down into multiple tokens, then longer code can potentially be processed, but the number of tokens increases and reaches the limit quickly
Solution Approach 1:
The patent changes the parameter of element representation by introducing substitute tokens with different length characteristics. Instead of using one token per word or character, the system uses variable-length substitute tokens that can represent multiple original code elements, effectively changing the tokenization parameter to reduce overall token count for long code sections.
Solution Approach 2:
The patent creates simplified copy representations of code elements through substitute tokens. These substitute tokens are shorter versions that capture the essential meaning or structure of the original long code elements, allowing the model to process the code structure without needing to handle every detail of the original lengthy identifiers.
3Loss of information
If the entire original code is passed to the model, then complete context is available, but the request exceeds the maximum length limit
Solution Approach 1:
The patent segments the code into a modified version with substitute elements and processes this segmented representation. The segmentation allows the complete code structure to be included in the request by using shorter substitute tokens, thereby retaining full context information while staying within the request length limit.
4Productivity
If multiple queries are used to process long code sections, then the code can be processed in chunks, but processing quality decreases due to limited context in each query
Solution Approach 1:
The patent changes the parameter of code representation by using substitute tokens that compress long identifiers into shorter forms. This parameter change allows the entire code to be processed in a single query with full context available, eliminating the need for multiple chunked queries and thereby maintaining high processing quality while improving productivity.
Data Source
Figure 1I~1III
Figure 2
Figure 3
AI summary
One aspect of the present disclosure relates to a method for modifying program code using a machine learning model. The method comprises accessing original code and replacing elements of the original code with substitute elements to obtain a modified code. The substitute elements are shorter than the corresponding elements of the original code. The method further comprises processing the modified code using a machine learning model.