Code Search Prompt Augmentation for Unseen Code Responses
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large language models perform poorly when generating responses for code elements they have not been trained on during training, resulting in vague and useless outputs.
Innovation Solution
A codebase segment table is created with source code segments indexed by embeddings, and the top-k closest embeddings to a user query and context are used to augment the prompt, providing examples from the codebase to improve the model's accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If a large language model is not trained on the source code of a specific codebase, then the model can be trained on a massively-large training dataset with billions of parameters, but the model performs poorly when generating responses for code elements it has not been trained on, resulting in vague and useless outputs
Solution Approach 1:
The system performs preliminary action by creating a codebase segment table with embeddings of code segments and their metadata before the actual code search task. This pre-computed embedding table enables rapid retrieval of relevant code examples during inference, allowing the model to access specific codebase knowledge without retraining on the entire codebase.
Solution Approach 2:
The system introduces an intermediary mechanism - a code search component that uses embedding-based similarity search to bridge the gap between the user query and the relevant code segments in the codebase. This intermediary retrieves the top-k most similar code segments and their metadata, which are then included in the prompt to the large language model, enabling the model to generate accurate responses for unseen code elements.
2Productivity
If the model is given only a user prompt without examples from the codebase, then the input is simple and fast to process, but the model generates vague and useless responses for code elements it has not been trained on
Solution Approach 1:
The system performs preliminary action by pre-computing and storing embeddings for all code segments and their metadata in a codebase segment table before the actual query processing. This pre-computation enables rapid retrieval during inference, so that when a user query arrives, the system can quickly find and attach relevant code examples without delaying the response generation.
Solution Approach 2:
The system creates simplified copies of relevant code segments and their metadata from the actual codebase, selecting only the top-k most similar segments based on embedding similarity. These copied examples are then embedded in the prompt to the large language model, providing the model with relevant context without requiring it to process the entire codebase, thus maintaining fast response generation while improving response quality.
3Measurement precision
If the system searches the entire codebase for every user query, then the most relevant examples can be found, but the search time and computational resources increase significantly
Solution Approach 1:
The system performs preliminary action by pre-computing embeddings for all code segments and organizing them in a codebase segment table with their associated metadata before any user queries are processed. This pre-computation transforms the expensive full-codebase search into a efficient embedding similarity search, where the system only needs to compare the query embedding against the pre-computed embeddings to find the top-k most similar segments, dramatically reducing search time while maintaining matching accuracy.
Solution Approach 2:
The system changes the search parameter from text-based full-codebase scanning to embedding-based similarity search. By representing code segments and queries as embeddings in a vector space, the system can measure semantic similarity efficiently using vector operations, finding the top-k most relevant code segments much faster than traditional text search methods while maintaining or improving matching accuracy.
Data Source
AI summary
A user query for information regarding data of a codebase is answered by a large language model given a prompt that includes examples of code segments from the codebase that are similar to the user query. The code segments from the codebase are associated with metadata that includes both natural language text and source code. The search for the examples of code segments from the codebase is based on embeddings of code segments and associated metadata that are closely similar to an embedding of the user query and context.