Pipelined SPL Code Generation with Retrieval-Augmented Verification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Individuals in the technology industry, including engineers and software developers, often lack familiarity with programming languages they are required to use, and existing large language models (LLMs) struggle to generate accurate and contextually relevant software code, particularly in complex languages like Search Processing Language (SPL), leading to inefficiencies and errors.
Innovation Solution
A pipelined search query generation engine utilizing a retrieval augmented generation (RAG) pipeline to process natural language queries, incorporating a vector database and LLMs to generate, explain, or answer SPL statements, and integrate schema verification to reduce hallucinations, ensuring accurate and contextually relevant code generation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If existing large language models are used to generate software code, then code generation capability is provided, but accuracy and contextual relevance deteriorate, particularly in complex languages like SPL
Solution Approach 1:
The patent introduces a retrieval augmented generation system that acts as an intermediary between the LLM and the code generation task. A retrieval module fetches relevant code snippets, documentation, and examples from a knowledge base before the LLM generates the code. This intermediary retrieval mechanism ensures the LLM has accurate, contextually relevant information about SPL syntax and semantics, thereby improving code accuracy without sacrificing generation capability
Solution Approach 2:
The system performs preliminary retrieval of relevant information from the knowledge base before the actual code generation occurs. By pre-fetching accurate SPL syntax rules, semantic definitions, and contextual examples, the system ensures the LLM works with verified information rather than generating from potentially inaccurate training data alone, thus improving reliability before generation happens
2Productivity
If LLMs are used to generate code in complex programming languages, then development efficiency is improved, but errors and hallucinations increase
Solution Approach 1:
The patent implements a feedback mechanism where the retrieved information from the knowledge base is used to verify and correct the LLM's generation. The system can compare generated code against retrieved accurate SPL syntax rules and semantic definitions, identifying and correcting hallucinations or errors in real-time, thus maintaining development efficiency while reducing harmful outputs
Solution Approach 2:
The retrieval module serves as a mediator that provides verified reference information to the LLM, preventing the LLM from generating hallucinated or incorrect code. By having the LLM reference accurate SPL documentation and examples retrieved from the knowledge base, the system reduces errors while maintaining the speed benefits of AI-generated code
3Extent of automation
If individuals use LLMs to generate code snippets and automate tasks, then repetitive coding work is reduced, but understanding of programming language syntax and semantics deteriorates
Solution Approach 1:
The retrieval module acts as an educational intermediary by providing the LLM with accurate SPL syntax rules, semantic definitions, and coding examples. This ensures that even when the LLM automates code generation, it references verified language knowledge, thereby maintaining understanding of syntax and semantics while enabling automation
Solution Approach 2:
The system performs preliminary retrieval of accurate programming language information before generation occurs. This pre-fetching of verified syntax and semantic knowledge ensures that the automated code generation process is grounded in correct language understanding, preventing the loss of technical knowledge while maintaining high automation capability
Data Source
AI summary
Disclosed herein are systems and methods for improving the auto-generation of pipelined search query statements by a large language model (LLM). In some examples, such a method includes operations of receiving a user-provided prompt, wherein the user-provided prompt is provided in natural language, identifying an objective of the user-provided prompt, and based on the objective, providing the user-provided prompt to a first operational pipeline of a plurality of operational pipelines, wherein each operational pipeline is associated with a unique prompt template. Additionally, the method may include generating, by the first pipeline, an auto-generated prompt based on a first unique prompt template of the first pipeline, providing the auto-generated prompt to a large language model (LLM), and receiving a response to the auto-generated prompt from the LLM. A graphical user interface (GUI) may then be generated that displays the response to the auto-generated prompt.