Prompt Builder Flow With Data Masking for Reliable LLM Responses
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing large language models (LLMs) suffer from hallucinations, lack consistency in responses due to third-party updates, and pose legal and privacy concerns when interacting with regulated industries like finance and healthcare, while existing systems fail to incorporate real-time data and risk data leakage.
Innovation Solution
A prompt builder system that retrieves and includes data within prompts to LLMs, masks sensitive information, and saves responses for reuse, ensuring compliance with legal and privacy requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If real-time data is integrated into LLM prompts, then the risk of data fabrication is reduced, but system complexity increases
Solution Approach 1:
The patent introduces a prompt builder as an intermediary component between the data source and the LLM. This prompt builder retrieves real-time data from databases, formats it into structured prompts, and passes it to the LLM. By inserting this intermediary layer, the system reduces data fabrication risks while managing complexity through modular design rather than direct integration of all components.
Solution Approach 2:
The system is divided into distinct functional modules: a data retrieval component that fetches real-time data, a prompt building component that structures the data into prompts, and an LLM component that generates responses. This segmentation allows each module to be optimized independently, reducing overall system complexity while maintaining reliability through specialized functions.
2Reliability
If sensitive information is masked to comply with regulatory standards, then legal and privacy requirements are met, but information completeness is reduced
Solution Approach 1:
The patent applies masking selectively to specific sensitive fields within the data structure rather than uniformly to all information. Sensitive personal identifiable information (PII) such as names, addresses, and social security numbers are masked, while non-sensitive operational data remains complete and unaltered. This local application of quality control maintains compliance while preserving necessary information completeness.
3Productivity
If third-party LLMs are updated sporadically, then model performance improves, but response consistency deteriorates
Solution Approach 1:
The patent implements a versioning system that records the specific LLM model version used for each prompt-response interaction. Before updating the LLM, the system captures the current version identifier and stores it with the prompt history. This preliminary action of version tracking allows the system to maintain consistency by referencing historical model versions even as new models are deployed, ensuring auditability and reproducibility of responses over time.
Data Source
AI summary
Disclosed herein are system, method, and computer program product embodiments for an improved prompt builder system. A system locates a data record based on matching a characteristic of the data record to a data record reference in a natural language prompt request. The system then masks a first field of the data record. For example, the first field may be altered or removed. The system then obtains a single-shot prompt response from a large language model (LLM) responsive to the natural language prompt request and the data record including the masked first field. The single-shot prompt response may include a second field from the data record.