Retrieval-Augmented Generation Prompt Compression With Data-Driven Templates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Generative AI systems require large amounts of supplemental data, leading to excessive resource utilization and costly API calls due to lengthy prompts, with existing compression techniques often altering meaning and being uncontrollable.
Innovation Solution
A method for compressing prompts using importance scores to identify key keywords, populating placeholders in an abstract template, and applying a budget to generate a controlled, comprehensible compressed text fragment for transmission to a generative question-answering system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If voluminous supplemental content is provided to a generative AI system, then answer accuracy is improved, but network resources and data transmission costs increase
Solution Approach 1:
The patent extracts only the most important keywords from supplemental content based on importance scores, rather than transmitting the entire content. This selective extraction maintains answer accuracy while significantly reducing data transmission volume to the generative AI system.
Solution Approach 2:
The patent transforms the supplemental content from a complete text format to a compressed keyword format by changing the parameter representation. Importance scores are used to rank and select keywords, converting voluminous text into a compact set of high-value terms that preserve essential information.
2Loss of information
If complete text fragments are transmitted to a generative AI system, then information completeness is improved, but processing time and computational resources increase
Solution Approach 1:
The system extracts only the most relevant keywords from text fragments based on importance scores derived from dataset statistics. This extraction process maintains the essential information needed for accurate answer generation while reducing the processing burden on the generative AI system.
Solution Approach 2:
Instead of processing complete text fragments, the system applies partial action by selecting only a subset of the most important keywords. This partial processing approach is sufficient for maintaining information completeness while significantly reducing computational time and resources.
3Quantity of substance
If traditional compression techniques are applied to prompts, then data size is reduced, but meaning accuracy deteriorates
Solution Approach 1:
The patent changes the compression approach from generic text compression to parameter-based keyword selection. Importance scores, derived from dataset keyword frequencies and relevance metrics, are used to select keywords that preserve meaning accuracy while reducing prompt size.
Solution Approach 2:
The system uses importance scores as feedback mechanisms to guide the compression process. These scores, calculated from dataset statistics, provide feedback on which keywords are most valuable, ensuring that compression maintains meaning accuracy by retaining high-importance terms.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Systems and methods are disclosed herein for compressing a prompt. In an example system, an importance score listing is obtained that includes a score indicative of an importance of a plurality of dataset keywords. From the importance score listing, a keyword importance score is identified for a plurality of keywords in a current text fragment, such as a text fragment to be compressed. A set of placeholders in an abstract prompt template is populated based on the current text fragment. The current text fragment is compressed based on the importance of the plurality of keywords in the current text fragment to generate a compressed text fragment. In an example, the compressed text fragment is included in the prompt for transmission to a computing entity, such as a large language model of a generative question-answering system.