Prompt Compression Under Token Limits for Stable LM Reasoning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large language models face challenges with computational complexity and performance degradation due to increased input context, particularly in key information extraction and reasoning processes, and lengthy prompts lead to significant cost increments with black-box API charges.
Innovation Solution
A method for compressing prompt information by obtaining target length constraint information and generating shorter, semantically relevant second prompt information using a generative target compression model, reducing token consumption and adapting to specific input limitations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the input context length is increased to improve the language model's reasoning and comprehension capability, then the model's understanding ability is improved, but the computational complexity increases and processing time is extended
Solution Approach 1:
The patent extracts and removes redundant information from the input prompt while preserving key semantic content. By identifying and eliminating unnecessary tokens, the system reduces the input context length from the original lengthy prompt to a compressed version that maintains essential meaning, thereby lowering computational complexity while preserving comprehension capability
Solution Approach 2:
The patent changes the parameter of input context length by applying compression algorithms that transform the original prompt into a shorter representation. This parameter transformation allows the model to process fewer tokens while maintaining the semantic integrity necessary for accurate comprehension and reasoning
2Measurement precision
If the input context length is increased to improve the language model's reasoning capability, then the model's reasoning ability is improved, but performance degradation occurs due to excessive input size
Solution Approach 1:
The patent extracts only the essential information needed for reasoning tasks while removing redundant content. By filtering out unnecessary tokens and preserving only critical semantic elements, the system maintains reasoning capability while avoiding performance degradation associated with excessively long inputs
Solution Approach 2:
The patent applies partial compression by retaining only the necessary portion of the input information required for effective reasoning. Rather than processing the complete original prompt, the system selectively preserves key elements that contribute to reasoning accuracy while discarding excess information that causes performance degradation
3Measurement precision
If a lengthy prompt is used to provide comprehensive context for the language model, then the model's understanding is improved, but the API cost increases significantly
Solution Approach 1:
The patent extracts and removes redundant information from the prompt while preserving key semantic content. By identifying and eliminating unnecessary tokens, the system reduces the input context length from the original lengthy prompt to a compressed version that maintains essential meaning, thereby lowering API costs while preserving understanding ability
Solution Approach 2:
The patent changes the parameter of input context length by applying compression algorithms that transform the original prompt into a shorter representation. This parameter transformation allows the model to process fewer tokens while maintaining the semantic integrity necessary for accurate comprehension, thus reducing API pricing based on token count
Data Source
AI summary
A method for compressing prompt information includes: obtaining first prompt information of a language model (LM); obtaining target length constraint information; and obtaining second prompt information of the LM by compressing the first prompt information based on the target length constraint information; in which a number of first tokens of the first prompt information is greater than a number of second tokens of the second prompt information, and semantics of the second prompt information is relevant to semantics of the first prompt information.