Generative Model Prompt Simplification Using Cached Intermediate Results
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Generative AI models require significant computational resources and data transfer between memories due to large matrix sizes and increased computation when processing lengthy prompts, leading to prolonged processing times.
Innovation Solution
Simplify prompts into simplified forms including intent and details, search for stored task records based on intent, and utilize intermediate computation results from previous tasks to execute the generative model, reducing computational load and data transfer.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the generative model processes longer prompts, then the model can handle more complex tasks and provide more detailed responses, but the computation time and data transfer requirements increase significantly
Solution Approach 1:
The patent segments the prompt processing into two parts: a simplified prompt that is processed by the generative model and a detailed prompt that is stored and retrieved. The detailed prompt is divided into multiple records that can be searched and retrieved based on the simplified prompt, allowing the system to handle complex prompts efficiently without increasing computation time.
Solution Approach 2:
The patent performs preliminary action by pre-processing and storing detailed prompts in a database before they are needed. When a new prompt arrives, the system searches for matching stored records and retrieves only the necessary portions, avoiding the need to re-process entire detailed prompts and significantly reducing computation time.
2Adaptability or versatility
If the generative model processes longer prompts, then the model can handle more complex tasks, but the amount of data transferred between memories increases
Solution Approach 1:
The patent extracts only the essential information from detailed prompts and stores it in a simplified format in the database. When processing a new prompt, the system retrieves only the necessary stored records rather than transferring entire detailed prompts, significantly reducing data transfer volume while maintaining the ability to handle complex tasks.
Solution Approach 2:
The patent creates a copy of the detailed prompt in a simplified format and stores it in the database. This copy is then retrieved and used for processing, allowing the system to handle complex prompts without transferring the full original data, thus reducing data transfer requirements.
3Measurement precision
If the generative model uses larger matrices for computation, then the model can process prompts more accurately, but the computational load and processing time increase
Solution Approach 1:
The patent segments the computation process by separating the prompt simplification step from the generative model processing step. The simplification step reduces the input size to a manageable level, and the generative model then processes this simplified prompt with appropriate matrix sizes, achieving accurate processing without excessive computational load.
Solution Approach 2:
The patent changes the parameter of prompt length by transforming long detailed prompts into shorter simplified prompts through the simplification step. This parameter change allows the generative model to process the input with appropriate matrix sizes, maintaining accuracy while reducing computational load.
Data Source
AI summary
A processing method of a generative model includes: obtaining a prompt; simplifying the prompt into a simplified prompt including an intent and details; searching for stored records of a previously performed tasks, based on the intent of the simplified prompt; based on identifying a stored record of a previously performed task corresponding to the intent of the simplified prompt, executing the generative model according to the simplified prompt using an intermediate computation result corresponding to the previously performed task; and outputting an execution result of the generative model.