Visual Token Caching for Multi-Turn Generative Dialogs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current generative models that process visual content through textual descriptions lead to information loss and increased computational and network resource consumption in multi-turn human-to-computer dialogs, necessitating additional interactions and prolonged processing.
Innovation Solution
The solution involves processing visual content to generate responses, thereby avoiding re-computation and reducing latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If visual content is converted to textual descriptions for processing by generative models, then the models can process visual information, but information loss occurs and computational resources are wasted
Solution Approach 1:
The patent creates a copy of the visual content in the form of an image representation (tokenized form) that can be stored and reused. Instead of repeatedly converting the same visual content to text descriptions, the system generates one image representation and reuses it across multiple turns, eliminating information loss while improving processing efficiency.
Solution Approach 2:
The patent performs preliminary processing by converting visual content to an image representation at the beginning of the conversation. This pre-computed representation is then stored and reused in subsequent turns, avoiding the need to repeatedly process the original visual content and eliminating redundant computational work.
2Loss of time
If text descriptions are generated for each visual content query, then subsequent queries can be processed, but additional computational processing is required and latency increases
Solution Approach 1:
The system performs the computationally intensive visual processing once at the beginning, creating an image representation that captures all necessary visual information. This preliminary action eliminates the need for repeated processing in subsequent turns, reducing both latency and computational resource consumption while maintaining full access to visual content details.
Solution Approach 2:
The patent creates a reusable copy of the visual content in tokenized form. This image representation serves as a permanent copy that can be accessed indefinitely without requiring the original visual processing, thereby eliminating redundant computational work and reducing energy consumption across multiple dialogue turns.
3Productivity
If the same visual content is processed multiple times through text description generation, then each query can be answered independently, but computational and network resources are wasted
Solution Approach 1:
The system performs visual content processing once in advance, creating an image representation that is stored for reuse. This preliminary processing eliminates the need for repeated network calls to visual processing services, reducing network resource consumption while enabling fast response generation through the reusable representation.
Solution Approach 2:
The patent creates a persistent copy of the visual content in the form of an image representation that can be reused across multiple queries. This copy eliminates the need to repeatedly process and transmit visual content over the network, thereby reducing network resource consumption while maintaining high productivity in generating responses.
Data Source
AI summary
Implementations relate to handling visual content across a multi-turn dialog. A user input that includes natural language content and visual content is received during the dialog. If the visual content is being received for the first time in the dialog, the visual content is processed to generate a corresponding tokenized representation of the visual content. The corresponding tokenized representation can be cached in a database in association with the dialog, or in association with a user account of a user of the user query. If the visual content is subsequently referenced in the dialog, the corresponding tokenized representation of the visual content is retrieved from the database. The corresponding tokenized representation of the visual content, corresponding tokenized representations of natural language content, and optionally other metadata can be processed, using a generative model, to generate a response responsive to the user input.