Spreadsheet Prompt Chaining for Lower-Latency LLM Responses
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing spreadsheet applications with integrated AI models are limited to spreadsheet data and functionalities, while large language models (LLMs) introduce latency, require significant compute power, and may generate irrelevant responses due to hallucinations.
Innovation Solution
An application service generates multiple prompts for an LLM service based on user input and spreadsheet data, determining the order in which to input these prompts to optimize the LLM's response, including requests for formulas, explanations, and self-evaluations to improve relevance and reduce latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If LLM integration is used in spreadsheet environments, then the capability to generate novel and open-ended responses is improved, but latency increases negatively impacting user experience
Solution Approach 1:
The patent segments the LLM interaction into multiple focused prompts rather than single comprehensive queries. The system breaks down user needs into specific task components (data analysis, formula generation, explanation) and processes them sequentially through targeted prompts, reducing overall latency while maintaining response quality
Solution Approach 2:
The system performs preliminary actions by pre-processing user input and spreadsheet data before LLM invocation, preparing contextual information in advance. This includes identifying relevant data ranges, formulating initial prompt structures, and organizing context so that the LLM can process information more efficiently when activated
2Adaptability or versatility
If LLM integration is used in spreadsheet environments, then general-purpose AI capabilities are improved, but compute power requirements increase tremendously
Solution Approach 1:
The system applies partial action by using LLM capabilities only when necessary and to the extent needed. Rather than continuously engaging the LLM, the system activates it selectively for specific tasks that require natural language processing, while relying on traditional spreadsheet functions for routine operations, thereby reducing overall compute power consumption
3Ease of operation
If LLM integration is used in spreadsheet environments, then natural language processing capability is improved, but hallucination issues worsen generating irrelevant responses
Solution Approach 1:
The patent implements feedback mechanisms where the system evaluates LLM outputs against the original user intent and spreadsheet context. It includes self-correction loops that verify generated responses for relevance and accuracy, and allows for iterative refinement of prompts based on output quality, thereby reducing hallucinations and improving response reliability
Solution Approach 2:
The system introduces an intermediary layer between the LLM and the final output that acts as a mediator. This intermediary validates LLM responses, checks them against spreadsheet data consistency, and filters out hallucinated or irrelevant information before presenting results to the user, thereby maintaining reliability while preserving natural language processing capabilities
Data Source
AI summary
Technology is disclosed herein for an application service which interfaces with an LLM service. In an implementation, the application service receives a natural language input from a user associated with a spreadsheet. The application service generates multiple prompts based on the natural language input and a portion of the spreadsheet and determines an order in which to input the prompts to an LLM service. The application service determines an order in which to input the prompts to an LLM service, then inputs the prompts to the LLM service according to the determined order.