Dynamic Query Chunking and Streaming for Reduced Wait Time
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The complexity of modern web services infrastructure makes it challenging to maintain high service performance and user experience, particularly in monitoring and logging across disparate systems, where prolonged query processing times can lead to user dissatisfaction and system stalls.
Innovation Solution
A system and method for dynamic query chunking and streaming, where incoming queries are broken into smaller chunks, processed, and results are streamed back to the user interface as they are completed, allowing for immediate partial results that converge to the final outcome, with strategies selected based on processing time and user experience metrics.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If queries are processed as single large units, then complete results are obtained, but user wait time increases and system stalls occur
Solution Approach 1:
The patent divides large queries into smaller chunked queries that can be processed independently and in parallel. Each chunked query represents a subset of the original query's data requirements, allowing the system to process multiple chunks simultaneously rather than waiting for one large query to complete, thereby reducing user wait time while maintaining overall processing throughput.
2Loss of time
If queries are chunked and processed in parallel, then user wait time is reduced, but system complexity increases
Solution Approach 1:
The patent introduces a query management system that acts as an intermediary between the user interface and the chunked query processing. This intermediary receives user queries, automatically determines whether chunking is appropriate, divides queries into chunks, coordinates parallel processing, and aggregates results. This centralized management layer handles the complexity internally while presenting a simple interface to users, thus reducing perceived system complexity.
3Ease of operation
If incremental results are streamed to users, then user experience is improved, but result accuracy convergence takes longer
Solution Approach 1:
The patent implements continuous streaming of incremental query results to the user interface as chunks are processed, rather than waiting for all chunks to complete. Each streamed result represents a partial answer that converges toward the complete result over time. This continuous delivery maintains user engagement and provides immediate value while the system continues processing remaining chunks in the background, thus improving user experience without sacrificing final result accuracy.
Data Source
Figure 1A
Figure 1B
Figure 1C~1D
AI summary
Instead of processing a query as-is, the query is chunked or broken down into a sequence of smaller chunked queries and the chunked results of those smaller queries are streamed back to the requestor. Chunking the query and streaming the chunked results can substantially decrease the user's time to value when running a query by returning some immediate results for display which are refined and eventually converge on the full results as each chunked query runs.