Data Delivery Queue Block Compression for Timeout Prevention
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In multi-tenant database environments, complex or lengthy query processing can lead to connection timeouts between clients and servers due to time-consuming data compression and transmission, especially when dealing with high volumes of repeating data, causing delays and disruptions in data delivery.
Innovation Solution
Implementing a data delivery queue that divides response data into blocks, compresses each block independently, and transmits them iteratively, with block size adjusted based on previous block performance to maintain connection activity and avoid timeouts, using a buffered pipeline approach to ensure continuous data delivery.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is compressed before transmission to reduce data size, then the quantity of transmitted data is reduced, but the processing time increases causing connection timeouts
Solution Approach 1:
The patent divides the complete response data into multiple blocks that are compressed and transmitted separately. This segmentation allows the system to transmit smaller portions of data through the compression process, reducing the time each block spends in compression while still achieving overall data reduction. The blocks are transmitted in sequence to the client, preventing connection timeouts.
Solution Approach 2:
The patent performs preliminary actions by dividing the data into blocks before compression begins. This preparation allows the compression process to work on smaller, manageable units rather than waiting to compress the entire dataset at once, thereby reducing the total compression time and preventing connection timeouts.
2Reliability
If data is divided into blocks and transmitted iteratively to maintain connection activity, then connection timeouts are prevented, but the system complexity increases
Solution Approach 1:
The patent implements segmentation by dividing response data into multiple blocks that are processed and transmitted independently. This segmentation simplifies the management of data flow during transmission, allowing each block to be handled as a discrete unit with its own compression and transmission timeline, thereby maintaining connection stability without excessive complexity.
Solution Approach 2:
The patent ensures continuity of useful action by iteratively transmitting blocks of data to the client. This continuous transmission keeps the connection active throughout the data delivery process, preventing timeouts. The systematic approach of processing and sending blocks in sequence maintains reliable data flow without requiring overly complex coordination mechanisms.
3Productivity
If block size is increased to reduce the number of transmissions, then the number of transmission operations is reduced, but the compression time for each block increases
Solution Approach 1:
The patent applies dynamics by adaptively determining block sizes based on the characteristics of the response data. Rather than using fixed block sizes, the system dynamically adjusts the granularity of data division to optimize the balance between compression time and transmission efficiency. This dynamic approach allows larger blocks when data patterns permit faster compression, while preventing timeout conditions through appropriate block sizing.
Data Source
AI summary
A computer system processes data requests using a data delivery queue. A query received from a client is processed to generate response data, wherein the response data is held in a queue prior to transmitting to the client. The response data in the queue is iteratively divided into a plurality of blocks, wherein each block of the plurality of blocks is compressed prior to transmitting to the client, and wherein a block size of a given block is determined based on an amount of time to compress and transmit a preceding block. The plurality of blocks are transmitted to the client. Embodiments of the present invention further include a method and program product for processing data requests using a data delivery queue in substantially the same manner described above.