Quasi-Locator Streaming for Large Data Across Database Servers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database management systems face inefficiencies and high computational costs due to the two-roundtrip approach for retrieving large object data (LOB) across multiple database servers, especially when executing queries with CTAS or ITAS constructs, which incur significant network latency and resource overhead.
Innovation Solution
Implementing a quasi-locator mechanism that allows for the automatic streaming of large data from a large data repository database server to a receiving database server without additional requests, reducing the need for two-roundtrip data fetching by indicating whether the data will be streamed directly, and enabling efficient data transfer and storage based on memory availability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the two-roundtrip approach is used for retrieving LOB data, then the client can maintain the LOB locator for subsequent requests, but significant network latency and computational costs are introduced when different LOBs are requested across many database servers
Solution Approach 1:
The system performs preliminary actions by including an indication in the initial LOB locator response that allows the database server to automatically stream the LOB data without waiting for a second request. This preliminary setup enables the server to proactively transfer the data, eliminating the need for the client to initiate a second roundtrip and reducing network latency while maintaining the locator maintenance capability.
2Ease of operation
If the two-roundtrip technique is used to fetch full LOB data for CTAS or ITAS constructs, then the querying database server can apply criteria on the LOB data, but the network is burdened and significant latency is introduced
Solution Approach 1:
The system extracts only the necessary LOB data by implementing automatic streaming that transfers data directly from the source database server to the querying server without requiring the client to request and receive the full LOB data in a second roundtrip. This extraction approach reduces network resource overhead while maintaining the capability to process and apply criteria on the LOB data.
3Productivity
If automatic streaming of LOB data is implemented, then network latency is reduced, but the system complexity increases due to the quasi-locator mechanism
Solution Approach 1:
The system merges the LOB locator functionality with automatic data streaming by combining these two previously separate operations into a single integrated mechanism. The quasi-locator indication is embedded within the existing locator structure, allowing the system to maintain backward compatibility while enabling automatic streaming, thus improving productivity without proportionally increasing complexity.
Data Source
AI summary
Techniques are described for efficient transfer of large data type data (large data) among database servers for executing a distributed query. In an implementation, a database server receives object reference data, including an indication of whether the object reference data is a quasi-locator. The receipt of the quasi-locator indicates that the large data is to be streamed from the large database repository without any further request from the receiving database server. Accordingly, the receiving database server receives and stores the requested large data without generating and sending any further requests to the large data repository database server.