Federated Query Data Serialization via Remote BLOB Conversion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional federated query processing requires significant time and resource costs due to data conversion in the federation server system, as it involves copying data multiple times, which can be inefficient when dealing with diverse remote database types.
Innovation Solution
The proposed solution moves data type conversion to the remote database system, using a modified database access API that enables serialization of query results into binary large object (BLOB) data, allowing for efficient data conversion and deserialization directly into an in-memory data structure within the federation server system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data conversion is performed in the federation server system using conventional federated query processing, then data can be accessed across diverse remote database types, but significant time and resource costs are incurred due to multiple data copying operations
Solution Approach 1:
The patent extracts the data conversion operation from the federation server system and relocates it to the remote database system. By integrating a serialization library into the remote database system's database image, the conversion process is performed at the source rather than being transported and converted at the destination, thereby reducing processing time and resource consumption in the federation server.
Solution Approach 2:
The patent implements preliminary action by performing data serialization at the remote database system before data transmission. The query results are converted into a serialized format during the query execution phase at the remote database, eliminating the need for subsequent conversion operations in the federation server system and reducing overall processing time.
2Adaptability or versatility
If data conversion is performed in the federation server system, then data from diverse remote database types can be integrated, but resource costs increase due to multiple data copying operations
Solution Approach 1:
The patent extracts the computationally intensive data conversion operation from the federation server system and relocates it to the remote database system. By performing serialization at the source system with access to local resources, the federation server conserves computational resources and reduces energy consumption while maintaining data integration capabilities.
Solution Approach 2:
The patent implements self-service by enabling the remote database system to perform its own data conversion and serialization operations through an integrated serialization library. This allows the remote database system to handle its own data format conversion needs without requiring resource-intensive processing at the federation server, thereby reducing overall system resource costs.
3Productivity
If a modified database access API with serialization support is implemented, then data conversion efficiency is improved, but system complexity increases
Solution Approach 1:
The patent introduces a serialization library as an intermediary component that is integrated into the remote database system's database image. This intermediary provides the necessary serialization functionality without requiring fundamental changes to the core database engine or federation server architecture, thereby improving conversion efficiency while limiting the increase in overall system complexity.
Data Source
AI summary
Techniques are described with respect to processing a federated query. An associated computer-implemented method includes compiling a query received from a client computing system to generate a query statement and a query access plan. The query access plan incorporates a modified database access application programming interface (API) that supports data serialization. The method further includes executing the query access plan to transmit the query statement to a remote database system. The method further includes fetching a query result set from the remote database system including serialized binary large object (BLOB) data. The method further includes deserializing the serialized BLOB data of the query result set and populating an in-memory data structure with deserialized query results. In an embodiment, the method further includes transmitting the deserialized query results to the client computing system. In an additional embodiment, the method further includes generating query serialization capabilities for the remote database system.