Query Masking for Distributed Database Shard Operations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Shard operations in distributed databases can disrupt data access and require substantial rework of queries, leading to reduced efficiency and increased costs due to the need for updating queries to align with redefined table structures and links between shards.
Innovation Solution
A method to mask shard operations by updating queries to perform operations on original table structures, allowing existing queries to access data as if the table has not undergone a shard operation, using a query manager that identifies affected commands and updates them to execute on multiple shards, maintaining data integrity and reducing the need for query updates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If shard operations are performed to divide tables into multiple shards, then data storage and processing capability is improved, but query complexity and maintenance cost increase significantly
Solution Approach 1:
The patent introduces a query mask as an intermediary layer between the query and the sharded table structure. This mask contains mapping information that translates queries written against the original table structure into appropriate shard-specific queries, thereby resolving the contradiction by maintaining simple query interfaces while supporting complex sharded storage
Solution Approach 2:
The patent performs preliminary actions by pre-computing and storing the query mask that maps original table structure to sharded structure before queries are executed. This preparation work is done in advance during table sharding operations, so that subsequent queries can execute efficiently without needing to understand the complex shard structure
2Manufacturing precision
If queries are updated to align with redefined table structures after sharding, then data access accuracy is improved, but development time and operational efficiency decrease
Solution Approach 1:
The patent creates a copy of the original table structure in the form of a query mask that mirrors the pre-shard table schema. This copy allows applications to continue writing queries against the familiar original structure without needing to update them, while the system automatically translates these queries to the actual sharded structure, thereby maintaining data access accuracy without requiring query updates
3Adaptability or versatility
If the table structure is redefined through sharding, then scalability and data distribution are improved, but compatibility with existing queries deteriorates
Solution Approach 1:
The query mask serves as a mediator that translates between the original table structure and the sharded structure. It maintains compatibility with existing queries by preserving the original table schema in the mask, while simultaneously enabling scalability through the underlying shard structure. The mask contains the mapping logic that allows seamless translation without requiring changes to existing applications
Data Source
AI summary
A computer-implemented method masks shard operations in a distributed database. The method includes performing a shard operation on a first table with a first column and a second column, where the shard operation divides the first table into a first shard with the first column and a second shard with the second column. The method also includes receiving a query, where the query is configured to perform at least a first command. The method further includes determining the first command is directed to the first column. The method includes updating the query to access the first shard in place of the first table. The method also includes executing the query and returning a set of results for the query to a source of the query.