Cosharded Index Tables for Faster Distributed Data Retrieval
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed relational databases face inefficiencies in data retrieval due to the separation of tables across multiple database servers, requiring extensive communication and operations to locate and retrieve data.
Innovation Solution
The method involves cosharding an index table by the keys of a data table or another index table, ensuring all data corresponding to a single row is stored in a single split, reducing the need for distributed operations and maintaining transactional integrity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is spread across multiple database servers, then storage capacity is increased, but data retrieval efficiency deteriorates due to required distributed operations
Solution Approach 1:
The patent segments the index table into multiple index table splits, where each split contains a subset of index entries. This segmentation allows the system to store more data across multiple servers while reducing the scope of distributed operations needed for data retrieval, as queries can be routed to specific splits based on key ranges.
2Adaptability or versatility
If data is separated across different database servers, then storage scalability is improved, but system complexity increases due to distributed operations
Solution Approach 1:
The patent performs preliminary sharding of the index table into splits during data insertion, organizing index entries by key ranges in advance. This preliminary action eliminates the need for complex distributed operations during query execution, as the routing decision can be made locally based on the query key and split routing information.
3Reliability
If distributed operations are used to locate data, then data availability across servers is improved, but processing time increases
Solution Approach 1:
The patent extracts the routing information (index table split routing) from the distributed system complexity and stores it locally with each data entry. This extraction allows the system to maintain data availability across servers while eliminating time-consuming distributed operations, as the routing decision can be made locally using the extracted routing information.
Data Source
AI summary
The technology relates to cosharding tables within a distributed storage system. A data table including one or more rows may be received. Each row in the data table may include an identifier key and pieces of data. Each piece of data in the data table may be indexed into individual rows of an index table, wherein each row in the index table includes data associated with the identifier key of the data table from which the piece of data in the respective row was indexed. The index table may be sharded into splits, wherein the sharding includes assigning each row of the index table into one of the splits based on the identifier key of the data table from which the piece of data in the respective row was indexed. The splits may be more stored into two or more or more portions of the distributed storage system.