Distributed Database Region Splitting via Reserved Blank Regions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed databases, improper initial split points lead to inefficient region splitting, resulting in poor data read/write performance and uneven data distribution, especially when frequent splits are needed due to changing data patterns.
Innovation Solution
A method for rapid multi-point region splitting in a distributed database, where a blank region is reserved and split based on pre-estimated key value ranges, with split points determined by service type, allowing for efficient data storage and improved read/write performance by preventing data loss during the splitting process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Stability of the object's composition
If region splitting is performed frequently to adapt to changing data patterns, then data distribution becomes more even, but data read/write performance deteriorates due to repeated split operations
Solution Approach 1:
The patent pre-calculates and reserves multiple split points during the initial table creation phase, based on predicted future data distribution patterns. This preliminary action allows the system to perform rapid region splitting without needing to recalculate split points during frequent data reorganization operations, thus maintaining data distribution quality while avoiding performance degradation from repeated split calculations.
Solution Approach 2:
The patent segments the region splitting process into multiple independent regions based on pre-calculated split points. Each region can be managed and split independently, allowing parallel processing and reducing the overall impact on read/write performance when splitting occurs. This segmentation enables the system to handle data distribution changes more efficiently.
2Productivity
If region splitting is performed with improper initial split points, then data read/write performance becomes poor, but the system still needs to handle frequent data reorganization
Solution Approach 1:
The patent performs preliminary calculation of optimal split points during table creation, considering predicted data growth patterns and access patterns. By determining split points in advance rather than reacting to data changes, the system avoids time-consuming reorganization operations and maintains consistent read/write performance even as data grows.
Solution Approach 2:
The patent changes the approach from dynamic split point calculation to static pre-calculated split points. This parameter change transforms the region splitting process from a time-consuming operational task to an efficient structural initialization task, significantly reducing the time required for data reorganization while maintaining good read/write performance.
3Stability of the object's composition
If data is read and stored in multiple files during region splitting, then region splitting can be performed, but split speed becomes low
Solution Approach 1:
The patent pre-determines all split points and region boundaries during table creation, so that during actual region splitting operations, the system only needs to redirect data to pre-identified regions rather than calculating and managing multiple temporary files. This preliminary preparation significantly accelerates the splitting process.
Solution Approach 2:
The patent extracts the complex file management and data redistribution logic from the region splitting process by using pre-calculated split points. This allows the splitting operation to focus solely on directing data to the correct pre-determined regions, eliminating the need for complex intermediate file management and dramatically improving split speed.
Data Source
AI summary
In an embodiment, a first region is set according to a preset condition, the first region is a blank region reserved for storing new data of a key-value type. An embodiment method includes: when the new data of the key-value type is to be stored in the first region, receiving a region split command; updating, according to the region split command, a state identifier of the first region and in a state table to an identifier indicating that a read/write service is no longer provided, splitting the first region into at least two second regions, updating state identifiers of the second regions in the state table to identifiers indicating that a read/write service can be provided; and recording, in a metadata table, storage location information of the second regions.