Distributed Table Storage Splitting via Asynchronous Logs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed storage systems, the need to split slice copies simultaneously to maintain consistency results in low availability, as the original slices do not provide sufficient service capacity with increasing data amounts and request pressures.
Innovation Solution
A distributed table storage processing method where a master server generates a splitting log and serial number, sends it to slave servers, and performs splitting storage only after receiving a threshold number of acknowledgment messages, ensuring that both servers execute the same splitting log sequence, even if they do not perform storage simultaneously.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If multiple slice copies are split simultaneously to maintain consistency, then data consistency is ensured, but system availability decreases
Solution Approach 1:
The master server generates and distributes the splitting log (containing splitting rules and parameters) to all slave servers before the actual splitting operation begins. This preliminary action allows slave servers to prepare in advance and execute splitting asynchronously without waiting for simultaneous coordination, thus maintaining data consistency while improving system availability.
Solution Approach 2:
The patent introduces a dynamic splitting mechanism where the master server sends a second splitting instruction to trigger actual splitting operations. Slave servers can execute the splitting at different times based on their local conditions, transforming the static simultaneous splitting requirement into a dynamic asynchronous process that maintains consistency through the pre-distributed splitting log.
2Productivity
If slice splitting is performed to increase service capacity, then productivity improves, but system complexity increases
Solution Approach 1:
The patent segments the slice splitting process into distinct components: a splitting log (containing splitting rules and parameters), acknowledgment messages, and triggering instructions. This segmentation allows the system to manage complex splitting operations through modular, independent message exchanges between the master server and slave servers, reducing overall system complexity while enabling scalable slice splitting.
3Reliability
If slice copies are stored on multiple servers for fault tolerance, then reliability improves, but coordination difficulty increases
Solution Approach 1:
The master server acts as an intermediary that coordinates the splitting operation across multiple slave servers. It generates the splitting log, distributes it to all slave servers, receives acknowledgment messages, and triggers the actual splitting through a second instruction. This intermediary approach simplifies coordination by centralizing control logic while allowing slave servers to operate independently, thus maintaining fault tolerance without excessive coordination complexity.
Data Source
AI summary
A distributed table storage processing method, device and system, a master server to which a to-be-split slice belongs generates a splitting log of the to-be-split slice and a serial number of the splitting log after receiving a first splitting instruction for the to-be-split slice sent by a control device; sends the splitting log and serial number of the splitting log to a slave server to which the to-be-split slice belongs, and the slave server sends a first acknowledgment message to the master server after storing the splitting log and the serial number; performs splitting storage on the to-be-split slice according to the splitting log to obtain a plurality of new slices when the number of the received first acknowledgment messages is not less than a first threshold; sends a second splitting instruction to each slave server, and the slave server performs splitting storage to obtain a plurality of new slices.