Database Reslicing via Asynchronous Write Queues
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing relational database systems face performance issues and unacceptably long downtime during reslicing operations due to the need to lock slices while copying data, which blocks database operations and affects user experience.
Innovation Solution
Implementing a method that creates write queues for new slices and asynchronously writes database transactions to these queues, allowing data to be copied and transactions to proceed without blocking, using a distribution function to determine where data is copied, enabling asynchronous data replication and reducing downtime.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If data is copied from a slice during reslicing operations, then data consistency is improved, but database operation availability deteriorates due to slice locking
Solution Approach 1:
The system creates new slices and write queues before the reslicing operation begins, preparing all necessary structures in advance. This allows the original slice to be locked for copying while transactions are redirected to the pre-prepared new slices, maintaining availability during the operation.
Solution Approach 2:
Write queues serve as intermediary structures between the original slice and new slices. Transactions are enqueued to the write queues instead of directly accessing slices, allowing the system to buffer and manage data flow during reslicing without blocking database operations.
2Manufacturing precision
If a slice is locked during data copying, then data consistency is improved, but operation speed deteriorates due to blocked transactions
Solution Approach 1:
The system segments the slice data into multiple new slices and distributes transactions across multiple write queues. This segmentation allows parallel processing of transactions on different slices while one original slice is being copied, maintaining overall system speed despite individual slice locking.
Solution Approach 2:
While one slice is locked for copying, the system continues to accept and process transactions by redirecting them to newly created slices through write queues. This ensures continuous useful action (transaction processing) without interruption, even during the locking and copying phase.
3Manufacturing precision
If data is copied synchronously during reslicing, then data consistency is improved, but time consumption deteriorates due to blocking operations
Solution Approach 1:
New slices and write queues are created before the reslicing operation begins, so that when copying starts, the destination structures are already ready. This preliminary preparation eliminates setup time during the actual copying process and allows transactions to be immediately redirected.
Solution Approach 2:
The system maintains continuous transaction processing by enqueueing transactions to write queues during the copying phase. This asynchronous queuing allows copying and transaction processing to occur concurrently without blocking, significantly reducing the total time required for reslicing while maintaining data consistency.
Data Source
AI summary
Systems and methods for reslicing data in a representation of a relational database are disclosed. In one embodiment, the database includes a representation including a first slice. The database system creates a plurality of new slice and to create a plurality of write queues. The database system copies units of data in the first slice to the new slices according to a distribution function. The distribution function determines, for each unit of data in the first slice, one of the new slices into which to copy the unit of data. The database system asynchronously writes one or more actions of a set of one or more asynchronous database transactions to the first slice when copying the data in the first slice to the new slices. The database asynchronously enqueues the one or more actions of the set of asynchronous database transactions in the write queues according to the distribution function.