Lazy Optimistic Concurrency Control for Database Archiving
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Archiving data in cloud platforms often leads to resource locking, resulting in poor system performance and data inconsistencies due to the need to prevent data changes during the archiving process, which can cause user modifications to be lost and lead to inconsistencies between databases.
Innovation Solution
Implementing lazy optimistic concurrency control (LOCC) allows for concurrent monitoring and writing of data, identifying changes in real-time without locking database records, thereby reducing latency and preventing data inconsistencies by logging modifications for later handling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If database locking is used to prevent data inconsistencies during archiving, then data consistency is maintained, but system performance deteriorates and resource availability decreases
Solution Approach 1:
The system performs preliminary actions by copying data records to be archived into a staging area before the actual archiving process. This preliminary copy operation allows the system to prepare for archiving without locking the source database records, enabling concurrent access and maintaining system performance while ensuring data consistency through the staged transfer process.
2Reliability
If database locking is used during archiving, then data consistency is maintained, but user access and resource availability are reduced
Solution Approach 1:
The system introduces a staging area as an intermediary between the source database and the archive database. Data records are first copied to this intermediate staging area, then archived from there. This intermediary mechanism allows the source database to remain unlocked and accessible to users throughout the archiving process, while still ensuring data consistency through the controlled transfer process.
3Productivity
If data is archived without locking, then system performance is maintained, but data inconsistencies occur due to concurrent modifications
Solution Approach 1:
The system implements a feedback mechanism where data records in the staging area are monitored for consistency with the source database during the archiving process. If modifications are detected in the source database, the system receives feedback and adjusts the archiving process accordingly, such as re-copying modified records or updating the staging area, thereby maintaining data consistency without requiring database locks.
4Reliability
If traditional archiving methods are used, then data consistency is ensured, but resource usage increases and latency increases
Solution Approach 1:
The archiving process is segmented into distinct phases: initial copy to staging area, consistency monitoring, and final archival. This segmentation allows different parts of the process to proceed concurrently without blocking each other. The source database copy operation can proceed without locking, while consistency is verified in subsequent stages, reducing overall archiving latency while maintaining data consistency.
Data Source
AI summary
Methods, systems, and devices supporting data storage are described. A database system may implement lazy optimistic concurrency control (LOCC). LOCC may support an efficient use of database connections for a data archiving process. The data archiving process may involve a first worker thread at an application server copying data records from a primary database and writing the data records to a secondary database. A second worker thread may operate concurrent to the first worker thread to monitor the primary database for changes (e.g., using a shared database connection), as the primary database may not lock the copied data records during the archiving process. Data records that are altered in the primary database during the archiving process may be logged for error handling by an administrator. The data records that are copied from the primary database and not altered during the archiving process may be deleted from the primary database.