Database Archive Circuit Breaker for Storage Management
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Database systems face storage space issues due to archive operations, which can lead to unavailability as they run out of disc space, particularly in LSM tree implementations where archive activity generates substantial disc space usage.
Innovation Solution
Implementing an 'archival circuit breaker' mechanism that pauses and cancels archive operations when storage space thresholds are met, allowing the system to dynamically adjust thresholds based on historical data and using machine learning models to manage disc space usage, thereby ensuring database availability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If archive operations are performed continuously to maintain data backup integrity, then data retention and recoverability are improved, but storage space is depleted leading to database unavailability
Solution Approach 1:
The system dynamically adjusts archive operation parameters including pause/resume timing, archive intervals, and thresholds based on real-time storage space monitoring and historical data analysis, transforming static continuous archiving into adaptive dynamic archiving that responds to storage conditions
Solution Approach 2:
The system changes operational parameters such as archive frequency, data retention periods, and space thresholds based on learned patterns from historical data, allowing optimization of the balance between backup integrity and storage space utilization
2Reliability
If archive operations are paused to preserve storage space, then database availability is maintained, but data backup integrity and point-in-time recoverability are compromised
Solution Approach 1:
The system performs preliminary actions by pausing archive operations before storage space is completely depleted, using predictive thresholds learned from historical data to prevent storage exhaustion while maintaining sufficient backup coverage for recoverability
Solution Approach 2:
The system uses feedback from historical archive data and real-time storage monitoring to continuously refine pause/resume decisions, learning optimal thresholds that balance availability requirements with backup integrity needs
Data Source
AI summary
Techniques are disclosed relating to archive operations for database systems. In some embodiments, a database system initiates one or more archive operations to archive one or more data extents for a database maintained by the database system. The system may halt archive activity for the database, in response to determining that archive operations for a threshold amount of data extents are initiated but not completed. The system may cancel at least one of the one or more archive operations. The system may determine to resume activity for the database based on determining that a threshold timer interval has elapsed and determining that a threshold amount of storage space is available for the database system. Disclosed embodiments may improve database availability, relative to traditional techniques.