Database Latch Release for Delta Fragment Garbage Collection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Database management systems face challenges in optimizing performance due to the complexity and volume of queries, particularly in managing memory allocation for delta fragments and releasing latches efficiently to allow for garbage collection during query execution.
Innovation Solution
The solution involves periodically identifying safe times during query execution to release and reacquire latches on delta fragment versions, allowing for garbage collection, and using a database execution engine with a query optimizer and execution engine to manage latch acquisition and release, including instantiating threads and performing garbage collection when necessary.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If latches are held on delta fragment versions during query execution to ensure data consistency, then reliability is improved, but memory cannot be reclaimed and resource overhead increases
Solution Approach 1:
The system performs preliminary actions by creating new delta fragment versions before releasing latches on old versions. This ensures that data consistency is maintained because all necessary operations are completed on the old version before it is released, allowing safe memory reclamation of the released version while preserving reliability.
Solution Approach 2:
The system discards old delta fragment versions by releasing latches on them after their data has been fully processed. This allows memory to be recovered and reused for new delta fragments, resolving the contradiction between maintaining data consistency (by holding latches) and reclaiming memory (by releasing latches).
2Productivity
If latches are released early to enable garbage collection, then memory reuse is improved, but data consistency may be compromised
Solution Approach 1:
The system performs preliminary actions by ensuring all query operations are completed on the old delta fragment version before releasing its latch. This preliminary completion of operations guarantees data consistency is maintained even after latch release, enabling safe early release for memory reuse without compromising reliability.
Solution Approach 2:
The system changes the state parameter of delta fragment versions from 'locked' to 'released' only after verifying that all operations are complete. This parameter change controls the timing of latch release to balance memory reuse efficiency with data consistency requirements.
3Productivity
If multiple versions of delta fragments are maintained to allow concurrent operations, then productivity is improved, but memory consumption increases
Solution Approach 1:
The system discards old delta fragment versions by releasing latches on them after their data has been fully processed by all operations. This allows memory to be recovered and reused, enabling multiple versions to be maintained for concurrent operations while controlling overall memory consumption through systematic release and reuse.
Solution Approach 2:
The system performs periodic latch release and version creation operations during query execution. This periodic action allows old versions to be released and memory reclaimed at regular intervals, enabling concurrent operations on multiple versions while controlling memory consumption through systematic periodic management.
Data Source
AI summary
A query is received at a database execution engine. The query is executed by the database execution engine, the executing including executing one or more threads that operate on versions of data of the delta fragment; acquiring and releasing, by a first thread, a latch on a first version of the data of the delta fragment, the releasing is performed prior to the first thread entering a wait state; and reclaiming the first version of the data of the delta fragment from memory and during execution of the query. Related systems, methods, and articles of manufacture are also described.