Write-Ahead Log Set Processing for Bulk Replay and Deletion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The process of replaying or deleting write-ahead logs (WALs) in a storage device based on key-value storage is complex due to the need to determine and process each WAL individually based on its execution result, leading to inefficiencies.
Innovation Solution
Organize WALs into WAL set key-value pairs with statuses (working, sealed, completed, and deleted states) to enable bulk processing, allowing replay or deletion at the set level, simplifying the operations by determining and processing WALs based on their set status.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If WALs are processed individually based on execution results, then the processing is accurate and reliable, but the process complexity increases and efficiency decreases
Solution Approach 1:
The patent merges multiple individual WAL processing operations into a single batch processing operation. By grouping WALs that share the same execution result status, the system performs replay or deletion operations on entire groups simultaneously rather than processing each WAL individually, thereby reducing process complexity while maintaining processing accuracy through status-based grouping
2Reliability
If WALs are processed individually based on execution results, then the processing is reliable, but the processing time increases
Solution Approach 1:
The patent combines multiple sequential WAL processing operations into parallel batch operations. By identifying WALs with identical execution results and processing them together in batches, the system significantly reduces total processing time while ensuring reliable processing through status-based grouping and controlled batch execution
3Difficulty of detecting and measuring
If WALs are organized in a tree map structure with individual execution results, then the data organization is structured and queryable, but the replay and deletion operations become complex
Solution Approach 1:
The patent introduces a grouping layer above the existing tree map structure that consolidates WALs with identical execution results into batches. This additional organizational layer enables efficient batch operations by allowing the system to identify and process groups of WALs simultaneously, reducing operation complexity while preserving the underlying structured data organization and queryability
Data Source
Figure 1
Figure 2~3
Figure 4
AI summary
This application provides a method and an apparatus for processing a write-ahead log. In embodiments of this application, a plurality of write-ahead logs WALs are recorded in a WAL set key-value pair, and the WALs recorded in the WAL set key-value pair are processed at a granularity of the WAL set key-value pair based on a WAL set key-value pair status recorded in a status key-value pair, thereby avoiding a prior-art problem that WALs can only be processed one by one, and helping simplify a process of replaying or deleting a WAL recorded in a storage device.