Concurrent Raft Log Execution via Conflict Verification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The Raft algorithm, commonly used in distributed storage systems for ensuring data consistency, faces limitations in high-concurrency scenarios due to its strict serialization design, leading to reduced throughput and potential system bottlenecks when logs arrive out of order, causing missing logs and blocking the committing process.
Innovation Solution
The concurrent Raft algorithm introduces a log out-of-order replication mechanism that allows logs to be copied and committed concurrently, using a log hole array (LHA) to verify conflicts and ensure data consistency, enabling logs to be executed out of order within a defined active window without waiting for previous logs to be executed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the conventional Raft algorithm is used to ensure data consistency in distributed storage systems, then data consistency is guaranteed, but system throughput is reduced due to strict serialization design
Solution Approach 1:
The patent segments the log execution process into independent units that can be processed in parallel. By dividing the serialized log commit process into separate, concurrently executable segments with proper conflict detection, the system achieves both parallelism and consistency guarantees.
Solution Approach 2:
The patent introduces dynamic conflict verification mechanisms that adaptively determine whether logs can be executed in parallel. The system dynamically adjusts the execution strategy based on conflict detection results, allowing flexible parallel execution when safe and maintaining serialization when conflicts exist.
2Reliability
If logs are executed in strict order to maintain consistency, then data reliability is ensured, but execution efficiency is reduced due to blocking
Solution Approach 1:
The patent performs preliminary conflict verification checks before log execution to identify potential conflicts in advance. By checking for conflicts beforehand and only blocking when necessary, the system avoids unnecessary execution delays while maintaining reliability.
Solution Approach 2:
The patent allows logs to skip the sequential waiting process when conflict verification confirms no conflicts exist. Non-conflicting logs can be executed immediately without waiting for previous logs, significantly reducing execution delay while maintaining data reliability.
3Reliability
If the system waits for previous logs to be executed before processing new logs, then consistency is maintained, but concurrency is reduced
Solution Approach 1:
The patent introduces conflict verification mechanisms as intermediaries between log arrival and log execution. This intermediary layer checks for potential conflicts and determines whether parallel execution is safe, enabling the system to handle concurrent logs while maintaining consistency through selective verification.
4Reliability
If strict serialization is implemented to prevent conflicts, then data integrity is ensured, but system performance deteriorates in high-concurrency scenarios
Solution Approach 1:
The patent changes the execution parameter from strict sequential order to conditional parallel execution. By modifying the log execution parameter based on conflict verification results, the system maintains data integrity while achieving high performance in concurrent scenarios through parameter-driven execution strategies.
Data Source
AI summary
This application provides a log execution method and apparatus, a computer device and a storage medium and relates to the technical field of databases. The method includes circularly scanning a log execution active window, the log execution active window comprising a plurality of logs which are not executed, and all the logs before the log execution active window having been executed; acquiring a conflict verification result of a log in the log execution active window based on storage range information of the log, the storage range information indicating a storage range of the log and a storage range of a target number of logs before the log, and the target number being equal to a size of the log execution active window; and executing the log if the conflict verification result is no conflict.