Distributed Storage Data Verification via Load Balancing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed storage systems, data verification processes often impair read/write performance at the user front end due to high bandwidth occupation and false positives caused by data inconsistency, especially during dynamic data changes.
Innovation Solution
A data verification method that employs load balancing and global traffic control to distribute verification tasks across storage nodes, reducing contention and resource overhead, and incorporates back-off retries to minimize impact on read/write operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data verification is performed on all replicas in distributed storage systems, then data consistency can be ensured, but network bandwidth and storage node resources are heavily occupied, causing performance degradation for user read/write operations
Solution Approach 1:
The patent segments the verification process by dividing data blocks into multiple slices and selecting only specific slices for verification rather than verifying entire replicas. This segmentation reduces the amount of data that needs to be read and transmitted during verification, thereby reducing network bandwidth occupation and storage node resource usage while still maintaining data consistency guarantees.
Solution Approach 2:
The patent applies partial action by performing verification on a subset of data slices rather than complete replicas. By selecting specific slices strategically, the system achieves sufficient verification coverage to detect inconsistencies without the overhead of verifying all data, thus balancing reliability requirements with performance constraints.
2Reliability
If verification requests are transmitted to storage nodes for entire replicas, then comprehensive verification can be achieved, but disk bandwidth resources are heavily occupied, causing glitches on user read request performance
Solution Approach 1:
The patent divides replicas into multiple slices and selects specific slices for verification instead of reading entire replicas. This segmentation significantly reduces disk bandwidth consumption during verification operations, minimizing interference with user read requests while maintaining adequate verification coverage to ensure data consistency.
3Reliability
If verification is performed during dynamic data changes, then data consistency can be checked, but false positives occur due to data transmission delays, leading to inaccurate conclusions
Solution Approach 1:
The patent performs preliminary actions by selecting verification slices and scheduling verification operations in advance, considering the dynamic state of data. By strategically choosing slices and timing verification operations, the system reduces the likelihood of capturing transient inconsistency states that would lead to false positives, thereby improving verification accuracy during dynamic data changes.
Data Source
AI summary
A method including determining a to-be-verified data block among a plurality of data blocks corresponding to a predetermined file in a distributed storage system, wherein a storage node where the to-be-verified data block is located comprises a storage node that meets a load balancing strategy in the distributed storage system; and verifying the to-be-verified data block. The present disclosure solves the technical problem in the conventional technologies in which the read/write performance at a user front end is affected during a data verification process.