Erasure Code Partial Write-In Version Verification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems fail to correctly identify faults during partial write-in operations in erasure code-based distributed storage systems, leading to incomplete data recovery due to inability to differentiate between updated and unchanged data blocks.
Innovation Solution
The proposed method involves a data node and client communication to verify version information, process updating data based on matching version results, and trigger check data block updates to ensure accurate data recovery by maintaining version information and check increments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If partial write-in operation is performed without version information verification, then write-in speed is improved, but fault identification capability deteriorates
Solution Approach 1:
The system performs preliminary version information verification before executing the partial write-in operation. The data node compares the version information carried in the write-in request with the locally stored version information to determine whether the write-in should proceed, thereby preventing faults caused by outdated or inconsistent data states.
Solution Approach 2:
The system implements a feedback mechanism where the data node sends version information to the client, the client updates its local version information based on the server's response, and this updated version information is used in subsequent write-in operations. This closed-loop feedback ensures data consistency and enables proper fault identification.
2Reliability
If version information verification is added to partial write-in operation, then fault identification capability is improved, but system complexity increases
Solution Approach 1:
The system extracts version information as a separate, independent element from the data blocks. Version information is stored and transmitted independently, allowing the verification process to focus solely on consistency checking without complicating the core data storage and retrieval operations.
Solution Approach 2:
The version information mechanism serves multiple functions: it enables fault identification, ensures data consistency, supports retry logic, and provides a basis for determining whether partial write-in operations should proceed. This multi-functionality justifies the added complexity by delivering comprehensive reliability improvements.
3Stability of the object's composition
If complete EC group update is required, then data consistency is improved, but write-in efficiency deteriorates
Solution Approach 1:
The system performs only the necessary partial update of the EC group based on the version information match result. When version information matches, only the specific data blocks that need updating are modified rather than forcing a complete EC group rewrite, thereby maintaining data consistency while improving write-in efficiency.
Data Source
AI summary
An erasure code-based partial write-in method and apparatus are provided. According to the method, a data node receives a first updating request carrying updating data and first version information from a client, wherein the first version information is information stored by the client and indicating a version of a target data block to be updated with the updating data, a data amount of the updating data is less than a data amount of one erasure code group which comprises m data blocks and k check data blocks, both m and k being natural numbers; the data node acquires second version information in response to the first updating request, wherein the second version information is information stored in the node and indicating the version of the target data block; the data node processes the updating data according to a matching result of the first version information and second version information.