Parallel File System Parity Writes During Storage Node Failures
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Failure of storage nodes in parallel file systems degrades performance by blocking file writes until the node is back online, leading to inefficiencies in high-performance computing environments.
Innovation Solution
A parallel file system that uses network-distributed erasure coding to split data into data and parity blocks, allowing writes to proceed even when a storage node is inaccessible, and reconstructs data using parity data when the node becomes available.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If storage nodes are used to store data in a parallel file system, then data redundancy and performance improvement are achieved, but write operations are blocked when a storage node becomes inaccessible
Solution Approach 1:
The patent segments data into multiple stripes distributed across different storage nodes, with each stripe having associated parity information. This segmentation allows the system to continue writing to accessible nodes even when some nodes are inaccessible, resolving the contradiction between maintaining data redundancy and preserving write performance during node failures.
Solution Approach 2:
The patent performs preliminary actions by pre-calculating and storing parity information for data stripes before write operations complete. When a storage node becomes inaccessible, the system can use this pre-computed parity information to reconstruct data on alternative nodes, allowing write operations to proceed without blocking while maintaining data redundancy.
2Stability of the object's composition
If data is written to inaccessible storage nodes, then data consistency is maintained, but write operations are blocked until the node is back online
Solution Approach 1:
The patent introduces an intermediary mechanism where parity information acts as a mediator between the inaccessible storage node and the write operation. Instead of blocking until the original node is available, the system uses parity data as an intermediary to reconstruct and store data on accessible nodes, maintaining data consistency while eliminating write blocking time.
Solution Approach 2:
The patent changes the state parameters of data storage by transitioning from requiring direct write access to the original storage node to using reconstructed data from parity information. This parameter change allows the system to maintain data consistency through mathematical reconstruction rather than direct node access, eliminating the time loss associated with waiting for node recovery.
3Adaptability or versatility
If parity data is stored on multiple storage nodes, then data reconstruction is enabled, but additional storage overhead is introduced
Solution Approach 1:
The patent uses copying by creating parity copies of data that can be used for reconstruction. Instead of duplicating entire data sets across multiple nodes, the system generates compact parity information that serves as a compressed copy, enabling data reconstruction while minimizing the storage overhead associated with maintaining data reconstruction capability.
Data Source
AI summary
A parallel file system (PFS) comprising a plurality of storage nodes. The PFS receives a request to write data in a file from an application running on a client system. The file is stored in a set of storage nodes. The PFS identifies a target storage node to which data needs to be written in response to the request to write data. The PFS detects that the target storage node is inaccessible. The PFS completes the request to write by writing parity data corresponding to the data being written to one or more other storage nodes. The PFS reconstructs the data of the target storage node when the target storage node is accessible.