Erasure-Coded Data Placement in Linked Nodes for Reliable Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Computing devices face data loss due to persistent storage failures, and existing methods lack effective solutions for reliable data storage and recovery across multiple nodes.
Innovation Solution
A method involving nodes that receive data requests, divide data into chunks, generate parity chunks using erasure coding, and store them across multiple nodes using data protection domain (DPD) information, enabling dynamic data management and protection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is stored using traditional replication methods, then data reliability is improved, but storage efficiency deteriorates due to redundant data copies
Solution Approach 1:
The patent segments data into multiple chunks and distributes them across different nodes in the cluster. Instead of storing complete redundant copies of entire datasets, the system divides data into smaller pieces (chunks) that can be independently stored and recovered. This segmentation allows for more flexible data protection strategies that reduce overall storage requirements while maintaining reliability.
Solution Approach 2:
The patent employs erasure coding schemes that transform data into a different representation using mathematical parameters. By applying encoding algorithms, the system converts original data into encoded chunks and parity information that can reconstruct the original data. This parameter transformation enables efficient storage where fewer physical chunks are needed compared to traditional replication, improving storage efficiency while maintaining data reliability.
2Reliability
If data is divided into multiple chunks and stored across nodes, then data protection is improved, but system complexity increases
Solution Approach 1:
The patent implements self-service mechanisms where nodes automatically perform data chunking, encoding, and distribution without requiring complex external coordination. Each node independently handles its portion of data protection tasks, making intelligent decisions about where to store chunks based on local DPD information. This distributed self-service approach reduces overall system complexity compared to centralized management while maintaining robust data protection.
Solution Approach 2:
The patent introduces dynamic data protection domains (DPDs) that automatically adapt to changing node connectivity and availability. Rather than using static, pre-configured data protection groups, the system dynamically forms and reconfigures DPDs based on real-time network conditions and node status. This dynamic behavior simplifies system management by automatically handling complexity through adaptive reconfiguration rather than requiring manual intervention or complex fixed architectures.
3Reliability
If parity chunks are generated and stored with data chunks, then data recovery capability is improved, but transmission overhead increases
Solution Approach 1:
The patent applies local quality optimization by generating and storing parity chunks locally at data source nodes or nearby nodes within the same data protection domain. Instead of transmitting all parity information across the entire network, the system confines parity generation and storage to local contexts where data chunks are created. This localized approach significantly reduces transmission overhead while maintaining full data recovery capability within each DPD.
Solution Approach 2:
The patent performs preliminary parity chunk generation and local storage before data transmission occurs. By pre-computing parity information and storing it locally alongside data chunks, the system eliminates the need for subsequent parity transmission during data operations. This preliminary action reduces transmission overhead to minimal levels while ensuring data recovery capability is already in place before any data loss events occur.
Data Source
AI summary
In general, embodiments of the invention relate to storing data and managing the stored data in linked nodes. More specifically, embodiments of the invention relate to nodes linked together in a daisy chain configuration such as, but not limited to, a single-chain configuration and a dual-chain configuration, which use data protection domain (DPD) information to determine where and/or how to store the data.