Partial Reconstruction of Unavailable Storage Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed storage systems face challenges with data availability and integrity due to increased chances of storage device failures, leading to temporary and permanent unavailability, which affects response time and increases the risk of data loss as the number of devices increases.
Innovation Solution
A method and system that detects unavailable storage devices, allows read or write requests to be fulfilled without fully reconstructing the device, and provides access to data sets once the device becomes available again, using data distribution across multiple devices and parity data for reconstruction, along with a journal for managing write transactions and a map data structure for tracking data locations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If the number of storage devices in a distributed system increases to accommodate larger data storage needs and improve parallel access performance, then data read and write speed improves, but the chances of storage device failures increase
Solution Approach 1:
The system performs preliminary actions by detecting when a storage device becomes unavailable and immediately initiating a partial reconstruction process that transfers only the specific data sets affected by the failure from alternative storage devices, rather than waiting for full device failure or performing complete system reconstruction
Solution Approach 2:
The invention segments the reconstruction process into partial reconstruction operations that handle individual data sets or portions of data, rather than requiring complete system-wide reconstruction. This allows the system to recover from device failures more efficiently by only reconstructing the necessary portions of data
2Reliability
If a storage device becomes unavailable in a distributed system, then data access to that device is lost, but full reconstruction of the device takes a long time affecting response time
Solution Approach 1:
The system applies partial action by performing only the necessary reconstruction of specific data sets that were stored on the unavailable device, rather than performing a complete reconstruction of the entire storage device. This partial reconstruction approach significantly reduces the time required while maintaining data availability
Solution Approach 2:
The system detects device unavailability and initiates partial reconstruction procedures as a preliminary response, allowing data access to resume quickly for affected data sets while the reconstruction process occurs in the background, rather than blocking data access until full reconstruction completes
3Productivity
If multiple storage devices are used to distribute data for improved performance, then parallel access is enabled, but the risk of permanent data loss increases due to multiple device failures
Solution Approach 1:
The system maintains copies of data sets across multiple storage devices in the distributed system. When a device fails, the data can be retrieved from alternative copies stored on other devices, preventing permanent data loss while maintaining the parallel access architecture
Solution Approach 2:
The system prepares for potential device failures by distributing data copies across multiple devices beforehand. This cushioning approach ensures that if one or more devices fail, the data remains protected and accessible through remaining copies, mitigating the increased risk associated with having multiple storage devices
Data Source
AI summary
In some embodiments, storage devices, such as a storage drive or a storage node, in an array of storage devices may be reintroduced into the array of storage devices after a period of temporary unavailability without fully rebuilding the entire previously unavailable storage device.