Distributed Storage Repair via Execution Path Task Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
As data storage systems grow in size and complexity, the probability of storage device failure increases, leading to prolonged data restoration times and challenges in executing computing tasks efficiently, especially as the growth of data outstrips networking and computing capacity.
Innovation Solution
The implementation of a distributed storage system that employs an execution path for tasks, including repair tasks, based on task type and system characteristics, such as network congestion and storage device availability, to optimize task distribution and execution across multiple storage computers, utilizing erasure coding for data protection and recovery.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data storage systems increase in size to accommodate growing data, then storage capacity is improved, but data restoration time increases and system reliability deteriorates
Solution Approach 1:
The patent segments the data restoration process into multiple independent tasks that can be executed in parallel across different storage computers. Each task handles a specific portion of the restoration work, allowing simultaneous processing and significantly reducing overall restoration time despite increasing storage capacity.
Solution Approach 2:
The patent introduces a new dimension of parallel task execution by distributing restoration tasks across multiple storage computers simultaneously. This transforms the single-threaded restoration process into a multi-dimensional parallel process, enabling linear scaling of restoration speed with system size.
2Quantity of substance
If the number of storage devices increases to expand storage capacity, then storage capacity is improved, but the probability of storage device failure increases
Solution Approach 1:
The patent performs preliminary actions by proactively monitoring storage device health and initiating restoration tasks before complete failure occurs. The system detects early signs of device failure and automatically triggers parallel restoration processes, preventing data loss and maintaining system reliability despite increased device count.
Solution Approach 2:
The patent implements feedback mechanisms that continuously monitor storage device status and dynamically adjust restoration operations. When device failures are detected, the system receives feedback and automatically initiates appropriate restoration tasks, maintaining system reliability through real-time monitoring and responsive action.
3Quantity of substance
If data sets grow larger to meet enterprise needs, then storage capacity is improved, but task execution becomes more difficult due to limited computing capacity
Solution Approach 1:
The patent segments large data sets into smaller manageable chunks that can be processed independently by different storage computers. This segmentation reduces the computational burden on individual devices while maintaining the ability to handle large overall data sets through parallel processing.
Solution Approach 2:
The patent creates a universal task execution framework that can handle various types of storage tasks (restoration, repair, maintenance) across multiple storage computers with different capabilities. This multi-functional system simplifies task execution by providing a standardized approach that adapts to different device configurations.
Data Source
AI summary
Embodiments are directed towards managing the distribution of tasks in a storage system. An execution path for tasks may be generated based on the type of the task and characteristics of the storage system such that the execution path includes storage computers in a storage system. The tasks may be provided to each storage computer in the execution path. A working set of intermediate results may be generated on the storage computer in the execution path. If there is more than one storage computer in the execution path, working sets may be iteratively communicated to a next storage computer in the execution path such that the next storage computer employs a previously generated working set to generate a next working set until each storage computer in the execution path has been employed to generate a working set. The results may be stored on the storage computers.