Dispersed Storage Task Processing Without Redundant Data Copies
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data processing systems face challenges in efficiently managing large volumes of data across distributed networks, particularly in terms of storage reliability, security, and task execution, especially when dealing with significant data sizes and complex tasks, without the need for redundant copies.
Innovation Solution
A distributed computing system that employs dispersed storage and task processing units, utilizing error encoding and decoding techniques to securely store and process data across geographically diverse locations, ensuring data integrity and reliability, even in the presence of failures, while allowing for efficient task distribution and execution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is stored in a distributed network without redundancy, then storage efficiency is improved, but reliability deteriorates
Solution Approach 1:
The patent segments data into multiple data slices and distributes them across different storage locations in the network. This segmentation allows the system to store data efficiently without creating full redundant copies, while still maintaining reliability through the distributed nature of the slices.
Solution Approach 2:
The patent introduces an error correction code as an intermediary element that mediates between the data slices and the storage locations. This error correction code enables the system to recover from failures without requiring traditional redundant copies, thus improving storage efficiency while maintaining reliability.
2Reliability
If data is encoded with error correction, then reliability is improved, but processing complexity increases
Solution Approach 1:
The patent divides the error correction process into segments that correspond to data slices. Each slice is independently encoded and processed, which simplifies the overall processing complexity compared to applying error correction to the entire data set at once, while still maintaining data integrity.
Solution Approach 2:
The error correction code is designed to automatically detect and correct errors without requiring complex external intervention. The system self-manages the error correction process through predefined algorithms, reducing processing complexity while ensuring data integrity.
3Reliability
If data is distributed across geographically diverse locations, then availability is improved, but system complexity increases
Solution Approach 1:
The patent segments data into slices that are distributed across geographically diverse locations. This segmentation strategy improves availability by ensuring that data remains accessible even if some locations become unavailable, while the modular nature of slice management keeps system complexity manageable.
Solution Approach 2:
The patent creates a universal error correction mechanism that functions across all geographically diverse locations. This multi-functional approach allows the same error correction code to operate consistently across different locations, simplifying system management while maintaining high availability.
4Productivity
If complex tasks are processed in a distributed manner, then processing capacity is improved, but coordination overhead increases
Solution Approach 1:
The patent segments complex tasks into smaller sub-tasks that can be independently processed across distributed computing resources. This segmentation increases processing capacity by utilizing multiple resources simultaneously, while the modular task structure reduces coordination overhead compared to managing complex tasks as single units.
Data Source
AI summary
A computing device of a storage network is operable to select a number of storage and execution units from a plurality of storage and execution units of the storage network for a task based on a capability level associated with each storage and execution unit of the plurality of storage and execution units to produce a selected subset of storage and execution units, where the task is associated with data. Processing parameters for the data are determined based on the selected subset of storage and execution units. The data is processed in accordance with the determined processing parameters to produce slice groupings. The selected subset of storage and execution units executes the task based on the slice groupings.