Storage Network Memory Selection Using Dispersed Data Slices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face challenges in efficiently and securely storing and processing large volumes of data across distributed networks, particularly in ensuring data integrity and availability in the face of failures, while maintaining security and scalability.
Innovation Solution
A distributed computing system that employs dispersed storage and task processing units (DST) to encode data across multiple geographically dispersed locations, utilizing error correction schemes and secure encoding to ensure data integrity and availability, with management and integrity verification mechanisms to handle failures and hacking attempts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is stored in a distributed network across multiple locations, then data availability and reliability are improved, but system complexity increases
Solution Approach 1:
The patent segments data into multiple data slices that are distributed across different storage units in the network. Each slice is independently stored and can be retrieved, allowing the system to maintain high availability while managing complexity through modular data organization rather than monolithic storage management
Solution Approach 2:
The patent introduces encoding functions as intermediaries that transform original data into encoded data slices before distribution. These encoding intermediaries simplify the storage system by handling complexity in data transformation and regeneration, allowing storage units to work with standardized encoded formats rather than diverse data structures
2Reliability
If error correction schemes are implemented to ensure data integrity, then data reliability is improved, but processing time and computational resources increase
Solution Approach 1:
The patent applies error correction encoding in advance during the data storage process, creating redundant encoded slices before distribution. This preliminary encoding action ensures data integrity is built-in from the start, eliminating the need for time-consuming error detection and correction during data retrieval and processing operations
Solution Approach 2:
The patent transforms data into different parameter representations through encoding functions, converting original data into encoded data slices with specific mathematical properties. This parameter transformation enables efficient error correction through algebraic operations on the encoded parameters, reducing processing time compared to traditional error correction methods
3Object-affected harmful factors
If secure encoding is used to protect data from hacking attempts, then data security is improved, but system complexity and processing overhead increase
Solution Approach 1:
The patent extracts security functionality into separate encoding and decoding operations that are performed at the boundaries of the storage system. The core storage units handle only simple encoded data without security processing complexity, while dedicated encoding/decoding components handle security transformations, separating security overhead from storage operations
Solution Approach 2:
The patent creates multiple encoded copies of data through the encoding function, where each copy contains redundant information that contributes to both security and error correction. These encoded copies can be distributed and stored independently, providing security through redundancy without requiring complex encryption key management at each storage node
Data Source
AI summary
Methods and apparatus for preference based selection of storage network memory for data storage. In an example, a computing device receives a data object for storage in memory of the storage network and determines a system level storage efficiency preference associated with the data object. The computing device selects a set of storage nodes of a plurality of sets of storage nodes for storage of the data object based, at least in part, on the system level storage efficiency preference. The computing device further determines dispersed storage error encoding parameters for the data object, encodes the data object in accordance with the dispersed storage error encoding parameters to produce encoded data slices, and generates system addressing information for the encoded data slices.