Distributed Data Storage Pattern Using ECC and Server Reliability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage systems face inefficiencies in memory usage and security concerns when duplicating data across multiple servers, particularly when third-party servers are involved, as they can be memory-intensive and vulnerable to data access risks.
Innovation Solution
A system and method that utilize error-correcting code schemes to transform raw data into distributable data, which is then distributed across multiple servers in a pattern that maximizes system reliability and entropy, ensuring no single server has the entire data and allowing for data reconstruction and redistribution in case of server failure.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is duplicated in full on each individual backup server, then data reliability is improved, but memory usage becomes intensive and inefficient
Solution Approach 1:
The patent divides the data into multiple segments or portions and distributes them across different backup servers. Instead of storing complete copies of data on each server, each server holds only a portion, thereby reducing memory usage while maintaining the ability to reconstruct the full data set through error-correcting code schemes.
Solution Approach 2:
The patent transforms the data using error-correcting code schemes, changing the representation of the data from its original form to an encoded form that can be reconstructed. This parameter change allows the system to store fewer bits per server while ensuring data reliability through mathematical reconstruction algorithms.
2Quantity of substance
If portions of data are stored on each individual server to reduce memory use, then memory efficiency is improved, but system usefulness is reduced
Solution Approach 1:
The patent introduces error-correcting code schemes as an intermediary mechanism between the distributed data portions and the original data. This intermediary allows the system to reconstruct the full data set from the distributed portions, thereby maintaining system usefulness while achieving memory efficiency.
Solution Approach 2:
The patent replaces the mechanical approach of storing complete data copies with a mathematical approach using error-correcting codes. Instead of physically having full copies on each server, the system uses algebraic reconstruction methods to recover data, substituting physical redundancy with mathematical redundancy.
3Quantity of substance
If data is distributed across multiple third-party servers, then storage capacity is increased, but security risk increases due to potential third-party access
Solution Approach 1:
The patent segments the data into multiple portions and distributes them across different third-party servers. No single server holds the complete data set, which reduces the security risk associated with third-party access. Even if a third party gains access to one server, they only obtain a portion of the encoded data, not the complete original data.
Solution Approach 2:
The patent converts the potential harm of third-party access into a benefit by using error-correcting code schemes. The encoded portions stored on third-party servers are useless in their original form without the reconstruction algorithm, transforming the security vulnerability into a protective mechanism where the encoding itself prevents meaningful access to the data.
Data Source
AI summary
Described are a system, method, and computer program product for generating a data storage server distribution pattern. The method includes determining a set of servers and raw data to be stored. The method also includes transforming the raw data according to an error-correcting code scheme to produce distributable data. The method further includes determining a server reliability of each server in the set of servers. The method further includes generating the data storage server distribution pattern based on maximizing a system reliability relative to maximizing a system entropy. System reliability may be based on a minimum reliability of the set of servers, and system entropy may be based on a cumulated information entropy of each server of the set of servers. The method further includes distributing the distributable data to be stored across at least two servers of the set of servers according to the data storage server distribution pattern.