Blockchain Data Segmentation for Privacy and Throughput
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Centralized databases face issues such as single points of failure, high dependency on network connectivity, limited access to data, and inefficiencies in handling large datasets, which hinder secure sharing and implementation of smart contracts in blockchain networks.
Innovation Solution
A blockchain-based system that splits large files into chunks, transmits them to peer-to-peer data store nodes, and generates storage plans based on chunk locations, utilizing encryption and smart contracts for secure and efficient data management and access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a centralized database is used to store large datasets, then data management and control are simplified, but the system suffers from single points of failure, limited access, and bottlenecks that reduce reliability and productivity
Solution Approach 1:
The patent divides the centralized database into multiple distributed nodes across a peer-to-peer network. Each node stores portions of the dataset, eliminating the single point of failure while maintaining coordinated data management through consensus algorithms and distributed hashing tables.
2Reliability
If all data is stored on the blockchain to ensure security and transparency, then data integrity is improved, but network bandwidth is overwhelmed and transaction throughput drops significantly
Solution Approach 1:
The patent segments data into two categories: critical metadata and consensus information are stored on-chain for integrity, while bulk dataset portions are stored off-chain in a distributed file system. This segmentation allows high throughput for data access while maintaining blockchain security for critical operations.
Solution Approach 2:
The patent introduces a distributed hashing table and file identification system as intermediaries between the blockchain and off-chain storage. These intermediaries enable secure retrieval and verification of off-chain data without requiring the entire dataset to be stored on-chain, thus maintaining data integrity while improving throughput.
3Productivity
If cryptographic hashes of datasets are stored on blockchain to maintain performance, then transaction throughput is maintained, but secure sharing and smart contract implementation on large datasets become impossible
Solution Approach 1:
The patent introduces file identification systems and distributed hashing tables as intermediaries that enable smart contracts to reference and operate on off-chain data. These intermediaries provide secure access mechanisms and verification protocols, allowing versatile data sharing and smart contract implementation without storing full datasets on-chain.
Solution Approach 2:
The patent moves data storage from the blockchain dimension to a separate distributed file system dimension, while maintaining cryptographic links between the two dimensions. This dimensional separation allows high-throughput off-chain storage while preserving blockchain security and smart contract capabilities through hash references and verification mechanisms.
4Productivity
If multiple devices access the same data simultaneously in a centralized database, then data accessibility is improved, but bottlenecks and conflict risks increase
Solution Approach 1:
The patent segments data into distributed chunks stored across multiple peer-to-peer nodes. Each node can independently serve data requests, enabling simultaneous access from multiple devices without creating bottlenecks or conflicts at a central point, thus improving both accessibility and reliability.
Data Source
AI summary
An example operation may include one or more of receiving, by a file processing node, a document file identification (ID) from a file owner node over a blockchain, acquiring, by the file processing node, a file storage plan executable script and an encrypted symmetric key for the document, decrypting the symmetric key by the file processing node, and executing the file storage plan executable script using the decrypted symmetric key as an input.