Blockchain Metadata and Distributed Storage for Big Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing big data sharing methods require expensive distributed storage systems and security measures, making them unsuitable for large-scale data management and integrity verification, especially with the blockchain technology which is inefficient for storing large amounts of data due to its resource-intensive hashing processes.
Innovation Solution
A method and device for sharing big data using a blockchain, where metadata is stored on a blockchain network and actual data is stored in a distributed storage system, allowing for secure and efficient sharing by utilizing a blockchain network to manage metadata and a distributed storage to store data, with nodes serving multiple roles in both systems for integrity and accessibility.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If blockchain technology is used to store and verify big data, then data integrity and security are improved, but system resource consumption and processing time increase due to hash calculation and verification processes
Solution Approach 1:
The patent segments the data storage system into two parts: metadata stored on the blockchain and actual big data stored in distributed storage. This segmentation allows the blockchain to only store references and verification information, significantly reducing its storage burden and resource consumption while maintaining data integrity through cryptographic hashing.
Solution Approach 2:
The patent introduces distributed storage as an intermediary layer between the blockchain and the actual data. The distributed storage system handles the heavy lifting of storing and retrieving large datasets, while the blockchain serves as a trust layer that verifies data integrity through hash comparisons, thus mediating between security requirements and resource efficiency.
2Reliability
If blockchain technology is used to manage big data, then data security and trust are enhanced, but data storage efficiency and accessibility deteriorate due to the resource-intensive nature of blockchain operations
Solution Approach 1:
The system segments functionality between blockchain and distributed storage: the blockchain stores only metadata (hashes, timestamps, ownership information) while distributed storage handles actual data operations. This segmentation enables high-speed data access and efficient storage while the blockchain provides security guarantees for the stored data.
Solution Approach 2:
The patent uses cryptographic hashing to create a compact copy of data characteristics (hash values) that are stored on the blockchain. These hash copies serve as verification tokens that prove data integrity without requiring the actual data to be replicated across the blockchain network, thus maintaining security while improving storage efficiency.
3Ease of operation
If a single service provider maintains a distributed storage system, then data management is centralized, but system cost and complexity increase due to the need for expensive security systems
Solution Approach 1:
The patent makes blockchain nodes multi-functional by enabling them to participate in both the blockchain consensus mechanism and the distributed storage system. Each node can store data locally, verify data integrity through hashing, and participate in consensus, thus eliminating the need for separate dedicated security infrastructure and reducing overall system complexity.
Solution Approach 2:
The distributed storage system implements self-verification through cryptographic hashing, where each node can independently verify data integrity without requiring centralized security checks. The system uses its own distributed architecture to provide security services, eliminating the need for expensive external security systems and reducing operational complexity.
Data Source
AI summary
Provided are a device and method for sharing big data using a blockchain. The method of sharing big data using a blockchain includes receiving a search condition from a user terminal, searching a blockchain network for metadata meeting the received search condition and acquiring found metadata, acquiring shared data stored in a distributed storage using the acquired metadata, and transmitting the acquired shared data to the user terminal.