Distributed Data Storage with Metadata Repository
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current cloud storage systems face challenges in data availability, security, and transmission efficiency due to differences in cloud provider functions and interfaces, leading to reduced data security, increased storage costs, and performance limitations.
Innovation Solution
The method involves dividing data into blocks and storing them across multiple servers, generating metadata for block locations, and using a check block for data integrity and recovery, with metadata stored in a repository to facilitate efficient access and availability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is stored in a single cloud storage server, then storage simplicity is maintained, but data availability and security are reduced
Solution Approach 1:
The patent divides data into multiple data blocks and stores them across different cloud storage servers. Each data block is independently stored, allowing the system to retrieve data even if some servers are unavailable. This segmentation resolves the contradiction by improving data availability through distribution while maintaining manageable complexity through systematic organization of blocks and their metadata.
2Reliability
If data is divided and stored across multiple servers, then data security and availability are improved, but system complexity increases
Solution Approach 1:
The patent introduces a metadata repository as an intermediary component that stores information about data block locations, checksums, and recovery relationships. This metadata layer simplifies the complexity of managing distributed data by providing a centralized lookup mechanism that coordinates access across multiple servers without requiring complex peer-to-peer coordination between storage nodes.
Solution Approach 2:
The patent pre-generates check blocks and stores their metadata in the repository before actual data retrieval operations. These check blocks contain redundancy information that enables data recovery if original data blocks are lost. By preparing recovery mechanisms in advance, the system improves data security and availability while managing complexity through predetermined recovery protocols rather than ad-hoc solutions.
3Adaptability or versatility
If an abstraction layer is designed between cloud storage server and client, then unified data processing functions are provided, but data transmission efficiency is reduced
Solution Approach 1:
The patent extracts the metadata management functionality from the cloud storage servers and places it in a separate metadata repository. This allows the actual data storage and retrieval operations to proceed directly between clients and storage servers with minimal abstraction overhead, while still providing unified access through the metadata layer. The separation reduces the performance penalty of abstraction by minimizing the involvement of the abstraction layer in actual data transmission paths.
Data Source
AI summary
Embodiments of the present disclosure relate to a method, an electronic device, and a computer program product for storing and accessing data. A method for storing data includes: dividing, in response to receiving a request for storing data from a client, the data into a plurality of data blocks; storing the plurality of data blocks in a plurality of servers respectively; generating metadata of the data to record corresponding addresses for storing the plurality of data blocks in the plurality of servers; and storing the generated metadata in a metadata repository. The embodiments of the present disclosure can effectively improve data transmission efficiency, data availability, and data security in a cloud storage system.