Distributed Storage Deduplication via Global Address Linking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed storage systems, the expansion of data storage space to accommodate increasing data volumes is costly and inefficient, particularly due to the need to manage and store duplicate data effectively.
Innovation Solution
A distributed storage device and method that separates data into chunks, translates addresses to global addresses, maps these addresses to storage nodes, and selectively stores or links identical data chunks to optimize storage space by eliminating duplicates and improving data management efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data storage space is expanded to accommodate increasing data volumes, then storage capacity is improved, but cost increases significantly
Solution Approach 1:
The patent merges duplicate data chunks by establishing links between identical data stored at different locations. Instead of storing multiple copies of the same data, the system combines them into a single stored instance with multiple reference links, thereby reducing total storage requirements while maintaining data accessibility.
Solution Approach 2:
The system creates logical copies through linking rather than physical duplication. When duplicate data is detected, the system establishes symbolic links or references to the existing stored data chunk, allowing multiple access points without actually copying the data bytes, thus saving storage space.
2Reliability
If duplicate data is stored to ensure data availability, then reliability is improved, but storage space efficiency deteriorates
Solution Approach 1:
The patent merges duplicate data chunks by establishing links between identical data stored at different locations. Instead of storing multiple copies of the same data, the system combines them into a single stored instance with multiple reference links, thereby reducing total storage requirements while maintaining data accessibility.
Solution Approach 2:
The system introduces an intermediary linking mechanism that connects multiple data requests to a single stored data chunk. This intermediary layer (the link structure) enables data availability and redundancy without requiring actual duplication of storage resources.
3Productivity
If data is divided into chunks and mapped to multiple storage nodes, then data management efficiency is improved, but system complexity increases
Solution Approach 1:
The patent segments data into manageable chunks that can be independently stored, managed, and linked across multiple storage nodes. This segmentation enables efficient data organization and facilitates the linking of duplicate chunks without requiring complex global data structures.
Solution Approach 2:
The linking mechanism serves multiple functions: it reduces storage redundancy, maintains data availability, enables efficient data retrieval, and supports data integrity verification. This multi-functional approach manages complexity by using a single versatile mechanism rather than multiple specialized systems.
Data Source
AI summary
The data storage device includes a separator configured to separate data requested to write by clients into data chunks, an address translator configured to translate first addresses generated by the data chunks into second addresses as global addresses, a storage node mapper configured to map the second addresses to a plurality of storage nodes, and a data store unit configured to select a target storage node among the plurality of storage nodes and store the data chunks in the target storage node. The data chunks include a plurality of data input/output unit blocks. If other data chunks that are the same with the data chunks are pre-stored in the plurality of storage nodes, the data store unit is configured to establish links between the same pre-stored data chunks and the second addresses, rather than stores the data chunks in the plurality of storage nodes.