Perfect Hash Bit Vector for Storage Data Transfer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Efficiently transferring data from a source storage device to a remote storage system is challenging due to the need for frequent on-disk lookups to determine what data needs to be transferred, which consumes processing resources and time.
Innovation Solution
A method using a perfect hash function and perfect hash bit vector to efficiently determine required segment references, reducing the need for on-disk lookups by populating a perfect hash bit vector with a target index file and source index file, thereby minimizing the data that needs to be transferred.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional data transfer methods are used with frequent on-disk lookups, then data transfer can be performed, but processing resources and time are excessively consumed
Solution Approach 1:
The patent applies preliminary action by pre-computing and storing hash values of segment references in a perfect hash structure before the data transfer process begins. This allows the system to quickly determine which segments need transfer without performing time-consuming on-disk lookups during the actual transfer operation, thus resolving the contradiction between data transfer efficiency and lookup time
Solution Approach 2:
The patent replaces the mechanical on-disk lookup system with a computational hash-based lookup system. Instead of physically searching disk storage to identify segments for transfer, the system uses perfect hash functions to compute segment identifiers in memory, substituting mechanical disk access with faster computational operations, thereby improving productivity while reducing time loss
2Productivity
If traditional data transfer methods are used with frequent on-disk lookups, then data transfer can be performed, but processing resources are excessively consumed
Solution Approach 1:
The patent pre-computes hash values and organizes segment reference information in a perfect hash structure before data transfer. This preliminary preparation reduces the computational burden during the actual transfer process, allowing the system to efficiently determine which segments need transfer with minimal processing resources, thus improving productivity while reducing energy consumption
Solution Approach 2:
The patent replaces resource-intensive mechanical disk access operations with efficient computational hash-based lookups in memory. This substitution dramatically reduces processing resource requirements while maintaining or improving data transfer efficiency, as computational operations consume significantly fewer resources than physical disk access
3Loss of time
If perfect hash function is used to populate perfect hash bit vector, then on-disk lookups are minimized, but additional processing steps are required
Solution Approach 1:
The patent introduces a perfect hash bit vector as an intermediary data structure that bridges the source and target storage systems. This intermediary structure stores pre-computed hash values and enables rapid determination of segments requiring transfer, reducing on-disk lookup time while the complexity of hash processing is confined to the preliminary population of this intermediary structure
Solution Approach 2:
The patent changes the parameter representation of segment references by transforming them into hash values through perfect hash functions. This parameter transformation enables compact storage in the bit vector and extremely fast lookup operations, trading the complexity of hash computation for significant gains in lookup speed and reducing on-disk access requirements
Data Source
AI summary
A method for transferring data includes populating a perfect hash bit vector (PHV) using a perfect hash function (PHF) and a target index file to obtain a populated PHV, determining required segment references using the populated PHV and received segment references, providing the required segment references to a source storage device, and receiving segments corresponding to the required segment references from the source storage device.