Prefix Fingerprint Hashing for Fast File Prefix Verification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional hashing algorithms like MD5 and SHA reduce speed performance when determining if one file is a prefix of another, failing to efficiently identify prefix relationships in data files.
Innovation Solution
Generating a prefix fingerprint by calculating hash values from different data sections of a file, allowing for efficient comparison of files by selecting appropriate hash values based on the size of the second file, and storing these fingerprints in a dataset for quick lookup.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional hashing algorithms (MD5, SHA) are used to determine file prefix relationships, then file integrity and completeness can be verified, but processing speed deteriorates significantly
Solution Approach 1:
The patent divides a file into multiple sections and generates separate hash values for each section. Instead of computing a single hash for the entire file, the system creates section-based fingerprints that can be compared independently. This segmentation allows for faster prefix matching by comparing only relevant sections rather than processing the whole file, thus improving speed while maintaining verification reliability.
Solution Approach 2:
The patent pre-generates hash values for different file sections and stores them in a database before actual file comparison is needed. When a prefix relationship needs to be determined, the system retrieves pre-computed hashes and compares them directly, avoiding the need to re-hash entire files during the comparison process. This preliminary action significantly reduces processing time while maintaining accuracy.
2Measurement precision
If complete file hashing is performed to verify file identity, then accurate file matching is achieved, but time consumption increases
Solution Approach 1:
The patent extracts and compares only the necessary portion of file data - specifically, it generates hash values from file sections up to the size of the smaller file when checking prefix relationships. This extraction approach avoids computing hashes for unnecessary portions of larger files, reducing comparison time while maintaining matching accuracy for the relevant data portions.
Solution Approach 2:
The patent applies partial hashing by computing hash values only for sections of files that are necessary for prefix determination. When checking if file A is a prefix of file B, the system computes hashes only for the portion of file A that needs to be verified, rather than hashing the entire file A. This partial action reduces computation time while preserving the accuracy needed for prefix matching.
Data Source
AI summary
A system and method for prefix fingerprints for a first file or a first data object. A prefix fingerprint comprises a plurality of hash values. The hash values of the prefix fingerprints are typically generated starting at the same offset within the file or data object, but are generated based on different data sizes. Later, a second file or second data object can be compared with the first file or first data object to determine if the second file or data object is a prefix of the first file or data object. A hash value is selected from the previously determined prefix fingerprint of the first file based on the size of the second file. A hash is generated for the second file using the same offset value and size as was used to generate the selected hash value from the prefix fingerprint. The hash values are then compared.