File Redundancy Detection Using Hash Pointers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing file redundancy detection tools are slow and ineffective in networked environments, particularly in cloud storage, as they fail to identify duplicate files across different users and locations, leading to wastage of storage space and bandwidth.
Innovation Solution
A server-based system that uses one-way hash identifiers to detect and remediate file redundancies by creating pointers to centrally-stored files, preventing duplicate uploads and allowing edits to be implemented on duplicates, thus preserving storage space and bandwidth.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional file redundancy detection tools are used, then file duplicates can be identified, but the detection process is slow and ineffective in networked environments
Solution Approach 1:
The system performs preliminary actions by computing hash identifiers for files during upload operations before actual storage occurs. This allows the server to proactively detect potential duplicates and prevent redundant storage, rather than detecting them after files are already stored. The hash computation and comparison happen in advance during the upload process itself.
Solution Approach 2:
The patent replaces traditional mechanical file comparison methods (comparing actual file contents byte-by-byte) with a computational hash-based identification system. By substituting the mechanical comparison process with cryptographic hash function computations, the system achieves much faster detection speeds while maintaining high accuracy in identifying duplicate files across the networked environment.
2Adaptability or versatility
If duplicate files are stored in different locations, then users can access files from multiple locations, but storage space and bandwidth are wasted
Solution Approach 1:
The system creates virtual copies or pointers to the same physical file storage location rather than actual duplicate file copies. When a file is uploaded and identified as a duplicate, the system creates a reference or pointer to the existing file instead of storing the actual file data again. This maintains the illusion of multiple locations while actually consuming minimal additional storage space and preventing redundant bandwidth usage during transfers.
Solution Approach 2:
The patent introduces an intermediary mechanism in the form of a file indexing and pointer management system. This intermediary layer sits between the user's file access requests and the actual storage locations, redirecting access to duplicate files through pointers to the canonical storage location. This mediator enables flexible file access while eliminating redundant data transmission and storage.
3Quantity of substance
If files are centrally stored with pointers in different locations, then storage space is optimized, but edit conflicts may occur between multiple pointers to the same file
Solution Approach 1:
The system implements feedback mechanisms that monitor and track edit operations across multiple pointers to the same file. When an edit is detected on a file accessible through one pointer, the system provides feedback to update or invalidate other pointers to maintain consistency. This feedback loop ensures that storage space optimization through centralization does not compromise edit reliability, as the system actively manages and coordinates changes across all access points.
Data Source
AI summary
Methods and systems for detecting and addressing file redundancies. A server may be configured to store a plurality of files associated with a plurality of identifiers. The identifiers may be generated using a hash function. The server may receive an indication of a first file. The indication of the first file may correspond to a user uploading the first file to the server, and may comprise a first identifier of the first file. The server may determine, by comparing the first identifier and the plurality of identifiers, that a second file is the same as the first file. Based on the second identifier, a pointer to the second file may be provided to prevent redundant storage of content similar to that of the second file. If an edit associated with a pointer, of a plurality of pointers to a centrally-stored file, is made, a new centrally-stored file may be created.