Image File Duplicate Detection via Hash Comparison
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Image files often result in storage duplication in non-volatile data storage of computing devices due to their large size and format variations, leading to inefficient use of storage space and processing resources.
Innovation Solution
A method involving hash value comparisons to identify duplicate image files, where received image files are checked against existing hash values in non-volatile storage, and if a match is found, the file is not saved, with optional conversion to a common format for further comparison to ensure only unique files are stored.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If image files are saved to non-volatile storage without duplicate checking, then image retrieval speed is improved, but storage space is wasted due to duplicates
Solution Approach 1:
The system performs preliminary hash value calculation and duplicate checking before saving image files to non-volatile storage. By computing the hash value of each received image file and comparing it with existing hash values in the duplication reference list, the system prevents duplicate files from being saved, thereby optimizing storage space while maintaining fast retrieval capability.
2Quantity of substance
If image files are converted to common format before storage, then storage efficiency is improved, but processing time increases due to decoding and conversion
Solution Approach 1:
The system performs preliminary hash value calculation on the received image file before conversion to common format. By checking the hash value against existing entries in the duplication reference list, the system determines whether conversion is necessary. This preliminary check avoids unnecessary decoding and conversion operations, reducing processing time while maintaining storage efficiency.
3Productivity
If hash value comparison is performed on original image format, then processing speed is improved, but storage duplication occurs due to format variations
Solution Approach 1:
The system segments the image file processing into two stages: first, calculate hash value of the received image file in its original format; second, if no duplicate is found, convert to common format and calculate hash value again. This segmentation allows the system to perform fast hash comparison on original formats while preventing storage duplication through subsequent format normalization.
Solution Approach 2:
The system introduces a duplication reference list as an intermediary mechanism that stores hash values of image files. This intermediary structure enables efficient comparison between received image files and existing stored files, allowing the system to identify duplicates across different formats while maintaining processing speed.
Data Source
Figure 1~2
Figure 3
AI summary
A hash value of an image file received at a computing device (10) is calculated and compared with hash values of other image files that are saved at the computing device (10). If the hash values are the same, the received image file is not saved at the computing device (10). If the hash values are not the same, the received image file is decoded or converted to a second format, and a hash value of the second format version of the received image file is compared with hash values of second format versions of other image files that are saved at the computing device (10). If the hash values of the second format versions of the image files are the same, then the received image file is not saved to the computing device (10); otherwise, the received image file is saved to the computing device (10).