Automatic Image Duplicate Detection Using Segmented Checksum and Fingerprint Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for detecting duplicate images in large collections are inefficient as they require user intervention and fail to account for subjective user perception of image similarity, leading to unnecessary computation time and incorrect duplicate identification.
Innovation Solution
A method that classifies transformed images based on user perception by determining fingerprint distance thresholds and associating actions with each classification zone, allowing for automatic management of images by distinguishing between identical, slightly modified, and largely modified copies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If fingerprint technique is used to detect near-duplicate images with various transformations, then the range of duplicate detection is improved, but the computation time increases significantly (up to 500 times slower than checksum)
Solution Approach 1:
The patent segments the duplicate detection process into two distinct stages: first using checksum for rapid filtering of exact duplicates, then using fingerprint technique only on the remaining non-duplicate images to detect near-duplicates. This segmentation allows the system to benefit from both methods without paying the full computational cost of fingerprinting all images.
Solution Approach 2:
The patent applies the computationally expensive fingerprint technique partially - only to images that were not already identified as duplicates by the faster checksum method. This partial application of the slower method reduces overall computation time while still achieving comprehensive duplicate detection.
2Loss of time
If checksum technique is used to detect exact duplicates, then the computation time is reduced, but the ability to detect near-duplicate images with transformations is lost
Solution Approach 1:
The patent divides the detection task into two segments: checksum handles exact duplicates (bit-by-bit identical), while fingerprint handles near-duplicates (visually similar but with transformations). Each method is applied to its appropriate segment of the problem space.
Solution Approach 2:
The checksum method acts as an intermediary filtering step that prepares the data for the more sophisticated fingerprint analysis. By first removing exact duplicates through checksum, the system reduces the workload for the fingerprint technique and ensures it focuses only on potentially interesting cases.
3Reliability
If the fingerprint technique is tuned to detect severe transformations, then the robustness of duplicate detection is improved, but the computation time increases and false positives may occur for images that users do not consider duplicates
Solution Approach 1:
The patent introduces dynamic, user-configurable thresholds that allow the fingerprint matching criteria to be adjusted based on user needs. Users can set minimum and maximum fingerprint distances to define their personal definition of duplicate, making the system adaptive rather than fixed.
Solution Approach 2:
The patent changes the parameters of the fingerprint technique by allowing users to configure distance thresholds. This parameter adjustment enables the same fingerprint algorithm to serve different user needs - from strict duplicate detection to more permissive near-duplicate detection - without changing the core algorithm.
4Measurement precision
If user intervention is required for each duplicate detection, then the accuracy of duplicate management is improved, but the automation level decreases and user time is consumed
Solution Approach 1:
The patent enables the system to serve itself by automatically applying user-defined rules to manage duplicates. Once users configure their preferences and thresholds, the system autonomously identifies and manages duplicates according to those preferences, reducing the need for continuous user intervention.
Solution Approach 2:
The patent incorporates feedback mechanisms where user responses to initial duplicate detections are used to refine and adjust the detection parameters. The system learns from user feedback and adapts its behavior, improving automation accuracy over time while reducing the frequency of user interventions needed.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present invention relates to the field of management of image data in data storage. In particular, the present invention relates to a method and device for automatic detection of duplicate images in data storage and corresponding device while taking into account user perception of what he considers to be duplicate images, which method and device are particularly efficient with regard to the personalized, automatic management of large amounts of image data.