Document Fingerprinting via Inverted Index for Copy Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge lies in effectively identifying and preventing unauthorized copying and distribution of proprietary digital documents, particularly in enterprises, where traditional methods like PrintScreen, camera capture, and optical character recognition (OCR) pose significant risks due to inefficiencies in document comparison and fingerprinting.
Innovation Solution
A method and system for document fingerprinting that extracts unique text content and geometric features from formatted documents, using n-grams and inverted indices to compare and identify similarities, thereby detecting unauthorized copies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional document comparison methods (sequential iteration through forward index) are used, then document matching can be performed, but the time, memory, and processing resources required are prohibitively high
Solution Approach 1:
The patent inverts the traditional forward index structure by creating an inverted index that maps words to documents instead of documents to words. This inversion allows the system to directly locate documents containing specific words without sequentially iterating through all documents, dramatically reducing query processing time while maintaining matching accuracy.
Solution Approach 2:
The patent segments the document comparison process into distinct phases: extracting salient words from documents, building an inverted index structure, and performing efficient queries on this segmented structure. This segmentation enables the system to handle large document collections by breaking down the complex comparison task into manageable, optimized components.
2Reliability
If traditional document comparison methods are used, then document similarity can be detected, but the processing resources and memory requirements are not technically realistic
Solution Approach 1:
The patent extracts only the essential elements needed for document comparison by identifying and extracting salient words from documents. This extraction process removes unnecessary data while retaining the critical information needed for reliable unauthorized copy detection, significantly reducing memory and processing resource requirements.
Solution Approach 2:
By inverting the index structure to map words to documents, the system reduces the complexity of resource requirements. The inverted index allows for efficient storage and retrieval operations, making the system's resource requirements technically realistic while maintaining high reliability in detecting unauthorized copies.
3Measurement precision
If comprehensive document fingerprinting is performed on large document collections, then unauthorized copying can be detected with high confidence, but the time and computational resources required are prohibitively high
Solution Approach 1:
The patent performs preliminary actions by pre-processing documents to extract salient words and pre-building the inverted index structure before actual comparison operations. This preliminary preparation enables subsequent queries to execute rapidly, maintaining high detection confidence while significantly improving overall document processing throughput.
Solution Approach 2:
The inverted index structure enables efficient productivity by allowing the system to quickly locate and compare only relevant documents rather than processing the entire collection. This inversion maintains high copying detection confidence while dramatically improving processing throughput by reducing the scope of required comparisons.
Data Source
AI summary
A method for comparing documents, including extracting, by a computer processor, a plurality of extracted elements from a first image of a first formatted document, wherein each of the plurality of extracted elements corresponds to a text element of the first formatted document, extracting, by the computer processor, a first plurality of text fingerprints from a sequence of the plurality of extracted elements to form a first text feature of the first image, comparing, by the computer processor, the first text feature and a second formatted document to generate a comparison result, and determining, in response to the comparison result meeting a pre-determined criterion, that each of the first formatted document and the second formatted document contains common text content.