NPF-Coded Document Similarity Detection Without Content Disclosure
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for privacy-preserving document similarity detection, such as homomorphic encryption and multi-party secure computation, face practical difficulties and are not suitable for scenarios where document content needs to remain secret while similarity is measured, particularly in applications like plagiarism detection, duplicate submission control, and fraud detection.
Innovation Solution
The method employs non-prefix-free (NPF) coding as a transform to provide privacy by replacing document symbols with variable bit-length codewords, utilizing the normalized compression distance (NCD) similarity metric, which preserves document regularities and allows similarity detection without revealing content, differing from previous solutions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If homomorphic encryption and multi-party secure computation are used for privacy-preserving document similarity detection, then document content privacy is protected, but computational complexity and practical usability deteriorate
Solution Approach 1:
The patent extracts only the essential similarity detection functionality from the complex homomorphic encryption framework. By using NPF coding to transform documents into a form that preserves similarity metrics while removing unnecessary cryptographic overhead, the solution achieves privacy protection without the prohibitive computational complexity of full homomorphic encryption schemes.
Solution Approach 2:
The patent employs lightweight NPF coding transforms that are computationally inexpensive compared to homomorphic encryption. These transforms provide sufficient privacy protection for similarity detection tasks without requiring the heavy mathematical infrastructure of cryptographic schemes, making the solution practical for real-world deployment.
2Measurement precision
If document contents are made public for similarity detection, then similarity measurement accuracy is improved, but document privacy is compromised
Solution Approach 1:
The patent applies local quality by transforming only the necessary portions of documents (using NPF coding) to enable similarity detection while preserving the privacy of the underlying content. The transform maintains local similarity patterns needed for accurate measurement while obscuring the global document structure that would reveal content information.
Solution Approach 2:
The NPF coding transform acts as an intermediary between the original private documents and the similarity detection process. This intermediary representation allows accurate similarity measurement without direct access to the original content, bridging the gap between privacy protection and measurement accuracy.
3Productivity
If variable-length non-prefix-free codes are used for coding, then coding efficiency is improved, but unique decodability deteriorates
Solution Approach 1:
The patent inverts the traditional approach by deliberately using non-prefix-free codes that sacrifice unique decodability for the benefit of privacy protection and coding efficiency. Instead of prioritizing perfect reconstruction, the system accepts that the transform is lossy in terms of exact recovery but preserves similarity metrics, which is sufficient for the detection task.
Data Source
Figure 1~2
Figure 3
AI summary
This invention is related to a document similarity detection method in which non- prefix-free (NPF) coding of the input documents as a transform is used to provide the privacy. The method comprises the following steps, encoding the symbols of the documents with non-prefix-free coding scheme, obtaining the transformed documents, calculating the similarity score of the documents with calculating the normalized compression distance of the encoded documents