Document Similarity Detection Using NPF Coding for Privacy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for privacy-preserving document similarity detection, such as homomorphic encryption and multi-party secure computation, face practical difficulties and do not effectively maintain the privacy of document contents while comparing similarities.
Innovation Solution
The use of non-prefix-free (NPF) coding as a transform method, which replaces document symbols with variable bit-length codewords, allowing for similarity detection using the normalized compression distance (NCD) metric without altering the document patterns, and providing privacy through lack of unique decodability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If homomorphic encryption and multi-party secure computation are used for privacy-preserving document similarity detection, then document privacy is protected, but computational complexity and practical implementation difficulties increase significantly
Solution Approach 1:
The patent extracts only the essential feature for similarity detection (normalized compression distance) while removing the complex cryptographic operations. Instead of using homomorphic encryption on full feature vectors, the method extracts a single scalar value (NCD) that preserves similarity information without requiring complex secure computation protocols.
Solution Approach 2:
The patent uses lightweight, computationally inexpensive operations (compression and distance calculation) instead of expensive cryptographic primitives. The NCD metric can be computed using standard compression algorithms without requiring specialized hardware or complex mathematical operations, making it practical for real-world deployment.
2Reliability
If document contents are encrypted to preserve privacy, then security is improved, but the ability to compute similarity metrics accurately deteriorates
Solution Approach 1:
The patent introduces NCD as an intermediary metric that bridges privacy protection and similarity detection. Instead of directly comparing encrypted document contents or features, the method computes NCD between documents, which preserves the similarity relationship while being computable on encrypted or private data without revealing the actual contents.
Solution Approach 2:
The patent transforms the similarity detection problem from operating on document contents or feature vectors to operating on a derived parameter (NCD value). This parameter transformation maintains the essential similarity information while enabling privacy-preserving computation, as NCD can be calculated without exposing the underlying document data.
3Measurement precision
If public comparison of document features is performed for similarity detection, then similarity accuracy is improved, but document privacy is compromised
Solution Approach 1:
The patent uses compression algorithms to create a compressed representation (copy) of the document that preserves the essential patterns and regularities needed for similarity detection. This compressed form can be shared or compared without revealing the original document contents, as the compression process discards unnecessary details while maintaining the core structure.
Data Source
AI summary
This invention is related to a document similarity detection method in which non-prefix-free (NPF) coding of the input documents as a transform is used to provide the privacy. The method includes the following steps, encoding the symbols of the documents with non-prefix-free coding scheme, obtaining the transformed documents, calculating the similarity score of the transformed documents with the normalized compression distance similarity metric.