Private Document Similarity Detection via Blockchain and Hashing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems for detecting similar documents across multiple private repositories face challenges in security, privacy, and efficiency, particularly when relying on trusted third parties, as they can be vulnerable to pre-image attacks and lack forward secrecy, and existing PSI protocols are inefficient and difficult to implement.
Innovation Solution
A system utilizing a permissioned Blockchain architecture with smart contracts to securely detect similar documents across multiple private repositories without a trusted third party, using cryptographic hash functions and Bloom filters to ensure privacy and security, and enabling participants to search for documents similar to those they already possess without revealing repository contents.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a trusted third party is used to detect similar documents across private repositories, then document similarity detection can be performed, but security and privacy are compromised due to pre-image attacks and lack of forward secrecy
Solution Approach 1:
The patent introduces cryptographic hash functions as an intermediary mechanism that allows document similarity detection without requiring a trusted third party to access actual document contents. Each participant computes hashes of their documents locally, and these hashes are compared to identify duplicates without exposing the underlying sensitive data.
Solution Approach 2:
The patent replaces the mechanical trust-based system (where participants must trust the third party not to leak data) with a cryptographic system based on hash functions. This substitution provides mathematical guarantees of security and forward secrecy, eliminating the need for trust assumptions.
2Object-affected harmful factors
If cryptographic hash functions are used for document similarity detection, then security and forward secrecy are improved, but system complexity increases due to implementation difficulties
Solution Approach 1:
The patent segments the document similarity detection system into independent modular components: document hashing module, hash comparison module, and duplicate identification module. Each component performs a specific cryptographic operation, making the overall complex system manageable through clear separation of concerns.
Solution Approach 2:
The system enables participants to perform document hashing and similarity detection themselves without requiring complex centralized infrastructure. Each participant independently computes hashes of their documents and compares them with others, simplifying the overall system architecture while maintaining security.
3Manufacturing precision
If all participants submit their documents to a centralized third party, then complete document matching can be achieved, but processing time and computational resources increase significantly
Solution Approach 1:
The patent extracts only the essential identifying feature of documents (their cryptographic hashes) for comparison purposes, rather than transmitting and processing entire documents. This extraction maintains matching accuracy while dramatically reducing data transmission and processing requirements.
Solution Approach 2:
The patent transforms documents from their original large-size format into compact hash representations. This parameter change from full document data to fixed-length hash values enables efficient comparison and storage while preserving the ability to accurately identify duplicate documents.
4Measurement precision
If participants share their actual documents with each other for similarity detection, then accurate duplicate identification can be achieved, but data privacy and PII protection are compromised
Solution Approach 1:
The patent uses cryptographic hash copies of documents instead of the actual documents for comparison. These hash copies serve as digital fingerprints that enable accurate duplicate identification while containing no personally identifiable information or sensitive content from the original documents.
Data Source
AI summary
A system and method for efficient and secure private similarity detection for large private document repositories reduces the amount of trust that the participants need to give to a third party and detects malicious participants. One use of this system and method is the efficient and secure detection of similar documents across large private document repositories.