GPU-Based Duplicate Document Detection System
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Financial institutions face challenges in efficiently detecting duplicate financial documents, which can lead to fraud and costly errors due to the large volume of documents processed annually, requiring a method to quickly identify and prevent duplicate processing in real-time.
Innovation Solution
A graphics processing subsystem is utilized to maintain and search hash values representing financial documents, allowing for parallel processing and real-time detection of duplicate documents by generating and matching hash values for features of incoming documents against a stored database.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional database search methods are used to detect duplicate documents, then the system can maintain a collection of processed documents, but the search time increases significantly when dealing with billions of documents
Solution Approach 1:
The patent divides the large database into multiple blocks and uses a graphics processing subsystem to search these blocks in parallel. Each block can be searched simultaneously by different processing units, significantly reducing the total search time while maintaining comprehensive coverage of the entire document collection.
Solution Approach 2:
The patent transitions from sequential CPU-based search to parallel GPU-based search, adding a dimensional aspect of parallelism. The graphics processing subsystem executes multiple search operations concurrently across different memory blocks, transforming a one-dimensional sequential search into a multi-dimensional parallel search process.
2Reliability
If real-time duplicate detection is implemented using traditional methods, then fraud can be detected, but the processing speed is insufficient for high-volume document flows
Solution Approach 1:
The patent replaces traditional CPU-based mechanical search processes with a graphics processing subsystem that uses parallel architecture. This substitution enables much faster processing speeds while maintaining the same fraud detection capability, as the GPU can evaluate millions of documents simultaneously rather than sequentially.
Solution Approach 2:
The patent changes the processing parameter from sequential to parallel execution. By utilizing the graphics processing subsystem's parallel architecture, the system can process documents at a much higher rate while maintaining real-time detection capability, thus improving productivity without sacrificing reliability.
3Loss of information
If comprehensive document archiving is maintained to enable future searches, then historical records are preserved, but the storage and search burden increases with billions of documents
Solution Approach 1:
The patent segments the large-scale search system into multiple manageable blocks that can be processed in parallel. This segmentation allows the system to maintain comprehensive archiving while reducing the complexity of searching through billions of documents, as each block can be independently and concurrently searched.
Solution Approach 2:
The patent uses hash values as digital copies of document features to enable rapid identification of duplicates. Instead of searching through entire documents, the system compares hash value copies, which significantly reduces the search complexity while maintaining the ability to detect all duplicates in the archived collection.
Data Source
AI summary
A method and system are disclosed for identifying in real time duplicate financial documents processed by a financial institution or check clearinghouse. A collection of hash values representative of previously processed financial documents are maintained in a memory, such as a GPU memory. When a new financial document enters the financial institution or check clearinghouse for processing, one or more features of the financial document are captured. A hash value is generated from the one or more features of the financial document. A search is performed in the collection of hash values for a matching hash value. If a match is found, a potential fraudulent event or operational error may be indicated. If a match is not found, the hash value representative of the new financial document is added to the collection of hash values.