Email Thread Attachment Indexing via Hash Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing email systems incur high CPU utilization and large search index sizes due to the re-indexing of duplicate attachments in message threads, as each message in a thread is treated as a new object, leading to inefficient document indexing operations.
Innovation Solution
The system associates email messages with message threads using a thread identifier, indexing attachments only once per thread, and uses unique identifiers like hash codes to avoid re-indexing duplicates, allowing for efficient indexing and searching across all messages.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If each message in a thread is treated as a new object for indexing, then search coverage is complete, but CPU utilization increases and search index size becomes large
Solution Approach 1:
The patent merges duplicate attachments within the same message thread into a single indexed object. By detecting that attachments are duplicates through comparison with previously indexed attachments in the thread, the system combines them into one unique entry, thereby reducing CPU utilization and search index size while maintaining complete search coverage through the thread-level indexing structure.
Solution Approach 2:
The patent discards redundant indexing operations for duplicate attachments by identifying them through thread context and attachment content comparison. Instead of processing each duplicate attachment separately, the system recovers the already-indexed attachment representation and reuses it, eliminating unnecessary CPU processing while preserving search functionality.
2Reliability
If each message in a thread is treated as a new object for indexing, then search index is updated completely, but search index size becomes large
Solution Approach 1:
The patent merges duplicate attachments within the same message thread into a single indexed object. By detecting that attachments are duplicates through comparison with previously indexed attachments in the thread, the system combines them into one unique entry, thereby reducing search index size while maintaining complete search coverage through the thread-level indexing structure.
Solution Approach 2:
The patent segments the indexing process at the thread level rather than at the message level. By organizing attachments into thread-based groups and indexing them once per thread, the system creates a segmented structure that reduces redundancy while preserving the ability to search across all messages in the thread, thus reducing index size without sacrificing search completeness.
3Reliability
If duplicate attachments are re-indexed in each message, then search index is maintained up-to-date, but document index operations become redundant
Solution Approach 1:
The patent performs preliminary detection of duplicate attachments by comparing attachment content or identifiers against previously indexed attachments in the same thread before performing indexing operations. This preliminary action identifies duplicates in advance, allowing the system to skip redundant indexing operations while maintaining search index freshness by still processing new and modified attachments.
Solution Approach 2:
The patent discards redundant indexing operations for duplicate attachments by identifying them through thread context and attachment content comparison. Instead of processing each duplicate attachment separately, the system recovers the already-indexed attachment representation and reuses it, eliminating unnecessary CPU processing while preserving search functionality.
Data Source
AI summary
When a message having at least one attachment is obtained for indexing, it is indexed as N+1 separate documents, where N is the number of attached documents. If the message is part of a message thread, then information regarding the last message in the thread is retrieved, and search index attachment meta data for the last message is extracted. A unique identifier is computed for the newly obtained attachments, and used to search for matches in the attachments for the last message in the thread. If there is a match, then the newly obtained attachment is not indexed, but the unique identifier of the previously indexed matching attachment is added to a body index document for the new message. A unique identifier associated with the new message is also added to a list of parent identifiers associated with the attachment.