Stateful Email Detection With Schemaless Fragments for Conversation Context
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Email scanners are limited in their ability to detect malicious content due to their inability to analyze emails within the context of a conversation and historical relationships between senders and recipients.
Innovation Solution
A method that stores schemaless data fragments from previous emails in a datastore, indexed by derived keys, and uses these fragments to determine the suspiciousness of new emails by matching them with new emails, allowing for enhanced threat detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If emails are scanned individually without context, then scanning speed is maintained, but detection accuracy deteriorates due to lack of conversation and historical relationship context
Solution Approach 1:
The patent segments email data into distinct fragments including conversation context, historical relationships between senders and recipients, and individual email content. These segmented fragments are stored separately and retrieved as needed, allowing the system to provide comprehensive context without processing entire email histories simultaneously, thus maintaining detection accuracy while managing system complexity.
Solution Approach 2:
The system performs preliminary actions by pre-processing and storing email data fragments, conversation contexts, and relationship histories in advance in a datastore. When a new email arrives, these pre-prepared fragments are quickly retrieved and provided to the scanner, eliminating the need to re-process historical data and enabling accurate contextual analysis without increasing real-time scanning complexity.
2Measurement precision
If comprehensive email history is stored for all users, then detection accuracy improves, but data storage requirements and processing overhead increase
Solution Approach 1:
The patent extracts only the essential and relevant portions of email data into fragmented form - such as specific conversation contexts, relationship metrics, and key content elements - rather than storing complete email histories. This extraction approach provides sufficient context for accurate threat detection while significantly reducing the volume of data that must be stored and processed.
Solution Approach 2:
The system applies local quality by storing different types of data fragments with different levels of detail and retention periods based on their specific utility. Frequently accessed or highly relevant fragments are maintained with higher fidelity, while less critical data is summarized or archived, optimizing storage efficiency while maintaining detection accuracy where it matters most.
Data Source
AI summary
A computer-implemented method includes sending email scan requests to an email scanner. The method further includes receiving, from the email scanner, a verdict of suspicion and one or more data fragments. The method further includes storing the one or more data fragments for each email of the plurality of emails in a datastore. The method further includes receiving a new email. The method further includes deriving one or more new keys for the new email. The method further includes retrieving one or more matching data fragments from the datastore by matching the one or more new keys with the one or more keys stored in the datastore. The method further includes providing, to the email scanner, the new email and the one or more matching data fragments. The method further includes receiving a new verdict of suspicion and one or more new data fragments.