Statistical Language Model for Email Attachment Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing email systems face high missed detection and false detection rates when attempting to determine if a user intended to attach a document, leading to user inconvenience and potential system disablement, as they rely on simple word spotting or predefined watch lists.
Innovation Solution
A statistical language model is used to analyze text strings in emails, computing probabilities based on stored values for sequences of words to determine if an attachment is intended, reducing false positives and negatives by adapting to user-specific writing patterns and integrating into conventional email programs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a simple word spotting approach with a predefined watch list is used to detect attachment references, then the system is easy to implement and fast, but the detection precision deteriorates resulting in high false detection rates and high missed detection rates
Solution Approach 1:
The patent changes the detection parameter from simple word spotting to statistical probability calculation. Instead of checking if words from a predefined list appear in the email, the system calculates the probability that a text string refers to an attachment by analyzing word sequences and their statistical patterns, thereby improving detection precision while maintaining reasonable system complexity
Solution Approach 2:
The patent replaces the mechanical word-matching system with a statistical language model that uses probability theory. The system substitutes the simple keyword search mechanism with a more sophisticated statistical analysis that evaluates the likelihood of attachment references based on learned patterns from training data, thus resolving the contradiction between simplicity and precision
2Reliability
If the watch list is expanded to include more words to reduce missed detections, then detection coverage improves, but false detections increase
Solution Approach 1:
The patent changes the detection approach from binary word matching to probabilistic assessment. By calculating the probability that a text string refers to an attachment based on statistical patterns of word sequences, the system can distinguish between true attachment references and false positives, thereby maintaining high detection coverage while reducing false detections
Solution Approach 2:
The patent introduces a statistical language model as an intermediary between the raw text and the detection decision. This intermediary component analyzes the statistical properties of word sequences and provides a probability score, which then determines whether to trigger a user prompt, effectively filtering out false positives while maintaining detection coverage
3Measurement precision
If a statistical language model is used to analyze text strings and compute probabilities, then detection precision improves reducing false positives and negatives, but device complexity and computational requirements increase
Solution Approach 1:
The patent applies preliminary action by pre-training the statistical language model on a corpus of email data before actual use. The model learns the statistical patterns of word sequences during this preliminary training phase, and during actual email processing, the system only needs to query the pre-learned probabilities, thereby reducing the computational complexity of real-time detection while maintaining high precision
Data Source
AI summary
A method for processing electronic mail includes computing a probability that a text string in an electronic mail message refers to an attachment as a function of a stored probability value for each of a plurality of sequences of words within the text string. Where the email message lacks an attachment, the method includes prompting a user if the computed probability indicates that the text string refers to an attachment.