Probabilistic Email Filtering Using Network Resource Reputation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current threat detection systems struggle to accurately identify unwanted electronic mail messages, such as spam or those containing viruses, as they often appear legitimate and may contain hyperlinks or URLs that conceal their true nature, making it difficult for systems to determine whether they are spam or associated with threats.
Innovation Solution
A method and apparatus that utilize probabilistic analysis, including Bayesian analysis, to evaluate electronic messages by training a filter using network resource identifiers from whitelists and blocklists, determining reputation scores, and adding suspicious identifiers to a blocklist for messaging gateways to filter out spam or threats.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If typical analysis of message content is used, then the analysis process is simple, but the accuracy of determining whether a message is spam or associated with threats is insufficient
Solution Approach 1:
The patent segments the analysis into multiple independent components: analyzing message content, analyzing hyperlinks separately, analyzing URLs separately, and analyzing network resources. Each component contributes to the overall spam score independently, allowing the system to achieve high accuracy through cumulative evidence while maintaining modular complexity that is easier to manage and tune than a monolithic analysis system.
Solution Approach 2:
The patent extends the analysis from traditional single-dimension message content analysis to multi-dimensional analysis by adding hyperlink analysis, URL analysis, and network resource analysis as separate dimensions. This dimensional expansion allows the system to detect spam more accurately by examining the message from multiple angles simultaneously, with each dimension contributing additional evidence to the probabilistic assessment.
2Reliability
If probabilistic analysis with multiple analysis dimensions is used, then the accuracy of threat detection is improved, but the complexity of the analysis system increases
Solution Approach 1:
The patent implements feedback mechanisms where the results of analyzing hyperlinks, URLs, and network resources feed back into the overall spam score calculation. The probabilistic model continuously refines its assessments by incorporating feedback from each analysis dimension, allowing the system to achieve high reliability through iterative refinement while managing complexity through structured feedback loops rather than ad-hoc complex interactions.
Solution Approach 2:
The patent changes the parameter space by introducing multiple scoring dimensions (message content score, hyperlink score, URL score, network resource score) instead of relying on a single parameter. Each dimension has its own probabilistic parameters that are trained and evaluated independently, then combined to produce the final spam assessment. This parameter expansion increases reliability while organizing complexity into manageable, independently-tuned components.
3Measurement precision
If comprehensive analysis of hyperlinks and URLs is performed, then the ability to detect harmful links is improved, but the processing time and system resources increase
Solution Approach 1:
The patent performs preliminary analysis of hyperlinks and URLs by extracting and evaluating them separately before final message classification. By conducting preliminary assessments of network resources and incorporating their results into the probabilistic model early in the process, the system can make faster decisions about whether to flag messages as spam, reducing overall processing time while maintaining high detection precision through advance evaluation of critical elements.
Data Source
AI summary
A computer system can evaluate electronic messages based on probabilistic analysis, including Bayesian analysis. In one embodiment, a data processing system comprises logic configured for perform training a probabilistic filter using first properties of one or more first network resource identifiers obtained from a whitelist; training the probabilistic filter using second properties of one or more second network resource identifiers obtained from a blocklist; testing third properties of a third network resource identifier using the probabilistic filter, resulting in creating a probability output; and adding the third network resource identifier to the blocklist when the probability output is greater than a specified threshold. The blocklist can be communicated to messaging gateways in the field for local use in evaluating messages that contain spam or other threats.