Webmail Detection via Statistical Field Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing network traffic analysis methods fail to effectively detect web-based email messages, as they rely on well-known email protocols and miss messages sent via HTTP by webmail clients like HOTMAIL and GMAIL.
Innovation Solution
A method and apparatus that extract web pages from network traffic, identify fields with shared document structures, and perform statistical analysis to detect email fields, allowing for the identification of web-based email messages regardless of the provider.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If network traffic analysis applications monitor well-known e-mail protocols (SMTP), then normal e-mail messages can be detected effectively, but web-based e-mail messages (webmail) sent via HTTP are missed
Solution Approach 1:
The system monitors multiple protocols (SMTP and HTTP) through a unified platform, enabling it to detect both traditional e-mail and web-based e-mail messages. The network traffic analysis application can identify e-mail fields within HTML web pages using statistical analysis, making the system versatile enough to handle various e-mail delivery methods without requiring separate specialized applications for each protocol.
2Measurement precision
If hard-coded applications are used for specific webmail providers, then detection accuracy for known providers is improved, but system complexity and resource costs increase
Solution Approach 1:
The system performs self-service detection by automatically identifying webmail messages through statistical analysis of fields in web pages. Instead of requiring hard-coded configurations for each provider, the application autonomously detects e-mail fields based on their statistical characteristics within the document structure, eliminating the need for manual provider-specific configuration while maintaining high detection accuracy.
Solution Approach 2:
The system changes the detection parameters from protocol-based (hard-coded provider configurations) to statistical-based (field distribution analysis). By analyzing the statistical properties of fields across web pages, the system can adapt to different webmail providers dynamically without changing its core structure, reducing complexity while maintaining precision.
3Adaptability or versatility
If statistical analysis of web page fields is performed to detect e-mail fields, then web-based e-mail detection capability is improved, but processing time and computational resources increase
Solution Approach 1:
The system applies partial action by performing statistical analysis only on specific fields that are likely to contain e-mail information, rather than analyzing every element in the web page. By focusing the analysis on fields with certain structural characteristics or positional patterns, the system reduces computational overhead while maintaining effective webmail detection capability.
Data Source
AI summary
Method and apparatus for detecting web-based electronic mail in network traffic is described. In some examples, web pages are extracted from the network traffic. Fields in each page of a group of the web pages that share a documents structure are identified. A statistical analysis of the fields of each page in the group of web pages is performed to identify any electronic mail (e-mail) fields. The group of web pages is indicated to include web-based e-mail messages if the fields of each page in the group of web pages include at least one e-mail field.