Email Spam Filtering via Inference Detection and Search Hit Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional email and Website filtering technologies rely on keyword lists and blacklists, which become outdated due to constantly changing spam content and malicious Website information, making it difficult to effectively filter undesired content.
Innovation Solution
A system that uses term-frequency inverse-document-frequency (TF.IDF) analysis to extract keywords from emails and Web content, constructs search queries, and analyzes search engine hits to determine the relevance of content to restricted topics, enabling the detection of inferences and filtering of obscured undesired content.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If keyword lists and blacklists are used for filtering, then filtering capability is provided, but the filtering becomes outdated due to constantly changing spam content and malicious Website information
Solution Approach 1:
The system dynamically updates keyword associations by continuously analyzing Web corpora and search engine hits. Instead of static keyword lists, the system maintains living keyword associations that evolve with changing spam tactics and content, allowing the filtering system to adapt automatically to new patterns without manual intervention
Solution Approach 2:
The system uses feedback from search engine hit analysis to refine keyword associations. By monitoring which keywords produce relevant hits and which do not, the system continuously improves its understanding of spam patterns and updates its filtering criteria accordingly, creating a self-improving filtering mechanism
2Device complexity
If conventional keyword lists are used, then simple filtering is achieved, but the system cannot detect inferred topics and obscured content
Solution Approach 1:
The system introduces search engine hit analysis as an intermediary mechanism between keyword extraction and filtering decisions. This intermediary layer analyzes the contextual relationships between keywords and potential spam topics through search results, enabling the system to detect inferred meanings and obscured content without requiring complex direct analysis of email semantics
Solution Approach 2:
The system adds a new dimension to filtering by incorporating search engine hit data as an additional feature space. Instead of relying solely on keyword presence in emails, the system now operates in a multi-dimensional space that includes keyword associations, search hit counts, and contextual relationships, enabling detection of topics that are not explicitly stated
Data Source
AI summary
A system is provided to detect email spam. During operation, the system receives an email, extracts a set of keywords from the email body, and constructs a first search query based a keyword extracted from the email body. The system further constructs a second search query based on the keyword in the first query and one additional word which pertains to a known spam word or to the subject of the email. Next, the system receives a first number of hits and a second number of hits in response to the first and second search queries, respectively. The system then determines whether the email is spam based on the first number and the second number. The system can also perform Website filtering using inference detection which is based on search results received in response to search queries formulated with keywords extracted from Websites.