Domain Reputation Scoring via Machine Learning for Phishing Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face challenges in quickly and accurately identifying new, uncategorized websites as malicious or benign due to latency issues and inadequate threat intelligence coverage, leading to poor user experience and ineffective protection against evolving malicious sites.
Innovation Solution
A cloud-based system utilizing machine learning to determine domain reputation by analyzing various inputs, including lexical analysis, DomainRank, popularity, and historical Autonomous System Number (ASN)/WHOIS reputation, to provide a reputation score for new domains, enabling near real-time phishing site detection, Command and Control (C2) detection, and smart browser isolation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If threat intelligence feeds are used for reputation blocking, then known malicious sites can be blocked, but new malicious sites with short lifetime cannot be detected due to latency and insufficient coverage
Solution Approach 1:
The system performs preliminary analysis of domain features (lexical analysis, DomainRank, popularity, ASN/WHOIS reputation) before a user actually visits the site. By pre-evaluating these features and storing them in a database, the system can quickly determine reputation without waiting for actual site behavior to manifest threats, thus reducing detection latency for new malicious sites.
Solution Approach 2:
The patent introduces an intermediary machine learning model that sits between the raw domain data and the final reputation decision. This ML model processes multiple input features (lexical analysis, DomainRank, popularity, ASN/WHOIS) and synthesizes them into a unified reputation score, enabling fast and accurate detection of new malicious sites without relying solely on slow threat intelligence feeds.
2Reliability
If all new uncategorized sites are blocked, then malicious sites are protected against, but legitimate new sites are also blocked leading to poor user experience
Solution Approach 1:
Instead of applying a uniform block policy to all new sites, the system applies differentiated treatment based on local quality metrics. By analyzing specific domain features (lexical characteristics, DomainRank score, popularity, ASN/WHOIS reputation) individually, the system can identify and allow legitimate new sites while blocking only those with malicious characteristics, thus maintaining both security and user experience.
Solution Approach 2:
The system changes the parameter of decision-making from binary (block or allow) to continuous (reputation score from 0-100). This parameter change enables nuanced decision-making where sites with high reputation scores are allowed even if they are new and uncategorized, while only sites with low scores are blocked, thereby preserving user experience while maintaining protection effectiveness.
3Measurement precision
If detailed scanning and analysis of new uncategorized sites is performed, then accurate identification of malicious sites is achieved, but latency increases leading to poor user experience
Solution Approach 1:
The patent segments the analysis process into multiple independent components: lexical analysis, DomainRank evaluation, popularity checking, and ASN/WHOIS reputation assessment. Each component analyzes a specific aspect of domain characteristics in parallel, and their results are combined by the ML model. This segmentation enables comprehensive analysis without sequential delays, maintaining both accuracy and speed.
Solution Approach 2:
The system creates a simplified representation (copy) of the domain's reputational characteristics through feature extraction and storage in a database. Instead of performing full detailed scanning every time, the system queries pre-computed feature copies from the database, which provides accurate identification while significantly reducing the time required for analysis.
Data Source
Figure 1
Figure 2
Figure 3~4
AI summary
Systems and methods include receiving (402) a domain for a determination of a likelihood the domain is malicious or benign; obtaining (404) data associated with the domain including log data from a cloud-based system (100) that performs monitoring of a plurality of users (102); analyzing (406) the domain with a plurality of components to assess the likelihood, wherein at least one of the plurality of components is a trained machine learning model; and combining (408) results of the plurality of components to predict the likelihood the domain is malicious or benign.