Legitimate Website Detection Using Machine-Learned Domain Features
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Organizations face challenges in identifying legitimate websites and removing false positives from domain discovery analysis, particularly when dealing with common word seed domains, which result in numerous domain matches and consume significant computing resources.
Innovation Solution
A computing platform generates a baseline dataset of feature vectors for known legitimate domains, using machine learning algorithms to compare and classify unknown domains, analyzing features such as WHOIS registration, HTTP response size, website template, domain age, HTML tag structure, and text content to determine legitimacy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If automated domain analysis processes are executed to analyze large numbers of domains, then domain threat detection capability is improved, but computing resource consumption increases
Solution Approach 1:
The system performs preliminary actions by generating feature vectors for domains and storing them in a database before actual threat analysis is needed. This pre-computation allows the system to quickly compare new domains against the pre-generated feature vectors during threat detection, reducing real-time computing resource consumption while maintaining detection capability
Solution Approach 2:
The system creates copies of domain feature vectors and stores them in a database for future comparison. Instead of重新 analyzing each domain from scratch, the system uses these pre-generated feature vector copies to efficiently determine if new domains match known patterns, reducing repeated computing resource usage
2Adaptability or versatility
If common word seed domains are analyzed, then domain match coverage is improved, but false positive identification increases
Solution Approach 1:
The system applies local quality by analyzing specific local features of domains (WHOIS registration data, HTTP response characteristics, website template structure, domain age, HTML tag structure, text content) rather than relying solely on broad keyword matching. This localized feature-based analysis improves precision in determining whether a domain match is legitimate or malicious
Solution Approach 2:
The system changes parameters by transitioning from simple keyword-based domain matching to a multi-parameter feature vector approach. By evaluating multiple parameters (registration data, response characteristics, structural features, content analysis) simultaneously, the system reduces false positives while maintaining comprehensive domain match coverage
3Measurement precision
If comprehensive domain analysis is performed to identify legitimate domains, then false positive removal is improved, but processing time increases
Solution Approach 1:
The system performs preliminary action by pre-generating and storing feature vectors for domains in a database before analysis is required. This allows the system to quickly retrieve and compare feature vectors during false positive identification, reducing processing time while maintaining comprehensive analysis capability
Solution Approach 2:
The system extracts and stores essential domain features (WHOIS data, HTTP response characteristics, template structure, domain age, HTML tags, text content) into separate feature vectors in advance. This extraction and pre-storage enables efficient comparison and identification of legitimate domains without repeatedly processing the entire domain data during analysis
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
Aspects of the disclosure relate to identifying legitimate websites and removing false positives from domain discovery analysis. Based on a list of known legitimate domains, a computing platform may generate (605) a baseline dataset of feature vectors corresponding to the known legitimate domains. Subsequently, the computing platform may receive (610) information identifying a first domain for analysis and may execute (615-645) one or more machine learning algorithms to compare the first domain to the baseline dataset. Based on execution (615-645) of the one or more machine learning algorithms, the computing platform may generate first domain classification information indicating that the first domain is a legitimate domain. In response to determining that the first domain is a legitimate domain, the computing platform may send (650) one or more commands directing a domain identification system to remove the first domain from a list of indeterminate domains maintained by the domain identification system.