HTML Feature Comparison for Malicious URL Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for detecting malicious web pages and domains are hindered by false positives, false negatives, and a reactive nature, as they rely on blacklists that struggle to differentiate between hacked websites and those intentionally created by malicious actors, and fail to alert owners of compromised sites effectively.
Innovation Solution
A system that analyzes elements of HTML files to determine if a URL is malicious by comparing features of a potentially malicious URL to its host URL, using a similarity value threshold to identify if the URL was created by a malicious actor, thereby distinguishing between hacked and intentionally malicious sites.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If blacklists are used to detect malicious URLs, then known malicious web pages can be blocked, but false positives and false negatives occur and the system is reactive
Solution Approach 1:
The system performs preliminary analysis of HTML file features before a URL is added to a blacklist. By proactively comparing features of potentially malicious URLs against their host URLs and calculating similarity values, the system can identify malicious sites before they cause harm, rather than reacting after blacklists are updated.
Solution Approach 2:
The patent replaces the mechanical blacklist approach with an automated feature comparison system. Instead of relying on manual blacklist maintenance, the system automatically retrieves HTML files, extracts features, compares them using similarity algorithms, and makes detection decisions, thereby improving both accuracy and response speed.
2Reliability
If blacklists are used to protect users, then known malicious sites can be blocked, but the system cannot differentiate between hacked websites and intentionally created malicious sites
Solution Approach 1:
The system analyzes local features of individual HTML files (such as head elements, meta tags, and content structure) rather than relying on global blacklist entries. By examining the specific characteristics of each URL's HTML file and comparing them to its host file, the system can accurately distinguish between legitimate hacked sites and intentionally malicious ones, reducing false positives.
Solution Approach 2:
The patent introduces an intermediary analysis layer between the URL and the blacklist. Instead of directly blocking URLs based on blacklist entries, the system uses HTML feature comparison as an intermediary step to make intelligent detection decisions, thereby improving differentiation accuracy and reducing false positives.
3Reliability
If HTML feature comparison is performed for every URL, then detection accuracy improves, but system complexity and processing time increase
Solution Approach 1:
The system extracts only the most relevant features from HTML files (such as head elements, meta tags, and key content structure) for comparison, rather than analyzing the entire HTML document. This selective extraction maintains detection accuracy while reducing processing complexity and time requirements.
Solution Approach 2:
The patent applies partial analysis by comparing only specific critical features of HTML files rather than performing a complete analysis. This approach provides sufficient detection accuracy for most cases while significantly reducing the computational resources and system complexity required.
Data Source
AI summary
Systems and methods are described for detecting compromised web pages and domains by analyzing of elements of hypertext markup language (HTML) files of a domain. In one embodiment, a security service receives a request including a potentially malicious uniform resource locator (URL) and retrieves a first HTML file to which the potentially malicious URL points and a second HTML file to which a host URL corresponding to the potentially malicious URL points. The security service determines whether the potentially malicious URL is a malicious URL by comparing features of the first HTML file to corresponding features of the second HTML file and when a similarity value resulting from the comparing is less than a threshold, then the security service concludes that the first HTML file was created by a malicious actor and responds to the request with an indication that the potentially malicious URL is a malicious URL.