Phishing Detection via URL Embeddings and HTML Encodings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for detecting phishing websites are inadequate in real-time detection and often result in high false positive rates, as phishing sites can quickly disappear and zero-day attacks exploit newly discovered vulnerabilities, making it challenging to maintain accurate lists of malicious URLs.
Innovation Solution
The application of machine learning and deep learning techniques, including transfer learning, to classify URLs and content pages as phishing or not, using URL feature hashes, HTML encodings, and image embeddings, allowing for real-time detection and reducing false positives through the use of ML/DL models that embed textual and visual contents of web pages.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional methods are used to maintain lists of malicious URLs, then detection accuracy may be maintained, but the system cannot keep up with rapidly disappearing phishing sites and zero-day attacks
Solution Approach 1:
The system performs preliminary analysis of URL features, HTML encodings, and image embeddings before phishing sites become fully operational or disappear. By pre-computing and storing these features, the system can quickly classify new URLs without needing to maintain extensive lists of known malicious sites, enabling rapid detection of zero-day attacks while maintaining accuracy.
Solution Approach 2:
Instead of maintaining and updating lists of malicious URLs, the system creates feature representations (copies) of URL characteristics, HTML structures, and visual elements. These feature copies can be rapidly compared against training data to identify phishing sites without requiring direct knowledge of specific malicious URLs, thus keeping up with rapidly changing threats.
2Measurement precision
If machine learning models process complete web page content, then detection accuracy improves, but processing time increases and real-time detection becomes difficult
Solution Approach 1:
The system extracts and processes only the most discriminative features from web pages: URL feature hashes, HTML encoding patterns, and image embeddings. By selecting only these key features rather than processing complete web page content, the system achieves high detection accuracy while maintaining fast processing speeds suitable for real-time detection.
Solution Approach 2:
The analysis is divided into separate segments: URL feature extraction, HTML encoding analysis, and image embedding processing. Each segment can be processed independently and in parallel, reducing overall processing time while maintaining comprehensive analysis for accurate phishing detection.
3Adaptability or versatility
If extensive lists of malicious URLs are maintained, then detection coverage improves, but the system becomes complex and difficult to update
Solution Approach 1:
The system transforms the problem from maintaining lists of malicious URLs to analyzing parameter-based features of URLs (hashes, encodings, embeddings). This parameter-based approach provides extensive detection coverage across diverse phishing sites while simplifying system maintenance, as the model learns general patterns rather than requiring updates for each specific malicious URL.
Data Source
AI summary
Disclosed is classifying a URL and a content page accessed via the URL as phishing or not. URL embedder extracts characters in a predetermined set from the URL to produce a character string trained using ground truth classification of the URL, producing a URL embedding. HTML parser accesses content at the URL and extracts HTML tokens from the content page. Further, HTML encoder, trained on HTML tokens extracted from content pages at example URLs, each example URL accompanied by a ground truth image captured from the content page accessed via the example URL, produces an HTML encoding of the tokens extracted from the page. Also, phishing classifier layers, trained on the URL embedding and the HTML encoding of example URLs, processes a concatenated input of the URL embedding and the HTML encoding to produce a likelihood score that the URL and content accessed via the URL presents a phishing risk.