Malicious URL Detection via HTML Layout and Content Relevance Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for detecting malicious URLs in Web pages are ineffective in identifying new threats and cannot prevent malware from being downloaded to user computers, as they rely on blacklists, crawling all Web sites, or simple regular expression matches, which fail to detect sophisticated malicious sites and newly formed threats.
Innovation Solution
A method that scans HTML code to determine layout, referring relationship, and content relevance features of embedded URLs, forming vectors that are input into a classifier algorithm to score the URL's maliciousness, intercepting or warning users about malicious URLs, and combining these analyses for enhanced detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If blacklist or simple RegExp matching is used to detect malicious URLs, then the detection process is simple and fast, but it cannot identify new threats at newly formed malicious sites
Solution Approach 1:
The patent transforms the detection approach from static blacklist matching to dynamic analysis by changing multiple parameters: analyzing HTML layout structure, examining referring relationships between pages, assessing content relevance, and evaluating URL characteristics. This multi-parameter analysis enables detection of new threats without sacrificing speed, as the system processes these parameters efficiently through automated algorithms rather than manual updates.
2Reliability
If crawling all Web sites is performed to find malicious sites, then comprehensive detection coverage is achieved, but the sheer magnitude of sites makes it impractical and time-consuming
Solution Approach 1:
The patent extracts and analyzes only the critical features from Web pages that indicate malicious content, rather than crawling and examining entire sites. By taking out key elements such as HTML layout patterns, referring relationship structures, content relevance metrics, and URL characteristics for analysis, the system achieves comprehensive detection coverage without the time cost of crawling all websites.
Solution Approach 2:
The detection process is segmented into multiple independent analysis components: HTML layout analysis, referring relationship analysis, content relevance analysis, and URL characteristic analysis. Each segment processes specific features independently and contributes to the overall detection decision, enabling efficient parallel processing and reducing total detection time while maintaining comprehensive coverage.
3Measurement precision
If sophisticated malicious sites are targeted, then detection accuracy improves, but the hacker can detect crawlers and take evasive action making detection difficult
Solution Approach 1:
The patent uses the legitimate Web page itself as an intermediary to detect malicious URLs. Instead of directly crawling malicious sites where hackers can detect and evade crawlers, the system analyzes the legitimate page's structure, its referring relationships, and content relevance to identify embedded malicious URLs. This intermediary approach maintains high detection accuracy while avoiding direct confrontation with sophisticated malicious sites.
Solution Approach 2:
The system leverages information already present in the legitimate Web page and its natural referring relationships to detect malicious content. By using self-contained features such as HTML layout patterns, internal linking structures, and content coherence metrics, the detection process does not require external crawling of malicious sites, thereby avoiding hacker detection and evasive actions while maintaining high identification accuracy.
Data Source
AI summary
Detection of malicious URLs in a Web page retrieved by a computer user is based in a backend security service or upon the user's computer. The HTML code download by the user is first scanned to detect any embedded links such as URLs found in frames or scripts. Features related to the layout of such a URL (position, visibility) are identified. Features related to the referring nature of the URL (page rank of parent, page rank of child) are identified. Features indicating the relevancy between the content of the parent Web page and the content of the Web page identified by the embedded URL identified. Each set of features is transformed into a binary vector and these vectors are fed into a decision engine such as a classifier algorithm. The classifier algorithm outputs a score indicating whether or not the suspect URL (and the Web page to which it links) is malicious or not. The user may be warned by a display message.