Malicious URL Detection via HTTP Request Chain Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for detecting malicious web addresses, such as those using static crawlers and active crawlers, are inadequate in addressing newly emerged malicious websites due to their limitations in capturing complete web page content and context information, leading to inaccurate detection results.
Innovation Solution
A method and system that utilize a HyperText Transfer Protocol (HTTP) request chain, which is a sequential linked list of HTTP request-response interactions, to comprehensively analyze web page content and detect malicious web addresses, incorporating a crawler subsystem to acquire and analyze the HTTP request chain, and a detection subsystem to classify URLs as malicious or not.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If static crawlers are used to retrieve web page contents, then the detection process is simple, but the detection capability decreases sharply due to inability to handle encrypted content and dependent web page jumps
Solution Approach 1:
The patent replaces the mechanical static crawling approach with a browser-based rendering system that uses webkit or gecko engines to actually execute and render web pages, including handling encrypted content and dependent jumps through real browser execution rather than simple HTTP requests
Solution Approach 2:
The system changes the fundamental parameter of web page retrieval from static HTTP content download to dynamic browser rendering, transforming how web pages are processed from passive data extraction to active execution and visualization
2Reliability
If active crawlers with browser kernel wrapping are used, then web page rendering capability is improved, but the system complexity increases and cannot address newly emerged malicious websites
Solution Approach 1:
The patent segments the detection system into independent modular components: a crawler subsystem for URL acquisition, a rendering subsystem for web page execution, and a detection subsystem for malicious content identification, allowing each module to be optimized independently
Solution Approach 2:
The system employs a universal browser-based rendering engine that can handle multiple types of web pages including encrypted content, dependent jumps, and various web technologies, making the detection system adaptable to newly emerged malicious websites without requiring specialized handlers
3Reliability
If manual summarization of fixed components is performed, then detection engine protection is achieved, but the process is time-consuming and labor-consuming
Solution Approach 1:
The system performs automatic extraction and analysis of web page characteristics through programmatic processing of rendered pages, eliminating the need for manual summarization while maintaining detection effectiveness through automated pattern recognition and feature extraction
Data Source
AI summary
The present application provides a method and system for detecting malicious web addresses. The method includes: receiving a uniform resource locator (URL) reported by a user; acquiring a HyperText Transfer Protocol (HTTP) request chain associated with the URL, wherein the HTTP request chain is a sequential linked list including information about multiple HTTP request-response interactions during an access to the URL; and analyzing the HTTP request chain to determine whether the URL is a malicious web address. The technical solution of the present application can provide an accurate result of malicious web address detection, can detect various newly emerging malicious web addresses, and are user-friendly. The user only needs to upload the URL and does not need to provide any other information.