Harmful Site Detection via Link Circulation Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing harmful site blocking techniques rely heavily on manual maintenance of databases, which is inefficient due to the constant emergence of new harmful sites and frequent changes in website content and addresses, making real-time analysis cumbersome and time-consuming.
Innovation Solution
A device and method that analyzes connections between harmful sites, normalizes URLs, and uses a database to store both harmful and non-harmful site URLs, allowing for efficient determination of harmful sites by creating a directed graph and calculating a harmful site rank based on statistical analysis, while removing non-harmful site URLs from consideration to reduce computation and enhance efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual maintenance of harmful site database is performed, then the database can be updated, but it is time-consuming and inefficient due to constant emergence of new harmful sites
Solution Approach 1:
The system automatically identifies and updates harmful sites through web crawling and link circulation analysis without requiring manual intervention. The harmful site collection device autonomously navigates web pages, extracts links, analyzes circulation patterns, and updates the database automatically, making the system self-maintaining and eliminating time-consuming manual updates while maintaining high accuracy through automated analysis algorithms
Solution Approach 2:
The system performs preliminary web crawling and link extraction to build a database of potential harmful sites before actual harmful site identification is needed. By pre-collecting and organizing link information, the system prepares data structures in advance that enable rapid automated analysis when new harmful sites emerge, reducing the time required for both database maintenance and real-time identification
2Measurement precision
If real-time analysis of website contents is performed, then harmful sites can be identified accurately, but the process is cumbersome and time-consuming
Solution Approach 1:
The system performs preliminary web crawling and link extraction to pre-process and organize web page information before actual harmful site analysis. By preparing data structures, extracting links in advance, and organizing circulation patterns beforehand, the system reduces the computational burden during real-time analysis, enabling both high accuracy through thorough analysis and high speed through optimized data structures
Solution Approach 2:
The system divides the harmful site identification process into separate modules: web crawling, link extraction, circulation analysis, and harmful site determination. Each module processes specific aspects independently, allowing parallel execution and optimized computation. This segmentation enables the system to maintain high accuracy through comprehensive analysis while improving productivity through efficient modular processing and reduced computational overhead
3Reliability
If all extracted URLs are analyzed for harmful sites, then no harmful sites are missed, but computational load increases significantly
Solution Approach 1:
The system extracts and removes non-harmful site URLs from the analysis pool by comparing extracted URLs against a pre-stored non-harmful site database. By filtering out known safe sites before进行深入分析, the system significantly reduces the number of URLs requiring computational analysis while ensuring no harmful sites are missed, as the filtering only removes previously verified non-harmful sites
Solution Approach 2:
The system performs partial analysis by focusing computational resources only on URLs that pass the initial non-harmful site filter and exhibit suspicious circulation patterns. Rather than analyzing every extracted URL equally, the system applies analysis selectively to URLs that require it based on their circulation characteristics, reducing overall computational load while maintaining complete detection of harmful sites through the circulation analysis mechanism
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Provided are a harmful site collection device and method for determining a harmful site by analyzing a connection between harmful sites. The harmful site collection device extracts a URL linked to a web page of a harmful site; checks a link circulation on the basis of link information on a web page of the URL linked to the harmful site to determine whether the web page of the URL linked to the harmful site is a harmful site; and, when a URL of a prestored non-harmful site is extracted while the link circulation is checked, stops checking the link circulation that includes the URL of the non-harmful site. Accordingly, the harmful site collection device can more easily determine a harmful site merely with information on a URL linked to a web page and can reduce the amount of computation using information on a URL of a prestored non-harmful site.