Website Vulnerability Crawling Prioritization for Critical Function Coverage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing vulnerability inspection systems often miss important functions of a website due to limitations on the number of webpages accessed, layers, and elapsed time during crawling, leading to incomplete inspections.
Innovation Solution
A system that utilizes computer processors to acquire webpages, determine inspection-requiring functions, identify executable operations, and prioritize operations based on relatedness to these functions, ensuring comprehensive crawling of critical functions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If upper limits are set on the number of webpages to be accessed, the number of layers, and the elapsed time during crawling, then the inspection process can be completed within resource constraints, but the crawling may stop before all pages are crawled, causing important functions to be missed
Solution Approach 1:
The system performs preliminary analysis of the webpage to identify important functions and operations before executing the crawling. By determining inspection-requiring functions and identifying executable operations in advance, the system prioritizes which pages and functions to crawl first, ensuring critical functions are captured even when page number limits are reached
Solution Approach 2:
The system continuously monitors crawling progress and compares it against identified important functions. When approaching resource limits, the feedback mechanism allows the system to adjust crawling priorities dynamically, focusing remaining resources on uncovered important functions rather than continuing sequential crawling of less critical pages
2Device complexity
If sequential access to webpages is performed by extracting links and forms from HTML data, then the crawling process is simple to implement, but the process may exceed time limits or resource constraints before completing comprehensive inspection
Solution Approach 1:
Before executing the crawling, the system analyzes the webpage structure to identify inspection-requiring functions and determines the priority order of operations. This preliminary planning creates a directed crawling path that avoids unnecessary pages and focuses on time-critical important functions, reducing total inspection time while maintaining simplicity in the crawling execution
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A vulnerability inspection server 10 according to an embodiment of the present invention suppresses missing of important functions in a vulnerability inspection of a website. The server 10 provides, to a user who operates a user terminal 30 communicatively connected to the server 10 through a communication network 20, a vulnerability inspection service in which a website 22 or the like specified by the user is inspected for vulnerabilities (flaws in terms of information security). During the crawling of the website 22, the server 10 determines an inspection-requiring function that requires an inspection for vulnerabilities on a webpage, determines the degrees of relatedness between respective multiple operations executable on the webpage and the inspection-requiring function, and executes an operation identified on the basis of the degrees of relatedness from among the executable operations. That is, the server 10 is able to execute the operation identified on the basis of the degrees of relatedness to the inspection-requiring function. This facilitates crawling of webpages corresponding to important functions.