Web Server URI Discovery via Configuration Parsing

Solution Overview

Problem

Existing methods for discovering Uniform Resource Identifiers (URIs) in web applications are inadequate, particularly in identifying unlinked URIs that are not reachable through traditional spidering techniques, leading to incomplete vulnerability assessments.

Innovation Solution

Evaluating web server configuration files to determine the existence and configuration of URIs, using collection agents to gather and parse configuration files from web servers, and generating URI discovery results that include combinations of hosts, ports, and paths.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If traditional spidering techniques are used to discover URIs, then the method is simple and widely applicable, but it fails to identify unlinked URIs that are not reachable through traditional spidering techniques

Engineering Contradiction:

ImproveURI discovery completenessVSAvoiddiscovery method complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The patent introduces configuration files as an intermediary medium to discover URIs. Instead of directly spidering through web pages, the system uses configuration files (such as Apache .conf files, Nginx config files, or IIS metabase files) as intermediaries that contain URI mappings and routing information. This intermediary approach enables discovery of unlinked URIs that are not accessible through traditional spidering techniques.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent replaces the mechanical spidering process with a configuration file parsing mechanism. Instead of using HTTP requests and link following (mechanical web crawling), the system substitutes this with configuration file reading and parsing operations. This substitution allows direct extraction of URI information from configuration data sources, bypassing the limitations of traditional spidering.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Measurement precision

If configuration files are evaluated to discover URIs, then unlinked URIs can be identified, but the process requires obtaining and parsing configuration files from web servers

Engineering Contradiction:

ImproveURI discovery completenessVSAvoidconfiguration file access difficulty

Core Design Contradiction:

Measurement precisionVSDifficulty of detecting and measuring

Solution Approach 1:

The patent implements self-service by having web servers automatically provide their own configuration files to the discovery system. The servers themselves reveal their URI configurations through their own configuration files, eliminating the need for external attackers to manually probe or guess URI structures. The configuration files are obtained through standardized protocols such as HTTP GET requests to specific paths.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The patent performs preliminary action by pre-configuring web servers to expose their configuration files through standardized access paths. Before the vulnerability assessment occurs, the configuration files are made available through predetermined URLs (such as /conf/, /config/, or server-specific paths), enabling the discovery system to retrieve and parse them without requiring complex authentication or authorization during the actual URI discovery process.

Inventive Principle:

Principle #10Preliminary action

3Measurement precision

If complete URI discovery is performed, then vulnerability assessment accuracy improves, but the time required for discovery increases

Engineering Contradiction:

Improvevulnerability assessment accuracyVSAvoiddiscovery time

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The patent performs preliminary action by pre-configuring web servers to expose their configuration files through standardized access paths. Before the vulnerability assessment occurs, the configuration files are made available through predetermined URLs (such as /conf/, /config/, or server-specific paths), enabling the discovery system to retrieve and parse them without requiring complex authentication or authorization during the actual URI discovery process.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent replaces the mechanical spidering process with a configuration file parsing mechanism. Instead of using HTTP requests and link following (mechanical web crawling), the system substitutes this with configuration file reading and parsing operations. This substitution allows direct extraction of URI information from configuration data sources, bypassing the limitations of traditional spidering.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS12554783B2Using discovered uniform resource identifier information to perform exploitation testing

Publication Date: 2026.02.17 RAPID7 INC

AI summary

A Uniform Resource Identifier (URI) discovery system is implemented that evaluates web configuration servers obtained from web servers to determine the existence and configuration of URIs hosted by the web servers. To discover URIs, the URI discovery system may obtain web server configuration files, and other metadata, from collection agents executing on web servers. The web server configuration files may then be parsed to evaluate the combinations of hosts, paths, and ports for the web server that may correspond to respective URIs. A URI discovery result may then be generated that describes the discovered URIs and includes configurations of the different URIs. The URI discovery result may be stored in an entry for the web server.