API Discovery via URL Schema Tree Clustering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current web application firewalls (WAFs) face challenges in accurately identifying network traffic when APIs are not provided or are incorrect, leading to potential denial of necessary traffic or allowance of malicious traffic.
Innovation Solution
The implementation of a URL/Schema tree-based API discovery system that monitors network traffic, generates a URL/Schema tree, and uses machine learning to reduce complexity by clustering and merging schema branches, enabling a network security appliance to determine the likelihood of malicious traffic.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a WAF uses an API to identify network traffic, then the accuracy of traffic identification is improved, but the system fails when the API is not provided or is incorrect, leading to denial of necessary traffic or allowance of malicious traffic
Solution Approach 1:
The system performs self-service by automatically discovering APIs through monitoring network traffic and generating URL/Schema trees without requiring external API providers. The WAF autonomously learns the expected traffic patterns for protected applications by analyzing actual traffic data, thereby resolving the reliability issue when external APIs are unavailable or incorrect.
Solution Approach 2:
The system implements feedback mechanisms by continuously monitoring network traffic, comparing it against discovered APIs, and using the results to refine and update the URL/Schema trees. This closed-loop approach allows the system to adapt to changing traffic patterns and improve identification accuracy over time while maintaining reliability through continuous validation.
2Measurement precision
If the WAF monitors and analyzes all network traffic to discover APIs, then the accuracy of traffic identification is improved, but the complexity of the system increases due to URL ambiguities and schema ambiguities
Solution Approach 1:
The system segments the complex task of API discovery into manageable components by creating URL/Schema trees that organize traffic patterns hierarchically. URLs are segmented into path components and schemas are broken down into data type definitions, allowing the system to handle complexity through structured organization rather than monolithic analysis.
Solution Approach 2:
The URL/Schema tree acts as an intermediary structure between raw network traffic and the final traffic identification decisions. This intermediate representation simplifies the complexity by providing a structured format that captures essential traffic patterns while filtering out ambiguities, making the system more manageable without sacrificing accuracy.
3Device complexity
If the WAF uses a simple traffic identification method, then the system complexity is reduced, but the ability to distinguish between legitimate and malicious traffic is compromised
Solution Approach 1:
The system adds another dimension to traffic analysis by incorporating schema validation alongside URL matching. Instead of relying solely on simple pattern matching, the system evaluates both the structural format (URL) and the data content (schema), creating a multi-dimensional identification approach that enhances detection capability while maintaining manageable complexity through the use of standardized schema formats.
Data Source
AI summary
An application programming interface discovery system is disclosed, including a processing resource to execute instructions to receive a tree structure, wherein the tree structure includes at least a first branch indicating a URL, a method, and a schema corresponding to a first network traffic, receive a second branch indicating a URL, a method, and a schema corresponding to a second network traffic, consider URLs and methods of a number of branches of the tree structure to as a cluster, wherein the cluster includes at least a subset of the URL and method of first branch and a subset of the URL and method of the second branch and merge at least some of the schema associated with the branches incorporated in the cluster, wherein at least the schema of the first branch is merged with the schema of the second branch.