Automatic Endpoint Discovery Through Two-Phase URL Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face challenges in automatically identifying variables and resources within URLs of API traffic, leading to complex processing and inflexibility in monitoring and communication with APIs, especially when endpoint changes occur.
Innovation Solution
A two-phase heuristic and algorithmic approach is employed to differentiate between variable names and resource names within URLs, using machine learning models to classify endpoints into a standard format, reducing processing complexity and enhancing flexibility.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If large amounts of different samples of the same endpoint are collected to identify variables properly, then variable identification accuracy is improved, but processing complexity increases
Solution Approach 1:
The endpoint identification process is segmented into two distinct phases: a first phase that identifies easy cases using heuristic approaches, and a second phase that handles hard cases using algorithmic approaches. This segmentation allows the system to process most endpoints efficiently while dedicating more resources only to difficult cases, thereby reducing overall processing complexity while maintaining accuracy.
Solution Approach 2:
The system performs preliminary classification of endpoints into easy and hard cases before applying the full algorithmic treatment. By identifying and handling easy cases upfront with simpler heuristic methods, the system avoids unnecessary complex processing for the majority of endpoints, reducing overall processing complexity while preserving identification accuracy.
2Measurement precision
If large amounts of different samples of the same endpoint are collected to identify variables properly, then variable identification accuracy is improved, but flexibility to changes in endpoints deteriorates
Solution Approach 1:
The system dynamically adapts its processing approach based on the characteristics of each endpoint. By classifying endpoints as easy or hard cases and applying appropriate processing methods, the system remains flexible to changes in endpoints while maintaining identification accuracy. The dual-phase approach allows the system to adjust its complexity level according to the specific endpoint being analyzed.
3Extent of automation
If AI machines autonomously produce path segments, then automatic endpoint identification is enabled, but difficulty in distinguishing variable fields from path segments increases
Solution Approach 1:
The system performs preliminary analysis of path segments to identify patterns and characteristics that distinguish variables from static path segments. By examining the structure and properties of autonomously generated path segments before final classification, the system reduces the difficulty of distinguishing variables even in fully automatic identification scenarios.
Solution Approach 2:
The automatic identification process is divided into phases that separately handle different aspects of endpoint analysis. The first phase uses heuristic approaches to identify obvious cases, while the second phase applies algorithmic methods for more complex distinctions. This segmentation makes the overall automatic identification process more manageable and accurate.
Data Source
AI summary
At least one processor may receive a plurality of uniform resource locator (URL) paths each comprising a respective one or more hierarchical path segments and divide each of the plurality of URL paths into tokens. The at least one processor may determine that at least one first hierarchical level of the plurality of URL paths represents at least one resource by performing a first statistical analysis and may determine that at least one second hierarchical level of the plurality of URL paths represents at least one variable by performing a second statistical analysis. The at least one processor may determine a standard format of the plurality of URL paths comprising the at least one resource and the at least one variable and perform processing utilizing the standard format for an application programming interface (API) associated with the plurality of URL paths.