Biased Random Test Generation for REST API Validation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional methods for testing Representational State Transfer (REST) APIs are limited in generating a range of tests to locate unknown deficiencies, often focusing on valid requests and failing to effectively traverse the spectrum of erroneous requests, which limits their ability to expose hidden or unexpected behaviors in systems under test.
Innovation Solution
A system and method for conducting model-based, biased, and random test generation through REST APIs, where the state of the system under test is continuously monitored and updated, biasing the generation of validation tests towards both valid and invalid requests, and using semantic checking to enhance testing robustness.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional testing methods focus on valid requests, then test generation is simpler and more predictable, but the ability to detect hidden or unexpected behaviors is reduced
Solution Approach 1:
The testing system dynamically adjusts the proportion of valid versus invalid requests based on test progress and system state. The test generation module transitions from static test sequences to dynamic adaptive testing, where the mix of request types evolves during execution to maximize defect detection while managing complexity.
Solution Approach 2:
The system changes key parameters of API requests including data types, value ranges, and structural properties to generate both valid and invalid requests. By systematically varying these parameters, the system expands test coverage to detect hidden behaviors without requiring completely separate testing frameworks.
2Adaptability or versatility
If random test generation is used to traverse erroneous requests, then coverage of error scenarios improves, but test predictability and reliability decrease
Solution Approach 1:
The testing system incorporates feedback loops where test results from previous iterations inform subsequent test generation. The system analyzes patterns in test outcomes and uses this feedback to adjust the random generation process, maintaining versatility in error scenario coverage while improving result consistency through learned patterns.
Solution Approach 2:
The system performs preliminary analysis of the API specification and system behavior patterns before executing random tests. This preliminary action establishes baseline expectations and constraints that guide the random generation process, ensuring that even random erroneous requests are grounded in realistic scenarios and produce consistent, interpretable results.
Data Source
AI summary
A system, program product, and method for validating a system under test (SUT). The method includes generating one or more application programming interface (API) requests. The method also includes selecting one or more random biases for one or more properties of the one or more API requests. The method further includes generating a random sample of one or more values from an input domain space, wherein the one or more values are associated with one or more respective fields of the API being requested.