Validation Framework for Runtime API Error Testing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing API systems face challenges in robustly handling and testing error conditions, particularly when calling external services, as they often require significant manual effort and resource allocation to identify and understand errors during development.
Innovation Solution
A validation framework is introduced that models potential failures in orchestrated transactions, allowing for the injection of error data and return codes without modifying client or server code, enabling automated and efficient error testing across distributed systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual error identification and testing is performed in existing API systems, then error handling robustness can be improved, but significant time and computational resources are consumed
Solution Approach 1:
The validation framework performs preliminary error modeling and return code generation before actual API calls are made. By pre-defining all possible failure scenarios and their expected return codes in a validation schema, the system prepares test conditions in advance, eliminating the need for time-consuming manual error identification during development and testing phases.
Solution Approach 2:
The system automatically generates return codes and error responses based on the validation schema without requiring manual intervention. The validation framework self-services by autonomously identifying error conditions, generating appropriate return codes, and verifying API behavior, thereby eliminating the need for developers to manually spend time understanding and documenting error scenarios.
2Reliability
If comprehensive error testing is implemented in distributed systems, then system reliability is improved, but the complexity of testing infrastructure increases
Solution Approach 1:
The validation framework is designed as a universal testing infrastructure that can validate multiple APIs across different services in a distributed system using a single unified schema format. The same validation framework handles error modeling, return code generation, and response verification for various API types (synchronous, asynchronous, streaming), eliminating the need for separate testing infrastructures for different API styles.
Solution Approach 2:
The system uses configurable parameters in the validation schema to control testing behavior, such as enabling or disabling specific error scenarios, adjusting timeout values, and modifying response expectations. By parameterizing the testing infrastructure, the system can adapt to different testing needs without requiring structural changes to the underlying framework, thus managing complexity through configuration rather than code changes.
3Productivity
If automated validation framework is deployed, then testing efficiency is improved, but the initial setup and configuration effort increases
Solution Approach 1:
The validation schema is prepared in advance with all possible error scenarios, return codes, and expected responses defined before deployment. This preliminary configuration of the validation framework allows for immediate automated testing upon deployment without requiring additional setup time, as the error modeling and test conditions are already established in the schema.
Solution Approach 2:
The validation framework uses templates and schemas that can be copied and reused across multiple APIs and services. Once a validation schema is created for one API, it can be replicated and adapted for similar APIs, significantly reducing the initial setup effort. The framework copies the validation logic and error modeling patterns across different service endpoints, minimizing repetitive configuration work.
Data Source
AI summary
Provided is a validation framework for modelling possible failures that might occur when an orchestrated transaction calls external services to ensure that error handling and reporting is robust and well designed. The disclosed techniques ensure that no changes are necessary to either the code making a call or the services that might be called. The techniques are not limited to web servers and REST APIs as they may be used to test and validate any kind of system that employs well defined APIs. The claimed subject matter, or “validation framework” may be added to an existing API or created as a new module that acts as a proxy server in a non-micro service type of system. Although described with respect to a gateway-API service, the claimed subject matter is equally applicable to other systems that process orchestrated transactions.