Testing Service Control Response Filtering for Software Validation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Software testing often detects unimportant or random differences between modified/new and deployed software systems, making it challenging to assess meaningful changes before full deployment.
Innovation Solution
A testing service that intercepts client requests, processes them through both candidate and production systems, and compares responses to identify meaningful differences using control responses from authority or production stacks, allowing for filtering and scoring of detected differences.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If comparison tests are performed between deployed software systems and modified/new software systems, then software validation capability is improved, but detection of unimportant or random differences increases
Solution Approach 1:
The patent introduces control responses from authority stacks as an intermediary reference point. Instead of directly comparing candidate system responses with production system responses (which yields noisy results), the invention uses control responses as a mediator to filter out random variations. The control responses serve as a baseline that helps distinguish meaningful differences from insignificant variations, thereby improving measurement precision while maintaining validation capability.
Solution Approach 2:
The invention creates control copies of responses from authority stacks (production or alternative systems) to serve as reference benchmarks. These control responses are generated by submitting control requests to authority stacks and storing the responses for comparison. By using these copied control responses as a reference frame, the system can more accurately identify meaningful differences in candidate system responses without being misled by random variations.
2Measurement precision
If control testing with authority stacks is implemented, then meaningful difference detection is improved, but system complexity increases
Solution Approach 1:
The testing service is designed to perform multiple functions: it can submit requests to candidate stacks, authority stacks, and production systems; it can generate, store, and compare both candidate responses and control responses; and it can evaluate differences against control data. This multi-functional design consolidates what would otherwise require separate systems into a single testing service, managing the complexity through unified architecture while achieving precise meaningful difference detection.
3Measurement precision
If multiple control requests are submitted to authority stacks, then control response accuracy is improved, but processing time increases
Solution Approach 1:
The system performs preliminary actions by submitting control requests to authority stacks and storing control responses in advance, before they are needed for comparison. Control requests and their corresponding control responses are pre-generated and archived, so when candidate responses need to be evaluated, the control data is already available. This preliminary preparation reduces the time required during actual testing while maintaining the accuracy benefits of multiple control requests.
Data Source
AI summary
The techniques described herein provide software testing of a candidate software system. In some examples, a testing service compares at least one candidate response to at least a first control response to obtain one or more candidate test differences. The testing service may compare at least a second control response of the plurality of control responses to at least one of the first control response of the plurality of control responses or a third control response of the plurality of control responses to obtain one or more control test differences. The testing service may then analyze the one or more candidate test differences based on the one or more control test differences to generate an evaluation of whether one or more of the candidate test differences are due to differences between the candidate software system and the control software system that generated the first control response.