Decoupling Test Verification from Execution via Data Correlation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional application testing procedures tightly couple action steps with verification steps, leading to inefficiencies such as re-running test executions when verification changes, difficulty in correlating outcomes with system events, and maintenance challenges, especially when verifying behaviors across different levels of an application under test.
Innovation Solution
The system separates test verifications from test executions by capturing and correlating data points during test executions, storing them in a directed acyclic graph structure, and executing queries to verify actual behavior independently of test execution states, allowing for decoupled verification and execution processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If test verifications are tightly coupled with test executions, then verification can be performed during test execution, but test executions must be re-run when verification changes even if action steps remain the same
Solution Approach 1:
The patent segments the testing process into two independent components: action steps that manipulate the AUT and verification steps that check outcomes. By separating verifications from executions, verification changes can be made without re-running action steps, eliminating wasted test execution time while maintaining verification reliability through independent verification management.
Solution Approach 2:
The patent captures and stores data points during test executions for later verification. This preliminary capture of execution data allows verifications to be performed independently later, so when verifications change, only the verification logic needs updating rather than re-executing the entire test, thus preserving verification accuracy without time loss.
2Reliability
If test verifications are tightly coupled with test executions, then verification can be performed during test execution, but verification of behaviors across different application levels becomes difficult
Solution Approach 1:
The patent introduces data points as an intermediary between test executions and verifications. These data points capture information from multiple application levels (UI, business logic, database) during execution and make it available for cross-level verification. This intermediary layer simplifies complex cross-level verification by providing a unified data interface without increasing overall system complexity.
3Reliability
If test verifications are tightly coupled with test executions, then verification can be performed during test execution, but maintenance becomes challenging when actions and verifications change
Solution Approach 1:
By segmenting verifications from executions, the patent enables independent maintenance of each component. When verifications change, only the verification definitions need updating, not the execution logic. This segmentation dramatically improves maintenance ease while preserving verification accuracy through dedicated verification management.
Solution Approach 2:
The patent allows verification parameters and criteria to be changed independently of execution parameters. Verification definitions can be modified, added, or removed without affecting the action steps, enabling easy maintenance and adaptation of verification logic while maintaining reliable verification of application behavior.
Data Source
AI summary
Example implementations relate to separating verifications from test executions. Some implementations may include a data capture engine that captures data points during test executions of the application under test. The data points may include, for example, application data, test data, and environment data. Additionally, some implementations may include a data correlation engine that correlates each of the data points with a particular test execution state of the application under test based on a sequence of events that occurred during the particular test execution state. Furthermore, some implementations may also include a test verification engine that, based on the correlation of the data points, verifies an actual behavior of the application under test separately from the particular test execution state.