Parallel Virtual Test Automation for Software Validation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional software testing methods for on-demand software models like SaaS, PaaS, and IaaS are inefficient due to serial execution of test cases, requiring extensive time and resources, and lack effective automation for debugging and troubleshooting, especially in complex systems with dynamic user interactions and frequent feature modifications.
Innovation Solution
Implementing end-to-end automation by executing test cases in parallel using virtual time, generating virtual test results, and advancing the virtual time until a stopping criterion is met, allowing concurrent execution of test cases with different environments and states, and handling errors and issues within the test sessions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If test cases are executed in series to ensure proper validation and verification, then testing accuracy is improved, but testing time increases significantly
Solution Approach 1:
The system performs preliminary actions by creating virtual copies of test environments and pre-configuring test states before actual test execution. Virtual test environments are instantiated in advance, allowing multiple test cases to be prepared and executed concurrently without interfering with each other's validation and verification processes
Solution Approach 2:
The patent creates virtual copies of test environments and test states that can be executed in parallel. Instead of running single test cases sequentially on one environment, the system generates multiple virtual environment instances that replicate necessary test conditions, enabling concurrent execution while maintaining testing accuracy through isolated validation of each virtual instance
2Reliability
If test environments are restored and reset between test cases to ensure clean testing, then testing reliability is improved, but testing efficiency decreases
Solution Approach 1:
The test system is segmented into multiple independent virtual environments, each capable of running test cases autonomously. Instead of restoring a single shared environment between tests, the system divides testing into separate virtual segments that can execute in parallel, eliminating the need for repeated restoration while maintaining reliability through environmental isolation
Solution Approach 2:
The system changes the fundamental parameter of environment sharing by transitioning from a single shared environment to multiple virtual environment instances. Each virtual environment maintains its own state independently, allowing test cases to run concurrently without requiring restoration, thus improving efficiency while preserving reliability through parameter-based environmental differentiation
3Productivity
If multiple test cases are grouped into buckets and executed together to improve efficiency, then testing speed is improved, but debugging capability deteriorates due to loss of individual test results
Solution Approach 1:
The system creates virtual copies of test environments for each test case within the bucket, ensuring that individual test results are preserved and accessible. Each virtual environment captures and stores its own test results independently, allowing debugging of specific test cases even when executed in parallel groups, thus maintaining debugging capability while achieving parallel execution speed
4Measurement precision
If serial test execution is used to maintain system state consistency, then test validation accuracy is improved, but automation capability deteriorates
Solution Approach 1:
Each virtual test environment is self-sufficient and self-managing, automatically configuring its own state and executing tests independently without requiring manual orchestration or sequential coordination. The virtual environments self-service by maintaining their own consistency and isolation, enabling full automation of parallel test execution while preserving validation accuracy through automated environmental management
Data Source
AI summary
Various aspects described herein are directed to a method and a system that implement end-to-end automation of software services. A set of test cases may be identified from existing test cases or determined anew for a software system. Virtual test results may be generated at least by executing the set of test cases in parallel on a virtual time or date. The virtual test results may then be used to validate or verify the behaviors of the software system. The virtual time or date is then advanced to the next virtual time or date by a temporal period until a stopping criterion is satisfied.