Test Step Execution and Evaluation Separation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing system testing methods require constant human intervention and are interrupted by the need for immediate evaluation, especially for computationally intensive verifications, leading to prolonged testing sequences and inefficiencies.
Innovation Solution
A computer-implemented method that allows for the separation of test step execution and evaluation, enabling automatic or postponed evaluation based on characteristics such as human availability and computational effort, allowing for later evaluation of test steps without interrupting the testing process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If immediate evaluation of each test step is performed, then test reliability is improved, but testing time is prolonged and productivity decreases
Solution Approach 1:
The patent segments the testing process into two independent phases: execution phase (where test steps are run and data is collected) and evaluation phase (where test steps are assessed). This segmentation allows the execution of multiple test steps to proceed in parallel without waiting for immediate evaluation, thereby improving productivity while maintaining reliability through structured evaluation of each test step.
Solution Approach 2:
The patent implements preliminary action by collecting and storing test data during execution without immediately evaluating it. The system prepares test data in advance and maintains it for later evaluation, allowing the execution phase to complete fully before the evaluation phase begins, thus reducing overall testing time while ensuring thorough evaluation.
2Reliability
If human tester is constantly present for immediate evaluation, then test reliability is improved, but device complexity and operational burden increase
Solution Approach 1:
The patent implements self-service by enabling the testing system to automatically collect, store, and manage test data without requiring constant human intervention. The system autonomously handles data collection during execution and prepares it for evaluation, reducing the operational burden on human testers while maintaining reliable test evaluation.
Solution Approach 2:
The patent introduces an intermediary data collection and management layer between test execution and human evaluation. This intermediary system automatically handles data storage and retrieval, acting as a buffer that eliminates the need for constant human presence while ensuring reliable evaluation when needed.
3Measurement precision
If computationally intensive verifications are performed immediately, then test accuracy is improved, but testing time is prolonged
Solution Approach 1:
The patent segments computationally intensive verification operations from the execution phase. Heavy computations are deferred to the evaluation phase after data collection is complete, allowing fast execution without time-consuming calculations during testing. This maintains accuracy by performing thorough verifications while reducing overall testing time.
Solution Approach 2:
The patent performs preliminary data collection and storage during execution, then conducts computationally intensive verifications during the evaluation phase. This preliminary action approach allows the system to gather all necessary data quickly and then process it thoroughly without delaying execution, thus maintaining both accuracy and efficiency.
Data Source
AI summary
A computer implemented method for testing a system includes: determining one or more test cases for testing the system, each of the one or more test cases comprising at least one test step; for each of the one or more test cases, carrying out the steps of: executing each of the at least one test step of the respective test case; collecting data generated during and/or after the execution of the at least one test step of the respective test case; and based on a characteristic of the respective test step: evaluating the collected data and setting a status for the at least one test step of the respective test case to passed or failed based on the collected data, or recording the collected data and setting the status for the at least one test step of the respective test case to postponed.


