Adaptive Automated Testing Heuristic Reduces Resource Consumption
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Automated testing processes consume significant resources, including time, electricity, and physical space, due to the need to run thousands of tests, many of which are redundant or unnecessary.
Innovation Solution
Implementing adaptive automated testing methods that log actions, apply heuristic rules to identify redundant tests, and prioritize testing based on device and website popularity, allowing for the skipping or reduction of redundant tests and sub-tests.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If thousands of tests are performed to ensure complete test coverage, then testing reliability is improved, but resource consumption (electricity, time, physical space) increases significantly
Solution Approach 1:
The patent extracts and removes redundant tests from the test suite by comparing test logs and identifying tests that have already been executed successfully. The system takes out unnecessary duplicate tests while retaining essential unique tests, thereby reducing resource consumption while maintaining adequate test coverage.
Solution Approach 2:
The patent discards redundant test cases that have already been proven successful by analyzing historical test logs. By recovering and reusing the knowledge from previous test results, the system avoids re-executing the same tests unnecessarily, reducing electricity consumption and time while maintaining reliability.
2Reliability
If thousands of tests are performed to ensure complete test coverage, then testing reliability is improved, but time consumption increases significantly
Solution Approach 1:
The patent extracts redundant tests from the execution queue by analyzing test logs to identify tests that have already been successfully executed. By taking out these duplicate tests before execution, the system significantly reduces testing duration while maintaining complete test coverage through the retention of unique essential tests.
Solution Approach 2:
The patent performs preliminary analysis of test logs before executing the test suite to identify and remove redundant tests in advance. This preliminary action of filtering out duplicate tests based on historical data reduces the overall testing time while ensuring that all necessary unique tests are still executed for complete coverage.
3Reliability
If a large quantity of test fixtures and test devices are used to run thousands of tests, then test coverage is improved, but physical space requirements and device complexity increase
Solution Approach 1:
The patent extracts and eliminates redundant test device allocations by analyzing test logs to identify tests that can be skipped. By removing duplicate test executions, the system reduces the number of physical test devices and fixtures needed, thereby reducing physical space requirements while maintaining comprehensive test coverage through unique test execution.
4Measurement precision
If all tests are executed regardless of redundancy, then measurement precision of test results is improved, but productivity decreases due to resource waste
Solution Approach 1:
The patent uses feedback from historical test logs to inform future test execution decisions. By analyzing previous test results and feeding this information back into the test selection process, the system identifies redundant tests with high confidence, allowing it to skip them safely without compromising result accuracy, thereby improving overall testing productivity and efficiency.
Data Source
AI summary
Adaptive automated testing methods and systems are disclosed. In an embodiment, an automatic test controller of an automated test system may collect a log of a plurality of actions executed by a plurality of devices of the automated test system and apply a heuristic to the log as the automated test system operates to determine whether to execute a next action by a device of the automated test system. When the next action is to be executed, the automatic test controller automatically directs the device to execute the next action. Otherwise, the automatic test controller indicates the next action was intentionally skipped by recording such omission in the log and automatically skipping the next action.