Automated Test Repair via Reinforcement Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automated software testing tools rely on random input methods, which are time-consuming and often fail to cover all valid operating conditions, leading to incomplete defect detection and manual intervention by quality assurance engineers, especially in complex and frequently updated software environments.
Innovation Solution
An application test execution and maintenance system incorporating an application learner and a test execution and repair manager, utilizing reinforcement learning and transfer learning to develop knowledge of application operations, detect failures, and automatically repair tests by identifying alternative sequences of operations, thereby reducing manual intervention and improving efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If random testing or fuzz testing is used, then automation is achieved, but testing coverage and defect detection reliability deteriorate
Solution Approach 1:
The system performs preliminary analysis of the application's valid operating conditions and defect patterns before executing tests. By pre-processing test data and identifying critical test paths based on historical defect information, the system transforms random testing into targeted testing, improving defect detection reliability while maintaining automation.
Solution Approach 2:
The system implements feedback mechanisms where test results and defect information are continuously fed back into the test generation process. This feedback loop allows the system to learn from previous test outcomes and adjust future test cases, progressively improving testing effectiveness and reliability without manual intervention.
2Extent of automation
If random testing is used, then automation is achieved, but testing time increases
Solution Approach 1:
The system pre-processes application information to identify critical test paths and valid operating conditions before actual testing begins. This preliminary analysis phase reduces the search space for subsequent automated testing, significantly cutting down the time required while maintaining comprehensive coverage.
Solution Approach 2:
The system extracts and focuses on critical test paths and high-risk areas from the entire application space. By taking out only the most important testing scenarios based on defect patterns and operational importance, the system achieves efficient testing without examining every possible random input.
3Manufacturing precision
If manual test repair is performed, then test accuracy is improved, but productivity deteriorates
Solution Approach 1:
The system performs automatic self-repair of broken test cases by analyzing the failure cause and generating corrected test steps autonomously. This self-service capability eliminates the need for manual intervention in test maintenance, preserving test accuracy through systematic repair while maintaining high productivity.
Solution Approach 2:
The system replaces manual mechanical repair processes with automated algorithmic repair mechanisms. By substituting human engineers' manual debugging and test correction work with automated analysis and repair tools, the system maintains test accuracy through systematic approaches while dramatically improving productivity.
4Reliability
If comprehensive test coverage is pursued, then defect detection improves, but device complexity increases
Solution Approach 1:
The system implements a universal testing framework that handles multiple testing objectives (coverage, defect detection, performance) through a single integrated platform. This multi-functional approach consolidates various testing capabilities into one system, achieving comprehensive defect detection without proportionally increasing system complexity.
Solution Approach 2:
The system pre-analyzes the application to automatically generate an optimized test plan that balances coverage and complexity. By performing preliminary analysis of valid operating conditions and defect patterns, the system identifies the most effective test cases without requiring manual design of complex test scenarios.
Data Source
AI summary
A test execution and maintenance system may include an application learner configured to develop knowledge of operations of an application based on one or more learning test. The test execution and maintenance system then may include a test execution and repair manager configured to execute a functional test of the application, detect a failure of the functional test, and repair the functional test based on the knowledge of operations of the application. Responsive to detecting the failure of the functional test, the application learner may optionally perform a re-learning process and update the previously acquired knowledge of operations of the application.