Machine Learning Test Case Selection for High-Risk Software
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automated software testing processes are inefficient, requiring significant time and resources, often missing defects due to redundant test cases and inadequate coverage, especially in complex software, and lack mechanisms to learn from past tests to focus on high-risk areas.
Innovation Solution
A computer-implemented method using machine learning (ML) models, such as decision trees, random forests, and neural networks, to analyze historical data and generate optimized test cases by prioritizing scenarios likely to reveal defects, adapting to code changes and user feedback.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If exhaustive testing is used to test all possible combinations of input variables, then defect detection coverage is improved, but testing time and processing resources increase significantly
Solution Approach 1:
The patent changes the approach from testing all possible parameter combinations (exhaustive testing) to selecting and prioritizing specific parameter combinations based on machine learning predictions. The system transforms the testing problem by using historical data to predict which input parameter combinations are most likely to reveal defects, thereby reducing the testing space while maintaining high defect detection coverage.
Solution Approach 2:
The patent implements feedback mechanisms by using historical test results and defect data to train machine learning models. These models learn from past testing experiences and provide feedback on which test cases are most valuable for defect detection. This feedback loop enables the system to continuously improve its test case selection accuracy over time, reducing redundant testing while increasing effectiveness.
2Productivity
If pairwise testing is used to reduce the number of test cases, then processing burden is reduced, but defect detection capability may be compromised
Solution Approach 1:
The patent uses feedback from historical test results and defect reports to train machine learning models that predict the likelihood of defects in different test cases. This feedback mechanism enables the system to intelligently select test cases that are most likely to reveal defects, going beyond simple pairwise testing to achieve both efficiency and effectiveness.
Solution Approach 2:
The patent changes the test case selection approach from fixed pairwise combinations to dynamic, priority-based selection using machine learning predictions. The system adjusts which test cases to execute based on predicted defect likelihood, code changes, and historical data, thereby optimizing the balance between testing coverage and processing efficiency.
3Ease of manufacture
If existing automated testing processes are used without learning mechanisms, then implementation is simpler, but redundant test cases are executed over multiple iterations
Solution Approach 1:
The patent introduces feedback mechanisms that capture historical test results, defect reports, and code change information to train machine learning models. These models provide intelligent feedback on which test cases are most valuable, enabling the system to learn from past iterations and avoid redundant testing in future iterations, thereby reducing wasted time while maintaining implementation feasibility.
4Productivity
If test cases are generated without focusing on high-risk areas, then test generation is faster, but defect detection in critical areas is insufficient
Solution Approach 1:
The patent changes the test case generation approach by using machine learning predictions to identify and prioritize high-risk areas. The system analyzes historical data, code changes, and defect patterns to predict which input parameter combinations are most likely to reveal defects. This enables the system to focus testing efforts on critical areas while maintaining overall testing speed and efficiency.
Data Source
AI summary
An automation testing system includes a processor and a memory storing historical data which at least comprising past test results including input parameters and testing outcomes for each test case included in the past test results. The processor is configured to identify input parameters for a first iteration of a first software application having first functionalities; execute an initial test on the first iteration to generate initial results; based on the input parameters and the initial results, collecting first historical data at least comprising first past test results for at least one second software application having second functionalities corresponding to the first functionalities; training a model employing AI or ML based on the initial results and the first historical data to generate a trained model; executing the trained model with the input parameters as input to generate a set of test cases for testing the first functionalities.