Dynamic Risk-Based Test Case Selection for Software Regression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional software testing methods are inefficient in identifying performance issues in complex software products due to exponential growth of regression test suites and inadequate risk assessment during runtime, leading to lengthy manual processes and increased testing time.
Innovation Solution
A computer-implemented method and system for optimizing test case execution by calculating failure probability levels and determining dynamic risk profiles, allowing for the selection and execution of optimal test cases based on weighted risk impact parameters and test rule parameters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all test cases are subjected to risk assessment, then comprehensive risk coverage is achieved, but testing time and manual effort increase significantly
Solution Approach 1:
The patent segments the test case set into different risk groups (high risk, medium risk, low risk) based on dynamic risk assessment. This segmentation allows the system to apply different execution strategies to different segments, focusing comprehensive assessment only on high-risk test cases while using simplified assessment for others, thereby reducing overall testing time while maintaining adequate risk coverage.
Solution Approach 2:
The patent implements partial action by performing detailed dynamic risk assessment only on a subset of test cases (particularly high-risk ones) rather than all test cases. The system uses static risk assessment for the remaining test cases, which requires less time and manual effort, thus achieving acceptable risk coverage with reduced testing time.
2Reliability
If regression test suites include all product features, then complete testing coverage is achieved, but test suite size grows exponentially
Solution Approach 1:
The patent extracts and identifies only the critical and high-risk test cases from the complete test suite based on dynamic risk assessment results. By taking out only the essential test cases that contribute most to testing coverage and risk detection, the system reduces the effective test suite size while maintaining adequate testing coverage for the product features.
Solution Approach 2:
The patent applies local quality by assigning different risk levels and assessment depths to different test cases based on their specific characteristics and associated risks. High-risk test cases receive detailed dynamic assessment and are prioritized for execution, while low-risk test cases receive simplified assessment, creating a quality-differentiated test suite that reduces overall size while maintaining coverage where it matters most.
3Ease of operation
If static risk factor values are used for risk assessment, then assessment process is simplified, but runtime risk variations cannot be detected
Solution Approach 1:
The patent transitions from static to dynamic risk assessment by continuously monitoring runtime parameters (such as system performance metrics, error rates, and operational conditions) and updating risk factor values accordingly. This dynamic approach allows the assessment process to adapt to changing runtime conditions, improving measurement precision while maintaining operational simplicity through automated monitoring and calculation.
Solution Approach 2:
The patent implements feedback mechanisms where runtime test results and system behavior are continuously fed back into the risk assessment model. This feedback loop allows the system to update risk factor values based on actual runtime observations, improving assessment accuracy without significantly complicating the process, as the feedback is automatically processed by the system.
4Reliability
If extensive testing is performed on complex software products, then software reliability is improved, but performance problem isolation becomes extremely difficult
Solution Approach 1:
The patent segments the complex test suite into smaller, risk-based groups and executes them in a prioritized sequence. By dividing the extensive testing into manageable segments (high-risk, medium-risk, low-risk groups), the system maintains software reliability through comprehensive testing while reducing the complexity of tracking and isolating performance problems, as failures can be more easily attributed to specific risk groups and test cases.
Data Source
AI summary
The present subject matter relates to a computer implemented method and a computer system for optimizing execution of test cases. The method comprises calculating failure probability level of plurality of test cases based on plurality of test results associated to each of the plurality of test cases and determining dynamic risk profile level based on weights assigned to the failure probability level and risk impact parameter of the plurality of test cases. The method further comprises determining one or more set of optimal test cases to be executed based on the dynamic risk profile level of the plurality of test cases satisfying one or more test rule parameters. Upon determining, the method comprises identifying sequence of executing the one or more set of optimal test cases based on the one or more test sequence parameters and executing the one or more set of optimal test cases in the identified sequence.