AI Patch Risk Model for Selective Regression Testing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current software update testing methods are inefficient and resource-intensive, as they treat all patches equally without considering varying risk levels, leading to prolonged testing times and potential impact on continuous delivery objectives.
Innovation Solution
A multi-layer artificial intelligence model is created using metadata from software patches to assess risk, allowing for selective regression test case execution based on risk levels, thereby accelerating deployment while maintaining software reliability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all software patches are tested using full regression testing, then software reliability is maintained, but testing time and resources increase significantly
Solution Approach 1:
The patent segments the regression test suite into multiple subsets based on patch characteristics and risk levels. Instead of executing all regression tests for every patch, the system divides tests into different groups (e.g., critical path tests, non-critical path tests) and selectively executes only the necessary subsets based on the patch's impact analysis, thereby reducing testing time while maintaining reliability for critical functions.
Solution Approach 2:
The patent applies partial action by executing only the necessary portion of regression tests based on patch risk assessment. For low-risk patches, only critical path tests are executed; for high-risk patches, more comprehensive testing is performed. This selective approach avoids the excessive action of running all regression tests for every patch, optimizing the balance between reliability and testing time.
2Reliability
If comprehensive regression testing is performed for all patches, then software quality is ensured, but continuous delivery efficiency is reduced
Solution Approach 1:
The patent introduces dynamic test selection based on patch characteristics. The regression test execution strategy is not static but adapts dynamically according to the patch's risk level, impact scope, and criticality. This dynamic approach allows the system to adjust testing intensity in real-time, ensuring software quality for critical patches while enabling faster delivery for low-risk patches, thus improving continuous delivery efficiency.
Solution Approach 2:
The patent changes key parameters such as test selection criteria, execution depth, and resource allocation based on patch attributes. By adjusting these parameters dynamically, the system optimizes the balance between quality assurance and delivery speed, allowing comprehensive testing when quality is paramount and expedited testing when delivery efficiency is prioritized.
3Productivity
If risk-based test selection is implemented, then testing efficiency is improved, but test coverage may be reduced
Solution Approach 1:
The patent applies local quality by ensuring high test coverage for critical areas while accepting reduced coverage for non-critical areas. The risk-based selection prioritizes testing of critical path functions, security-sensitive modules, and high-impact features with comprehensive coverage, while using lighter testing for stable, low-risk components. This localized quality approach maintains overall system reliability while improving testing efficiency.
Data Source
AI summary
Building a first layer model of a three-layer model based on attributes that are sensitive features is provided. A first dimension reduction of sensitive features removes each sensitive feature having an indicator that it is present in a patch and does not contribute to one or more of three probabilities. A second dimension reduction of insensitive features is performed using vectorizing and using one-hot encoding. The remaining insensitive features are main features. One or more second layer models of the three-layer model is built based on the main features. The third layer model is built based on a verification dataset and the first layer model. Regression test coverage is recommended based on prediction result of the third layer model, and wherein regression tests are selected. The training dataset is updated based on probability calculations of the first layer model.