AI Regression Testing Prioritization Using Organization FSM Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing regression testing methods are resource-intensive and time-consuming, particularly when software updates affect multiple organizations, leading to high overhead and limited time for fixing new or re-emerging faults.
Innovation Solution
Implementing AI-based regression testing using machine learning models, including a regression testing module that utilizes a Finite State Machine (FSM) model and Bayesian smoothing to prioritize organizations for testing based on bug probability, reducing the number of organizations tested while achieving high coverage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional regression testing is performed on all organizations, then testing coverage is improved, but resource consumption and time overhead increase significantly
Solution Approach 1:
The patent creates a virtual copy of the organization ecosystem through simulated environments that mirror real organizational structures, data flows, and operational patterns. These virtual copies allow comprehensive testing to be performed on replicas rather than actual organizations, maintaining high testing coverage while eliminating the resource consumption and time overhead of testing on real systems.
Solution Approach 2:
The patent performs preliminary risk assessment and impact analysis before executing regression tests. By identifying high-risk areas and organizations most likely to be affected by changes upfront, the system prioritizes testing efforts on critical paths and high-impact areas, achieving effective coverage with reduced resource expenditure on low-risk areas.
2Reliability
If traditional regression testing is performed on all organizations, then fault detection capability is improved, but lead time for fixing regressions increases
Solution Approach 1:
The patent performs preliminary risk assessment and impact analysis before executing regression tests. By identifying high-risk areas and organizations most likely to be affected by changes upfront, the system prioritizes testing efforts on critical paths and high-impact areas, achieving effective fault detection with reduced testing duration and faster identification of regressions that need fixing.
Solution Approach 2:
The patent segments the regression testing process into independent, parallelizable units based on organizational units, system modules, or test scenarios. This segmentation allows multiple test suites to execute concurrently across distributed resources, significantly reducing the total lead time for completing comprehensive fault detection while maintaining detection capability.
3Productivity
If AI-based regression testing is implemented, then resource utilization is improved, but testing coverage may be reduced
Solution Approach 1:
The patent implements AI models that learn from historical test results, bug patterns, and organizational characteristics. The system continuously receives feedback from executed tests and adjusts its predictions and prioritization strategies accordingly. This feedback loop enables the AI to identify high-value test cases with high confidence, allowing reduced testing coverage in low-risk areas while maintaining comprehensive coverage in critical areas, thus improving resource utilization without sacrificing overall reliability.
Data Source
AI summary
Embodiments described herein provide regression testing using artificial intelligence. A regression testing network model for a first plurality of organizations using a common codebase is provided. The regression testing network model provides an organization finite state machine (FSM) model for each organization. A first dataset including samples of the organization FSM models based on regression testing for one or more previous releases of the common codebase prior to a first release of the common codebase is received. A training dataset is generated based on the first dataset. The regression testing network model using the training dataset. A second plurality of organizations for regression testing for the first release is determined, from the first plurality of organizations, using the trained regression testing network model.