Machine Learning Test Selection for Software Regression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Regression testing in software systems is time-consuming and resource-intensive, especially when only minor changes are made, as it often requires executing the entire test suite, which can be inefficient and may miss relevant tests.
Innovation Solution
A computer-implemented method using a machine learning model to generate a test mapping, determine a defective score for tests, and select a subset of tests based on historical data, component criticality, and defect definitions, allowing for the execution of only relevant tests corresponding to specific code changes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the entire regression test suite is executed after each code change, then all potential defects are detected, but the time and resources consumed increase significantly
Solution Approach 1:
The patent segments the regression test suite into multiple individual test cases and selectively executes only those tests that are relevant to the specific code change. This is achieved by analyzing the changed files and identifying which test cases correspond to those files, thereby dividing the monolithic test execution into targeted, smaller units that can be run efficiently without compromising defect detection for the changed areas.
Solution Approach 2:
The patent applies local quality by focusing testing resources on the specific areas affected by code changes rather than uniformly executing all tests. The system determines which test cases have high relevance to the changed files and prioritizes their execution, allocating testing effort locally where it is most needed based on the nature and scope of the modification.
2Reliability
If manual test selection is performed after code changes, then relevant tests can be identified, but the process becomes time and resource intensive
Solution Approach 1:
The patent implements self-service by enabling the system to automatically select relevant test cases without requiring manual intervention. The system analyzes code changes, maps them to corresponding test files using predefined mappings, and autonomously determines which tests should be executed, thereby eliminating the time-consuming manual test selection process while maintaining high test relevance.
Solution Approach 2:
The patent replaces the mechanical process of manual test selection with an automated computational system. Instead of developers manually reviewing and selecting tests, the system uses file mapping relationships and change analysis algorithms to automatically identify and select relevant test cases, substituting human effort with automated processing that is both faster and more consistent.
3Reliability
If the entire regression test suite is executed, then comprehensive quality validation is achieved, but resource consumption becomes excessive
Solution Approach 1:
The patent extracts and executes only the necessary subset of test cases that are relevant to the code changes, removing unnecessary tests from the execution set. By analyzing the relationship between changed files and test files through predefined mappings, the system extracts only those tests that need to be run, thereby reducing computational resource consumption while maintaining quality validation for the affected areas.
4Reliability
If frequent regression testing is performed, then system quality is maintained, but the cumulative time and resources required become unsustainable
Solution Approach 1:
The patent applies dynamics by making the test execution scope adaptive and variable based on the nature of code changes. Rather than executing a fixed entire test suite every time, the system dynamically adjusts which tests are run based on the specific files changed and their relationship to test cases. This dynamic approach allows frequent testing to maintain quality while reducing the cumulative time burden by executing only the necessary subset of tests each time.
Data Source
AI summary
Provided is a computer-implemented method, system, and computer program product for automatic selection of tests for software system regression testing using machine learning including generating a test mapping including at least one test of a plurality of tests corresponding to a source file. The plurality of tests and the at least one source file are associated with a software repository. Further, determining a defective score for the at least one test based on historical test data of the at least one test, receiving a component criticality score and a defect definition corresponding to the source file, generating a key value corresponding to at least one test based on the defective score, component criticality score, and defect definition, determining a subset of tests of the plurality of tests based on the key value corresponding to the at least one test; and executing the subset of tests with the software repository.