Test Case Ranking via Call Graph Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing software testing process is inefficient due to the need to run all test cases, which can be time-consuming and resource-intensive, and may not effectively identify errors, leading to undetected software errors and reduced developer productivity.
Innovation Solution
Ranking test cases using a call graph generated from the software codebase, where test case nodes are connected to method nodes, allowing for automatic ranking based on criteria such as outgoing and incoming edges, and distance from code changes, to identify the most relevant test cases for efficient testing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all test cases are executed to ensure thorough testing, then software error detection capability is improved, but testing time and computing resource consumption increase significantly
Solution Approach 1:
The patent extracts and identifies a subset of critical test cases from the complete test suite by analyzing code change information. Instead of executing all test cases, the system selectively extracts only those test cases that are most likely to detect errors in the modified code, thereby reducing testing time while maintaining error detection capability.
Solution Approach 2:
The patent applies local quality by differentiating between test cases based on their relevance to specific code changes. Test cases are ranked and prioritized according to their proximity and relationship to modified code regions, ensuring that testing resources are concentrated on areas with highest error risk rather than uniformly testing all code paths.
2Reliability
If all test cases are executed to ensure comprehensive coverage, then software error detection capability is improved, but computing resource consumption increases significantly
Solution Approach 1:
The system extracts only the necessary test cases related to code changes from the full test suite. By using call graph analysis and code change information, it identifies and extracts a minimal subset of test cases that provide maximum error detection value, significantly reducing computing resource consumption.
Solution Approach 2:
The patent applies partial action by executing only a portion of the test cases - specifically those that are most relevant to the code changes - rather than running the complete test suite. This partial execution approach achieves sufficient error detection coverage while consuming fewer computing resources.
3Reliability
If unnecessary test cases are executed, then testing thoroughness is maintained, but software developer productivity decreases
Solution Approach 1:
The system extracts and eliminates unnecessary test cases from the execution set by analyzing the relationship between code changes and test case relevance. By removing test cases that do not pertain to modified code, the system maintains thoroughness for critical areas while improving developer productivity through faster feedback cycles.
Solution Approach 2:
The patent implements partial action by running only the essential test cases needed to verify code changes rather than executing the entire test suite. This approach provides sufficient testing thoroughness for the changed code while significantly reducing the time developers wait for test results, thereby improving productivity.
4Productivity
If test cases are ranked and prioritized based on call graph analysis, then testing efficiency is improved, but device complexity increases
Solution Approach 1:
The patent applies preliminary action by pre-computing the call graph and pre-ranking test cases before actual testing begins. This upfront analysis creates a ready-to-use prioritization framework that simplifies the testing execution phase, improving efficiency without adding complexity during the actual testing process.
Solution Approach 2:
The call graph serves as an intermediary data structure that mediates between the code base and the test case selection process. By introducing this intermediate representation, the system can efficiently analyze code relationships and rank test cases without directly complicating the testing execution mechanism itself.
Data Source
AI summary
Technologies are described for ranking test cases related to changes in a software code base using at least a portion of a call graph that is generated from the software code base. The at least a portion of the call graph comprises method nodes corresponding to methods in the software code base. The method nodes represent control flow relationships between the methods in the software code base. The at least a portion of the call graph also comprises test case nodes. Each test case node is connected to one or more method nodes corresponding to coverage of the one or more method nodes by the test case node. Using the at least a portion of the call graph, the test case nodes are automatically ranked. Ranking of the test case nodes is performed by evaluating various criteria associated with the at least a portion of the call graph.