Automated Software Test Failure Isolation Method
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In software development, identifying the cause of a test failure in the latest software code revision is challenging due to the large number of revisions and the need for manual debugging, which is time-consuming and resource-intensive, especially when tests are performed infrequently.
Innovation Solution
A method that automatically identifies the cause of a test failure by analyzing changes between revisions, generating modified versions of the software code, and determining whether the test passes or fails on these modified versions, thereby isolating the revision that introduced the fault and reducing the need for manual intervention.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If tests are performed frequently on each revision, then software quality and reliability are improved, but the time and resources required for debugging increase significantly
Solution Approach 1:
The patent segments the debugging process by automatically identifying and isolating the specific revision that introduced the fault. Instead of manually examining all revisions, the system divides the search space by comparing test results across revisions and pinpointing the exact change that caused the failure, thereby reducing debugging time while maintaining frequent testing
Solution Approach 2:
The patent performs preliminary analysis by automatically comparing test results across multiple revisions and identifying the faulty revision before manual debugging begins. This preliminary automated investigation prepares the debugging process by narrowing down the search space, allowing developers to focus only on the identified problematic revision rather than examining all revisions manually
2Measurement precision
If manual debugging is performed to identify the cause of test failure, then accurate fault identification is achieved, but the process becomes time-consuming and resource-intensive
Solution Approach 1:
The patent implements self-service debugging by enabling the system to automatically identify the faulty revision and generate debugged versions without human intervention. The system serves itself by comparing test results, analyzing revision differences, and pinpointing the cause of failure autonomously, thereby maintaining high accuracy while dramatically improving debugging efficiency
Solution Approach 2:
The patent replaces the mechanical manual debugging process with an automated computational system. Instead of developers manually examining code changes and test failures, the system uses automated comparison algorithms and data analysis to identify the faulty revision, substituting human manual labor with machine-based automation while preserving diagnostic accuracy
3Productivity
If tests are performed at regular intervals rather than on each revision, then resource consumption is reduced, but the ability to identify the specific change causing test failure is lost
Solution Approach 1:
The patent uses feedback by systematically comparing test results across different revisions to identify patterns and determine which specific change caused the failure. The system collects test outcome data from multiple revisions and uses this feedback loop to pinpoint the exact revision that introduced the fault, thereby recovering causality information even when testing is performed at intervals rather than on every revision
Data Source
AI summary
According to an aspect of the present inventive concept there is provided a method for identifying a cause for a failure of a computer-implemented test performed on a latest software code revision in a sequence of a plurality of software code revisions, the method comprising:identifying a first revision of said sequence of revisions for which said test passes and a second revision of said sequence of revisions which is subsequent to the first revision and for which said test fails,determining a first change between the software code of the first revision and the software code of the second revision,generating a first modified version of the software code of the latest revision using the determined first change,determining whether said test passes or fails when performed on the first modified version,in response to determining that said test fails when performed on the first modified version:identifying a third revision and a fourth revision of said sequence which are intermediate the latest revision and the second revision, wherein the third revision is a revision for which said test passes when said revision is modified using the determined first change, and wherein the fourth revision is subsequent to the third revision and is a revision for which said test fails when said revision is modified using the determined first change,determining a second change between the software code of the third revision and the fourth revision,generating a second modified version of the software code of the latest revision using the first change and the second change,determining whether said test passes or fails when performed on the second modified version,in response to determining that said test passes when performed on the second modified version, providing a message indicating at least the fourth revision as including a cause for said test failing when performed on said latest software code revision.