AI Models for Automated Test Failure Root Cause Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Manual analysis of failed test cases in software testing is time-consuming and overwhelming, as it requires determining the cause of failures and identifying the appropriate team or component to address them, especially with hundreds of tests across different environments.
Innovation Solution
A computer-implemented method using artificial intelligence models to differentiate between bug failures and test failures, cluster failed tests, identify root causes, and predict the root cause of unclassified failures, thereby automating the analysis process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual analysis of failed test cases is performed, then detailed investigation of each failure can be conducted, but the process becomes time-consuming and overwhelming
Solution Approach 1:
The patent replaces manual mechanical analysis with an AI-based automated system. The machine learning model automatically classifies failure types (bug failures vs. test failures) and identifies root causes, substituting human cognitive processing with computational algorithms that operate continuously without fatigue or time loss.
Solution Approach 2:
The system performs self-service by automatically analyzing its own outputs. The AI model processes test failure data, categorizes failures, and generates root cause analyses without requiring external human intervention for each individual failure case, enabling the system to serve itself in the analysis process.
2Adaptability or versatility
If comprehensive failure analysis is performed across hundreds of tests, then complete coverage of all failure types is achieved, but the complexity and overwhelming nature of the task increases
Solution Approach 1:
The patent segments the complex analysis task into distinct manageable components: (1) classifying failure types using the first machine learning model, (2) clustering similar failures together, and (3) identifying root causes for each cluster. This segmentation breaks down the overwhelming complexity into systematic, sequential steps that are easier to implement and maintain.
Solution Approach 2:
The AI system provides universal functionality by handling multiple failure types and analysis tasks through a single integrated approach. The same machine learning model and clustering mechanism work across different test environments, failure categories, and scales, making the system adaptable and versatile without requiring separate specialized processes for each failure type.
3Productivity
If automated test automation is implemented, then repetitive testing tasks are streamlined, but the complexity of the testing system increases
Solution Approach 1:
The patent introduces an AI intermediary layer between the automated test execution and the analysis phase. This intermediary (the machine learning model) processes test results, classifies failures, and identifies root causes, acting as a mediator that simplifies the overall system architecture by providing a clear separation of concerns between testing and analysis functions.
Data Source
AI summary
A computer-implemented method, system and computer program product for analyzing test result failures using artificial intelligence models. A first machine learning model is trained to differentiate between a bug failure and a test failure within the test failures based on the failure attributes and historical failures. The failure type for each failed test in test failure groups is then determined using the first machine learning model. The failed tests in the test failure groups are then clustered into a set of clusters according to the failure attributes and the determined failure type for each failed test. A root cause failure for each cluster is identified based on the set of clusters and the failure attributes. The root cause of an unclassified failure is predicted using a second machine learning model trained to predict a root cause of the unclassified failure based on identifying the root cause failure for each cluster.