AI Diagnosis System for Software Testing Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Software testing faces challenges in efficiently identifying the root cause of bugs due to the separation between code writers and testers, with existing model-based diagnosis methods generating multiple candidate diagnoses that require additional test steps to differentiate the correct diagnosis, often involving a trial-and-error process.
Innovation Solution
An AI-based method combining model-based diagnosis and planning techniques to automatically suggest further test steps, iteratively pruning incorrect diagnoses and minimizing tester efforts, using algorithms like Highest Probability Focused Testing, Lowest Cost Focused Testing, Entropy-Based Focus Testing, and Markov Decision Process-Based Focused Testing to identify the correct faulty software component.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If model-based diagnosis is used to generate candidate diagnoses, then the completeness of diagnosis coverage is improved, but the number of test steps required to differentiate correct diagnosis increases
Solution Approach 1:
The patent applies preliminary action by pre-computing and storing execution traces and coverage information for multiple candidate diagnoses before actual testing. When a bug is detected, the system retrieves pre-computed diagnosis candidates and their associated test implications, enabling rapid differentiation without extensive new testing. This preliminary preparation reduces the number of test steps needed at diagnosis time.
Solution Approach 2:
The patent implements feedback mechanisms where test results from executed paths are fed back into the diagnosis system to eliminate incorrect candidate diagnoses. The system uses observed bug behavior compared against expected behavior for each candidate diagnosis, systematically narrowing down the candidate set based on test outcomes, thereby reducing the number of tests needed to identify the correct diagnosis.
2Reliability
If additional test steps are performed to differentiate candidate diagnoses, then the reliability of identifying the correct diagnosis is improved, but the complexity of the testing process increases
Solution Approach 1:
The patent segments the testing process into distinct phases: generating candidate diagnoses, selecting differentiating test paths, executing tests, and eliminating incorrect candidates. Each phase handles a specific subset of the diagnosis task, making the overall complex process more manageable and systematic rather than requiring all tests to be planned simultaneously.
Solution Approach 2:
The patent introduces an intermediary diagnostic system that acts as a mediator between the bug detection and the final diagnosis. This intermediary systematically manages candidate diagnoses, selects optimal test paths, and coordinates the differentiation process, reducing the complexity burden on testers while maintaining high reliability through structured intermediate processing.
3Productivity
If automated test planning is implemented to minimize tester efforts, then the productivity of the testing process is improved, but the requirement for software code familiarity increases
Solution Approach 1:
The patent applies self-service by implementing automated systems that perform test path selection, candidate diagnosis generation, and differentiation planning without requiring deep code familiarity from testers. The system autonomously analyzes software behavior, generates hypotheses about bug causes, and determines optimal tests to confirm or eliminate these hypotheses, making the process self-sufficient regarding code knowledge.
Solution Approach 2:
The patent replaces the mechanical requirement for human code familiarity with an automated computational system. Instead of relying on testers to understand code structure and logic to plan effective tests, the system uses automated analysis of execution traces, coverage data, and bug behavior to generate and evaluate candidate diagnoses, substituting human expertise with algorithmic processing.
Data Source
AI summary
An artificial intelligence based method for improving a software testing process, according to which upon finding a bug, a set of candidate diagnoses is proposed to the tester, based on a Model-Based Diagnosis (MBD) process. A planning process is used for automatically suggesting further test steps to be performed by the tester, to identify the correct diagnosis for the developer in the form of faulty software component that caused the bug, while minimizing the tests steps performed by the tester. Additional information is provided to the MBD process, based on the outputs of the further test steps, thereby pruning incorrect candidate diagnoses. These steps are iteratively repeated while in each time, minimizing the set of candidate diagnoses, until a single diagnosis remains in the set.