AI Error Analyzer for Automated Software Testing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Manual review of automated test results is time-consuming and often pointless due to numerous error messages generated, with many errors being non-application defects rather than software defects.
Innovation Solution
Employing AI and machine learning to analyze error messages, categorizing them as application defects, non-application defects, or needing further analysis, and executing remedial actions, thereby reducing the manual review burden and optimizing the software development process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual review of automated test results is performed, then all error messages can be examined, but the review process becomes extremely time consuming
Solution Approach 1:
An AI-based classification system is introduced as an intermediary between automated test execution and human review. The system automatically categorizes error messages into application defects, non-application defects, and needs analysis, filtering and prioritizing errors before they reach human reviewers. This intermediary processing reduces the time developers spend reviewing test results while maintaining comprehensive error examination through structured classification.
2Measurement precision
If all error messages are reviewed manually, then no errors are missed, but many errors may be generated by non-application defects making the review process largely pointless
Solution Approach 1:
The error review process is segmented into distinct categories: application defects, non-application defects, and needs analysis. The AI classification system automatically divides error messages into these segments, allowing human reviewers to focus specifically on application defects that require their expertise. This segmentation maintains complete error detection while dramatically improving review efficiency by eliminating pointless analysis of non-application errors.
Solution Approach 2:
Non-application defects and needs analysis errors are extracted from the main review workflow and handled separately through automated classification. By taking out these error types that do not require human review, the system maintains comprehensive error detection capability while preventing reviewers from wasting time on errors that will not lead to application fixes.
3Loss of time
If AI classification is implemented, then review time is reduced, but the system complexity increases
Solution Approach 1:
The AI classification system operates autonomously to categorize error messages without requiring manual configuration or intervention for each error. The system self-manages the classification process, automatically assigning errors to appropriate categories based on learned patterns from training data. This self-service capability reduces review time while keeping system complexity manageable by eliminating the need for manual system configuration and maintenance.
Data Source
AI summary
Apparatus and methods for using artificial intelligence to process and remediate test failures are provided. The methods may include monitoring an execution of an automated test on a software application. The automated test may include a plurality of steps. The methods may include receiving a first error message and a second error message during the execution of a step included in the plurality of steps. The methods may include processing each of the first and second error messages individually and, after completion of the step, again as a group. The processing may include determining if the error messages were generated by a defect in the software application.