Automatic Error Diagnosis in Test Environments
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current testing environments require significant human intervention and time for log data analysis to diagnose failures in test logs, as existing automated systems are limited to anomaly detection and do not provide reasons for test failures.
Innovation Solution
A method and system that transform test logs into vectors for a machine learning model, utilizing a defect database with known failures to facilitate automatic error diagnosis, where test logs are pre-processed and labeled, and the model undergoes supervised training to recognize and classify failures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If log data analyzing systems incorporate a vast usage of scripts for post-processing, then detailed analysis of test logs is achieved, but significant time and human intervention are required
Solution Approach 1:
The patent replaces manual script-based log analysis with an automated machine learning system. The ML model automatically processes test logs, extracts failure reasons, and generates diagnoses without requiring human intervention or complex scripting, thereby reducing processing time while maintaining analysis accuracy.
Solution Approach 2:
The system enables self-service automated error diagnosis where the machine learning model independently analyzes test logs, identifies failures, and provides diagnoses without external human assistance. The system serves itself by automatically training on historical data and continuously improving its diagnostic capabilities.
2Productivity
If automated detection systems use machine learning for log data analysis, then execution time is reduced, but the system is limited to anomaly detection only and cannot provide reasons for failures
Solution Approach 1:
The patent segments the log analysis process into distinct components: anomaly detection, failure reason extraction, and diagnostic generation. The machine learning model is trained to perform multiple functions sequentially, first detecting anomalies and then extracting specific failure reasons from the log data, thereby providing both speed and detailed information.
Solution Approach 2:
The system combines multiple machine learning capabilities into a composite diagnostic system. It integrates anomaly detection, pattern recognition, and natural language generation to produce comprehensive failure diagnoses that include both the detection of issues and the explanation of reasons, creating a multi-functional analytical system.
3Extent of automation
If a defect database with known failures is created and supervised training is performed, then automatic recognition of failures is achieved, but the system requires initial data preparation and model training
Solution Approach 1:
The patent applies preliminary action by pre-processing and labeling historical test log data before training the machine learning model. The defect database is prepared in advance with known failure patterns, enabling the model to learn from structured data. This upfront preparation simplifies the ongoing automated diagnosis process.
Solution Approach 2:
The defect database serves as an intermediary between historical failure data and the machine learning model. It structures and organizes failure patterns in a format suitable for training, acting as a bridge that translates raw log data into learnable patterns, thereby simplifying the training process while enabling high-level automation.
Data Source
AI summary
A method for automatic error diagnosis in a test environment is provided. The method comprises the step of providing a plurality of test logs associated with known types of failures, each comprising a set of files. The method further comprises the step of arranging the plurality of test logs in a defect database. Moreover, the method comprises the step of transforming the set of files of the plurality of test logs into vectors adapted to be fed into a machine learning model.