RNN Build Log Analysis for Fatal Error Identification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Identifying the root cause of failures in Continuous Integration (CI) builds is time-consuming and does not scale with a large number of failures, as existing methods rely on manual investigation of build logs which can be cumbersome and prone to bias.
Innovation Solution
A method using a recurrent neural network (RNN) trained on a balanced dataset of build log examples from a CI pipeline to predict whether a build log indicates a successful or failed build, and to identify specific regions of the build log that meet a fatal error criterion condition.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual investigation of build logs is used to identify root cause of failures, then accuracy of failure analysis is maintained, but time consumption and resource requirements increase significantly
Solution Approach 1:
An RNN-based analysis system is introduced as an intermediary between build log data and human developers. The system processes build logs, monitors belief predictions across successive regions, and automatically identifies fatal error regions, filtering out non-fatal errors before human review. This intermediary handles the time-consuming manual analysis while preserving accurate failure identification through the trained neural network model.
2Measurement precision
If manual investigation of build logs is performed, then detailed failure analysis is achieved, but scalability to large numbers of failures deteriorates
Solution Approach 1:
The analysis system performs self-service by automatically processing build logs without human intervention. The RNN model autonomously monitors belief predictions, identifies regions where predictions fall below threshold or derivatives reach minima, and flags only fatal error regions. This self-service capability enables the system to handle large volumes of build failures at scale while maintaining detailed analysis quality, as the trained model consistently applies the same analytical criteria to all inputs.
3Loss of time
If automated RNN-based analysis is implemented to reduce time consumption, then processing speed improves, but system complexity increases
Solution Approach 1:
The RNN model is trained in advance on a balanced dataset of positive and negative build log examples before deployment. This preliminary training action equips the model with the knowledge to automatically distinguish fatal from non-fatal errors. During operation, the pre-trained system simply monitors belief predictions and applies predetermined criteria (threshold crossing, derivative minima), reducing operational complexity despite the initial training investment.
4Measurement precision
If balanced dataset with equal positive and negative examples is used for training, then model accuracy improves, but data preparation complexity increases
Solution Approach 1:
The training dataset is structured with balanced parameters: equal numbers of positive build log examples (indicating successful builds) and negative build log examples (indicating failed builds). This parameter balancing ensures the RNN model learns to distinguish between different build outcomes accurately. The balanced composition prevents bias toward either class, improving model accuracy in identifying fatal errors while the systematic data structure manages preparation complexity.
Data Source
AI summary
A computing device trains a recurrent neural network (RNN), using a balanced dataset, to predict whether logs input to the RNN are indicative of respective successful computer code or respective failed computer code, the balanced dataset comprising positive log examples and negative log examples from a continuous integration (CI) pipeline, the positive log examples labelled as being indicative of successful computer code, and the negative log examples labelled as being indicative of failed computer code. The computing device inputs a log to the RNN, and monitors evolution of belief predictions of the RNN, as the RNN is analyzing the log, according to successive regions of the log. The computing devices determines, based on the evolution of the belief predictions, that a given region of the log meets a log fatal error criterion condition, and outputs an indication of the given region.