Network Log Missing Value Imputation for Root Cause Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional root cause analysis techniques for network communication defects are limited by missing values in parameter data, reliance on human-defined logic, and inability to distinguish between complex relationships in network parameters, leading to biased analysis and unclear defect classification.
Innovation Solution
A method utilizing machine learning and deep learning to process missing values in network log data by separating data into non-time series and time series, imputing values using Gaussian mixture models and Low-Rank Autoregressive Tensor Completion, and training Fully Connected Neural Networks and 1-Dimensional Convolutional Neural Networks to classify defect root causes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional human-defined logic is used to determine root cause, then the analysis process is simple and interpretable, but the analysis accuracy deteriorates due to missing values and inability to capture complex relationships
Solution Approach 1:
The patent introduces machine learning models and data processing intermediaries (imputation models, feature engineering pipelines) between the raw network log data and the root cause determination. These intermediaries handle missing values through statistical imputation, transform raw parameters into meaningful features, and capture complex non-linear relationships that traditional logic cannot detect, thereby improving accuracy while managing complexity through modular architecture
Solution Approach 2:
The patent replaces traditional mechanical rule-based logic systems with data-driven machine learning models. Instead of manually defined decision trees and threshold-based rules, the system uses trained models (such as random forests, neural networks) that automatically learn patterns from historical data, enabling the system to handle complex relationships and missing values without requiring explicit programming of all possible scenarios
2Measurement precision
If machine learning techniques are applied to analyze network parameters, then the ability to recognize complex relationships improves, but the handling of missing values becomes more challenging
Solution Approach 1:
The patent applies preliminary data processing and imputation techniques before feeding data into machine learning models. Missing values are handled through statistical methods (mean imputation, regression imputation, or model-based imputation) and data preprocessing steps are performed in advance to ensure data quality, allowing the ML models to focus on pattern recognition rather than handling missing data during training or inference
Solution Approach 2:
The patent transforms raw network parameters into engineered features that are more suitable for machine learning analysis. This includes creating derived parameters, applying transformations (log, normalization), and selecting subsets of parameters that have the strongest predictive power. By changing the parameter representation, the system can work effectively even when some original parameters are missing
3Measurement precision
If complete data is required for accurate root cause determination, then the analysis accuracy improves, but the analysis time increases due to data collection and preprocessing
Solution Approach 1:
The patent implements a tiered analysis approach where the system can perform root cause determination with partial data available. Instead of requiring complete data collection from all network elements before analysis, the system can operate with available data subsets, using imputation for critical missing values and flagging non-critical gaps. This allows faster initial analysis that can be refined as more data becomes available, reducing the time loss while maintaining acceptable accuracy
Data Source
AI summary
A method for processing missing values of network log data and a method for classifying communication defect root cause thereof, includes utilizing machine learning or deep learning techniques to analyze the root causes of various defects occurring in a network environment while obtaining a complete dataset by obtaining appropriate imputation values according to the characteristics of the parameters in which the missing values exist.