Automatic Network Data Labeling for Supervised Machine Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional approaches for anomaly detection in telecommunications networks are reactive, require extensive expertise, and struggle with scalability and accuracy due to the lack of automated tools for labeling raw data, which is essential for supervised machine learning.
Innovation Solution
The development of machine learning systems and methods that automatically label network data by statistically correlating Performance Monitoring (PM) data with target events, reducing PM data to a single probability value, and using forecast models to predict abnormal behavior, enabling proactive remedial actions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual labeling of network data is performed by experts, then data accuracy for supervised machine learning is improved, but time consumption and operational complexity increase significantly
Solution Approach 1:
The system performs automatic self-labeling of network data by utilizing unsupervised machine learning algorithms to identify patterns and anomalies in raw network data, eliminating the need for manual expert labeling while maintaining high accuracy through iterative refinement and validation mechanisms
Solution Approach 2:
An automated labeling system acts as an intermediary between raw network data and supervised machine learning models, using unsupervised learning algorithms to generate initial labels that are then refined through feedback loops and validation processes, reducing both time and expert dependency
2Ease of manufacture
If rule-based engines with hard-coded thresholds are used for anomaly detection, then implementation simplicity is improved, but detection accuracy and adaptability to complex networks deteriorate
Solution Approach 1:
The system dynamically adjusts detection parameters and thresholds based on learned patterns from network data, transitioning from static hard-coded values to adaptive parameters that evolve with network conditions, improving accuracy while maintaining implementation simplicity through automated parameter tuning
Solution Approach 2:
The anomaly detection system transitions from static rule-based thresholds to dynamic, adaptive thresholds that automatically adjust based on network conditions and learned patterns, enabling the system to handle complex networks while maintaining ease of implementation through automated adaptation
3Device complexity
If reactive anomaly detection approaches are used, then system complexity is reduced, but response time and operational efficiency worsen due to slow reaction to failures
Solution Approach 1:
The system performs preliminary unsupervised learning analysis on network data to establish baseline patterns and detect potential anomalies before they manifest as failures, enabling proactive identification and response to issues while maintaining manageable system complexity through incremental implementation
Data Source
AI summary
Systems and methods include obtaining network data including first data of devices and services in the network, Performance Monitoring (PM) data associated with the devices and services and with associated timestamps, and second data including any of tickets, alarms, and events affecting some of the devices and services and with associated timestamps; obtaining one or more target events from the second data based on associated operational impact in the network; determining the PM data that is statistically correlated with the one or more target events; determining the statistically correlated PM data over a corresponding time based on the associated timestamps of the PM data and the one or more target events; and providing labels for the determined statistically correlated PM data with an associated label based on the associated target event of the one or more target events.