Communication Network Data Fault Detection Using Binary Classifier and Clustering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Communication networks face issues with data integrity due to invalid data patterns, which can lead to poor user experiences, inaccurate results, and delayed issue resolution, as existing systems do not effectively validate data before processing and consumption.
Innovation Solution
A machine learning-based data source management system that applies a binary classifier and clustering model to detect and verify invalid data patterns, replacing the affected data source with a reliable alternative to ensure accurate data consumption and monitoring in communication networks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data from multiple sources is collected and used for network monitoring, then the quantity and coverage of monitored data is improved, but the risk of invalid data patterns affecting network decisions increases
Solution Approach 1:
The patent introduces an intermediary validation system that sits between data sources and the network monitoring system. This intermediary applies machine learning models (binary classifier and clustering model) to validate data patterns before they are consumed, filtering out invalid data while preserving valid data from multiple sources. The intermediary resolves the contradiction by enabling quantity improvement without proportionally increasing reliability risks.
Solution Approach 2:
The patent implements preliminary validation actions by training machine learning models on historical data patterns before deploying them for validation. The binary classifier and clustering model are pre-trained to recognize valid versus invalid data patterns, allowing the system to proactively filter data before it reaches the monitoring system. This preliminary action prevents invalid data from compromising reliability while maintaining data quantity.
2Measurement precision
If machine learning models are applied to validate data patterns, then data validation accuracy is improved, but the processing time and computational complexity increase
Solution Approach 1:
The patent segments the validation process into two distinct stages: first, a binary classifier quickly determines whether data patterns are potentially valid; second, a clustering model provides deeper verification for ambiguous cases. This segmentation allows the system to achieve high validation accuracy while minimizing processing time by applying the more computationally intensive clustering model only when necessary, rather than to all data points.
Solution Approach 2:
The patent applies partial validation action by using the binary classifier as a first-line filter that handles most validation cases efficiently. Only data points that the binary classifier identifies as uncertain or potentially invalid proceed to the more computationally intensive clustering model. This partial application of the full validation process maintains high accuracy while reducing overall processing time and computational burden.
3Productivity
If invalid data patterns are detected and data sources are replaced, then network performance and user experience are improved, but the system complexity and automation requirements increase
Solution Approach 1:
The patent implements self-service by enabling the system to automatically detect invalid data patterns, validate replacements, and switch between data sources without human intervention. The machine learning models continuously monitor data patterns and autonomously trigger data source replacements when validation failures are detected. This self-service capability improves network performance through rapid response to invalid data while managing system complexity through automation rather than manual processes.
Solution Approach 2:
The patent establishes feedback loops where validation results from the binary classifier and clustering model continuously inform data source selection and replacement decisions. The system learns from validation outcomes and adjusts its behavior accordingly, creating a closed-loop control system. This feedback mechanism improves network performance by ensuring only valid data influences network decisions while managing complexity through systematic, rule-based automation rather than ad-hoc manual intervention.
Data Source
AI summary
A processing system may apply a binary classifier to detect whether a first data pattern of a first data source associated with a communication network performance indicator is consistent with prior data patterns of the first data source that are labeled as correct data patterns, determine, via the binary classifier, that the first data pattern is not consistent, apply a clustering model to a first input data set comprising the first data pattern and invalid data patterns of the first data source to obtain a first plurality of clusters, verify that the first data pattern is an invalid data pattern when the first plurality of clusters is the same as a second plurality of clusters generated by applying the clustering model to a second input data set comprising the invalid data patterns, and replace the first data source with a replacement data source as an active data source in response.