Clustered ML Modeling for Network Forecasting at Scale
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current network modeling approaches face inefficiencies in forecasting network service capabilities due to the complexity of managing hundreds of thousands of network elements, leading to inaccurate forecasts and time delays, especially when hyperparameter tuning is not feasible for every model.
Innovation Solution
A scalable machine learning modeling service using AI/ML logic and historical data to forecast network services, focusing on representative network elements for hyperparameter tuning, thereby enhancing forecasting at a micro-level without searching hyperparameter spaces for each element.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If hyperparameter tuning is performed for every network element model, then model performance is maximized, but computational complexity and time delays increase significantly
Solution Approach 1:
The patent segments the large set of network elements into multiple clusters, where each cluster is represented by a representative network element. Hyperparameter tuning is performed only on these representative elements rather than every individual element, reducing computational complexity while maintaining model performance through cluster-based generalization.
Solution Approach 2:
The patent creates representative network elements that serve as copies or proxies for groups of similar network elements. These representatives are used for hyperparameter tuning, and the resulting models are then applied to all elements within each cluster, avoiding the need to tune every individual element while maintaining predictive accuracy.
2Reliability
If hyperparameter tuning is performed for every network element model, then model performance is maximized, but processing time increases significantly
Solution Approach 1:
The patent divides the network elements into clusters and performs hyperparameter tuning only on representative elements from each cluster. This segmentation approach reduces the total number of tuning operations from hundreds of thousands to a manageable number, significantly decreasing processing time while maintaining model accuracy through cluster-based generalization.
Solution Approach 2:
The patent performs hyperparameter tuning on representative network elements in advance, before actual forecasting is needed. These pre-tuned representative models serve as templates that can be quickly applied to all elements in their respective clusters, eliminating the need for time-consuming individual tuning during operational forecasting.
3Measurement precision
If individual modeling is performed for each network element, then forecasting accuracy is improved, but scalability is reduced
Solution Approach 1:
The patent segments network elements into clusters based on similarity metrics, allowing individualized modeling within each cluster while maintaining scalability across the entire network. This approach preserves forecasting accuracy for individual elements through cluster-specific representative models while enabling system-wide scalability by avoiding the need for fully individualized modeling of every element.
Solution Approach 2:
The patent changes the modeling parameter from individual element-specific parameters to cluster-level representative parameters. By tuning hyperparameters at the cluster representative level rather than individual element level, the system achieves scalability while maintaining sufficient forecasting accuracy through the representative nature of clustered models.
Data Source
AI summary
A method, a network device, and a non-transitory computer-readable storage medium are described in relation to a scalable machine learning modeling service. The scalable machine learning modeling service including receiving analytics datasets pertaining to a network services metric and collected for a plurality of network elements over past intervals; performing a clustering procedure to generate a plurality of clusters formed using the analytics data for the plurality of network elements, each cluster including multiple network elements; selecting a representative network element of the multiple network elements for each of the clusters; determining a set of model hyperparameters based on searching of hyperparameter spaces associated with each representative network element; applying the set of the model hyperparameters for tuning each of the multiple network elements in a cluster; and model training each of the multiple network elements in the cluster to score the network services metric for upcoming intervals.