Resource Usage Forecasting with RANSAC Multi-Trend Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing network security systems face challenges in robust long-term forecasting of telemetry data due to sensitivity to minor changes, data drift, overfitting, high computational cost, and data irregularities, especially in multi-trend environments with varying telemetry time intervals.
Innovation Solution
A resampled metric data pipeline using the RANSAC algorithm for iterative segmentation and a breadth-first search binary tree strategy to classify inliers and outliers, combined with max pooling and flexible regression models, enhances forecasting robustness and efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If complex forecasting models are used to handle multi-trend environments, then forecasting accuracy improves, but computational cost and complexity increase
Solution Approach 1:
The patent segments the forecasting problem by separating inlier data points from outlier data points using RANSAC algorithm. The inlier data is used for training the forecasting model while outliers are discarded. This segmentation allows simpler models to achieve better accuracy by focusing only on representative data patterns, reducing computational complexity compared to processing all data with complex models.
Solution Approach 2:
The patent extracts and removes outlier data points from the dataset using RANSAC-based outlier detection. By taking out only the problematic data points (outliers) and retaining only the inlier data for model training, the system reduces the data processing burden and enables simpler, more efficient forecasting models to achieve high accuracy.
2Ease of operation
If traditional regression models are used for long-term forecasting, then ease of operation is maintained, but reliability deteriorates due to sensitivity to data drift and minor changes
Solution Approach 1:
The patent performs preliminary action by pre-processing the data to identify and remove outliers before training the regression model. Using RANSAC algorithm in advance to segment and filter the data ensures that the subsequent regression model trains on clean, representative inlier data only, making it more reliable and less sensitive to data drift while maintaining operational simplicity.
Solution Approach 2:
The patent converts the harmful effect of outlier data points into a beneficial filtering process. By using RANSAC to detect and remove outliers, the system transforms the presence of problematic data into an opportunity to create a cleaner, more reliable training dataset, thereby improving model robustness without complicating the overall operation.
3Quantity of substance
If data is processed at original time intervals, then data completeness is maintained, but productivity decreases due to high computational cost
Solution Approach 1:
The patent extracts and removes a subset of data points (outliers) from the complete dataset, retaining only the essential inlier data for processing. This extraction reduces the volume of data that needs to be processed while maintaining the representativeness and completeness of the underlying patterns, thereby significantly improving computational efficiency and productivity.
Solution Approach 2:
The patent segments the complete dataset into inlier and outlier portions using RANSAC algorithm. By processing only the inlier segment for forecasting while discarding the outlier segment, the system maintains data completeness for the relevant patterns while reducing computational workload, thus improving productivity without sacrificing essential data integrity.
Data Source
AI summary
The present application discloses a method, system, and computer system for generating a forecast, such as a long-term capacity resource forecast, based on a forecast model for a system activity. The method includes (a) processing and recursively modelling a set of resampled metric data in connection with segmenting the metric data into relevant data and non-relevant data to obtain a forecast model for system activity, wherein the set of resampled metric data pertains to the system activity, and (b) generating a forecast based at least in part on the forecast model for the system activity.