Time Series Clustering for Scalable Item-Level Demand Forecasting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current time series forecasting methods are computationally expensive and inaccurate when applied to large datasets with thousands or millions of items, leading to infeasible computation times and information loss at the item level.
Innovation Solution
A method involving domain-based segmentation to identify statistically forecastable and non-forecastable time series, followed by clustering these series into groups for aggregate forecasting, using a two-step regression strategy with bias and variance correction, and individual forecasting for high-error items.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If time series forecasting is performed on each individual item in a large dataset, then forecasting accuracy is maintained, but computational expense and time consumption become infeasible
Solution Approach 1:
The patent segments the large dataset into multiple clusters based on similarity in demand patterns. Instead of forecasting each item individually, items within the same cluster share common forecasting models and parameters, significantly reducing computational complexity while maintaining forecasting accuracy through cluster-specific modeling.
Solution Approach 2:
The patent merges items with similar demand characteristics into the same cluster. By combining multiple items into clusters and performing aggregate forecasting at the cluster level, the system reduces the number of forecasting operations from thousands of individual items to a manageable number of clusters, thereby improving computational efficiency.
2Productivity
If forecasting is aggregated at a product category or class level, then computation expense is reduced, but information loss at the item level leads to inaccurate forecasts
Solution Approach 1:
The patent segments items into fine-grained clusters based on similarity in demand patterns rather than aggregating at coarse product category levels. This segmentation preserves item-level characteristics while still enabling computational efficiency through clustering, avoiding the information loss associated with broad category aggregation.
Solution Approach 2:
The patent applies local quality by creating clusters with homogeneous demand characteristics, where each cluster receives tailored forecasting parameters and models suited to its specific pattern. This ensures that forecasting accuracy is maintained for items with similar behaviors while still achieving computational efficiency through the clustered structure.
3Loss of information
If traditional forecasting models are applied to big data sets, then comprehensive item-level analysis is possible, but the process is time-consuming and computationally expensive
Solution Approach 1:
The patent performs preliminary clustering of items based on their demand patterns before applying forecasting models. This preliminary action groups items with similar characteristics together, so that subsequent forecasting operations can be performed efficiently at the cluster level while still capturing item-level nuances through the cluster structure.
Solution Approach 2:
The patent segments the dataset into clusters that preserve item-level characteristics through similarity-based grouping. This segmentation allows the system to process data at a reduced scale (cluster level) while maintaining the ability to retrieve and apply item-specific information when needed, thus reducing time loss without sacrificing item-level detail.
Data Source
AI summary
A method and system for time series forecasting on a big data set are provided. The method includes receiving a plurality of time series, each of the time series representing a historical demand pattern for an item, performing a domain-based segmentation to identify a plurality of statistically forecastable time series from the plurality of time series, grouping the plurality of statistically forecastable time series into one or more clusters, for each cluster, generating an aggregate time series based on time series included in the cluster, performing a future demand forecast at a cluster level by performing time series forecasting of the aggregate time series for each cluster, and determining a future demand forecast for each item based on the time series forecasting of the aggregate time series.