Aggregated Data Stream Forecasting with Segmented Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data modeling techniques face challenges in building accurate models with granular data due to high noise and increased storage and processing costs, especially when dealing with irregularly spaced observations and varying pricing periods.
Innovation Solution
The method involves aggregating data to a lower common multiple for modeling, combining a daily sales model with an intra-day demand pattern model, using electronic processors to generate outputs at different frequencies, and multiplying outputs to achieve granular forecasts while minimizing storage and processing resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If data is aggregated to a finer granularity (e.g., hourly observations), then forecast precision is improved, but model accuracy deteriorates due to increased noise
Solution Approach 1:
The patent segments the forecasting task into two distinct components: a daily aggregation model that ensures accuracy by working with coarse-grained data, and an intra-day pattern model that captures temporal patterns. These segmented models work together through multiplication to produce the final forecast, allowing each component to operate at its optimal granularity level without suffering from noise issues.
2Measurement precision
If data is aggregated to a finer granularity, then more detailed forecasts are obtained, but storage costs increase
Solution Approach 1:
The patent segments the data storage and processing into two parts: daily aggregated data for the aggregation model and pooled historical patterns for the intra-day model. This segmentation allows the system to maintain detailed forecast outputs without storing large volumes of fine-grained raw data, significantly reducing storage costs while preserving forecast granularity.
3Measurement precision
If data is aggregated to a finer granularity, then more detailed forecasts are obtained, but processing resources increase
Solution Approach 1:
The patent segments the processing workload into two manageable components: a daily aggregation model that processes summarized data and an intra-day pattern model that applies pre-computed temporal patterns. This segmentation dramatically reduces processing resources compared to building a single fine-grained model, as each component operates on aggregated or pooled data rather than raw fine-grained data.
4Quantity of substance
If fewer data points are used for processing, then storage and computation costs are reduced, but data accuracy deteriorates
Solution Approach 1:
The patent applies preliminary action by pre-computing and storing intra-day demand patterns from pooled historical data. These pre-computed patterns capture temporal variations without requiring real-time processing of fine-grained data. When generating forecasts, the system multiplies the daily model output by these pre-computed patterns, achieving accurate detailed forecasts while using fewer data points during actual processing.
Data Source
AI summary
Methods and systems for data modelling. One method includes receiving a data stream including a first plurality of data points, aggregating the first plurality of data points to a second plurality of data points including values at a first frequency, and building, with at least one electronic processor, a first model based on the second plurality of data points, wherein the first model is configured to generate data values at the first frequency. The method also includes accessing a second model based on a third plurality of data points, wherein the second model is configured to generate data values at a second frequency shorter than the first frequency, generating a first data output using the first model, generating a second data output using the second model, multiplying the first data output by the second data output to generate a third data output, and outputting the third data output for display.