Big Data Forecasting via Observation Sketches
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for forecasting mobile supply inventory based on massive datasets of mobile signals are exorbitantly expensive and time-consuming, making them impractical for efficient use in location-based information services.
Innovation Solution
A system and method that transform massive observations into categorical sketches, allowing for faster and more accurate forecasting by using a sketch generator to create observation sketches with hash signatures, and then performing set operations to determine cardinalities for forecast constraints, thereby reducing processing requirements and improving forecasting performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional forecasting methods are used on massive mobile signal datasets, then forecasting accuracy can be maintained, but processing time and computational cost become exorbitantly expensive and time-consuming
Solution Approach 1:
The patent segments the massive mobile signal dataset into multiple partitions, and further segments the forecasting task into sketch generation phase and query processing phase. By dividing the data into manageable chunks that can be processed independently and in parallel, the system achieves both accuracy and efficiency in forecasting mobile supply inventory.
Solution Approach 2:
The patent introduces observation sketches as an intermediary data structure between the raw massive dataset and the forecasting queries. These sketches serve as compressed summaries that capture essential characteristics of the data, enabling fast approximate queries while maintaining acceptable accuracy. The sketches act as a mediator that reduces the gap between data volume and processing speed.
2Reliability
If conventional forecasting methods are used on massive datasets, then comprehensive analysis can be performed, but computational cost and resource requirements become impractical
Solution Approach 1:
The patent changes the parameter representation by transforming raw mobile signal observations into sketch data structures with controlled size parameters. By adjusting sketch parameters such as sketch size and update frequency, the system can balance between computational complexity and forecasting reliability, making the system practical for deployment on standard hardware infrastructure.
3Measurement precision
If multiple passes over large datasets are performed, then accurate forecasts can be generated, but memory usage and processing overhead increase significantly
Solution Approach 1:
The patent performs preliminary action by pre-computing and storing observation sketches during off-peak hours or in the background. These pre-computed sketches are then readily available for fast query processing during peak periods. This preliminary processing eliminates the need for multiple passes over the raw data during actual forecasting operations, reducing both memory usage and processing overhead.
Data Source
AI summary
A method of forecasting comprises, for each respective time period of a plurality of time periods, transforming a plurality of observations from the respective time period into observation sketches. The method then proceeds to extract a set of targeted attributes and constraints from a forecast request, and construct a set expression using the targeted attributes and the associated constraints. The method further comprises selecting one or more sample periods from the plurality of time periods, and for each sample period of the one or more selected sample periods: mapping the set of targeted attributes to observation sketches in the sample period to identify a subset of sketches, and performing a set operation on the subset of sketches using the set expression to determine a cardinality corresponding to the sample period; and generating a forecast result using one or more cardinalities corresponding to the one or more sample periods.