Hierarchical Forecasting System for Sparse Data Scenarios
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data forecasting methods face challenges when there is insufficient historical data, leading to inaccurate predictions, especially for new or sparse data scenarios.
Innovation Solution
A computer-implemented method and system that assesses the sufficiency of historical data and leverages a categorical hierarchy to build machine learning models by combining data from ancestor nodes or similar nodes to generate forecasts, using techniques like stacking multiple models to improve accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional forecasting methods are used with insufficient historical data, then forecast accuracy deteriorates, but if hierarchical data aggregation is applied, then forecast reliability improves
Solution Approach 1:
The patent combines historical data from multiple hierarchical levels (subject level, ancestor node levels, and peer node levels) to create a comprehensive training dataset. This merging of data sources ensures sufficient training data is available while maintaining forecast reliability through the hierarchical structure that preserves subject-specific patterns.
Solution Approach 2:
The patent implements a nested hierarchical structure where subject-level data is embedded within ancestor node data, which is in turn embedded within peer node data. This nesting allows the model to learn patterns at multiple scales simultaneously, improving both reliability and accuracy by capturing both specific and general patterns.
2Ease of manufacture
If subject-specific machine learning models are built with limited historical data, then model training becomes infeasible, but if data from ancestor nodes is incorporated, then model building becomes feasible
Solution Approach 1:
The patent applies local quality by incorporating data from nodes at the same hierarchical level (peers) that share similar characteristics with the subject. This allows the model to learn subject-specific patterns from comparable entities while still having sufficient training data, thus maintaining subject-specific information quality.
Solution Approach 2:
The patent creates a multi-functional data aggregation strategy that universally applies to any subject in the hierarchy. The same hierarchical data collection and model training process can be applied to any subject node, making the system universally applicable while adapting to each subject's specific data availability and hierarchical position.
3Quantity of substance
If hierarchical data aggregation from multiple levels is performed, then data sufficiency improves, but system complexity increases
Solution Approach 1:
The patent segments the data aggregation process into distinct hierarchical levels (subject level, ancestor node levels, and peer node levels). This segmentation allows the system to systematically collect and process data from each level independently, managing complexity through structured organization while maximizing data quantity from the hierarchy.
Data Source
AI summary
A system and method for generation of automated forecasts for a subject based on one or more input parameters. The subject located at an end node of a hierarchy. The method includes: receiving historical data associated with the subject; determining the sufficiency of the historical data based on a feasibility of building a machine learning model to generate a forecast with a predetermined level of accuracy using the historical data; building the machine learning model using the historical data when there is sufficiency of the historical data; building the machine learning model using historical data associated with an ancestor node on the hierarchy when there is not sufficiency of the historical data; generating a forecast for the subject using the machine learning model based on the one or more input parameters; and outputting the forecast.