In-Database Scalable Regression for Retail Panel Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The computational intensity of performing regressions for demand models in Retail Science, due to a large number of demand variables, leads to unreasonable or unreliable results and is computationally intensive, making it difficult to perform within standard database systems.
Innovation Solution
A two-step method is employed to transform the demand model and historical data, allowing regressions to be performed in-database using standard database functions, leveraging native parallel processing, by constructing double-ratio techniques that cancel out base demand and seasonality effects, enabling the estimation of elasticity parameters using SQL code for efficient computation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a separate regression is performed for each segment of retail panel data (by item and sales location), then accurate sales forecasts are produced, but the computational complexity and time required greatly increase
Solution Approach 1:
The patent applies segmentation by dividing the retail panel data into distinct segments based on item and sales location characteristics. This allows regressions to be performed on smaller, more manageable subsets of data while maintaining forecast accuracy for each segment. The segmentation strategy reduces the overall computational burden compared to processing all data uniformly.
Solution Approach 2:
The patent merges computation with database storage by performing regressions in-database rather than transferring data to external systems. This integration allows the database system to leverage its native parallel processing capabilities and avoids the overhead of data transfer, thereby improving computational efficiency while maintaining forecast accuracy.
2Measurement precision
If thousands or hundreds of thousands of regressions are performed for each retail panel data segment, then accurate demand parameters are determined, but the computational intensity becomes unmanageable
Solution Approach 1:
The patent performs preliminary actions by pre-processing and transforming the retail panel data into appropriate formats before regression analysis. This includes organizing data by segments and preparing it for in-database processing, which reduces the complexity of the actual regression computations and enables efficient parallel execution.
Solution Approach 2:
The patent replaces traditional external computation systems with database-native parallel processing capabilities. By leveraging the database system's built-in parallel execution engine, the computationally intensive regression tasks are distributed and executed efficiently within the database environment, reducing overall computational complexity.
3Ease of operation
If retail panel data is transferred to external computation systems for regression analysis, then regressions can be performed, but data transfer time and computational overhead increase
Solution Approach 1:
The patent implements self-service by enabling the database system to perform regression analysis directly on stored retail panel data without requiring external computation systems. The database leverages its own parallel processing capabilities to execute regressions in-database, eliminating data transfer overhead and reducing total computation time while maintaining operational ease.
Data Source
AI summary
Systems, methods, and other embodiments associated with scalable regression for retail panel are described. In one embodiment, a method includes performing a regression that estimates elasticity of demand for a retail item, wherein the regression is performed on a transformation of a demand model that does not include variables associated with base demand or seasonality. In a subsequent processing step, the method includes estimating a base demand and seasonality for the retail item based, at least in part, on the estimated elasticity of demand. The method may be performed in a database that stores retail panel data for the retail item and other retail items.