Forecasting Model Validation Using Proxy Regression Features
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current techniques for validating forecasting machine learning models fail to identify important and significant features, leading to inefficient use of computing resources and handling of customer complaints due to incorrect model deployment.
Innovation Solution
A validation system that processes historical time series data and output data using a proxy regression model to determine feature importance, generates perturbed data, and evaluates the model to identify top features, thereby validating the forecasting model before deployment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If current validation techniques are used, then model deployment proceeds without detailed feature analysis, but computing resources are wasted and customer complaints increase due to incorrect models
Solution Approach 1:
The patent performs validation analysis before model deployment by creating perturbed data and training proxy models in advance. This preliminary action identifies important features and detects potential model errors before they reach production, preventing wasted resources and customer complaints while ensuring model reliability.
Solution Approach 2:
The patent introduces a proxy regression model as an intermediary to analyze feature importance and validate the forecasting model. This intermediary model processes perturbed data and provides validation insights without requiring direct analysis of the main forecasting model, enabling efficient pre-deployment validation.
2Reliability
If comprehensive validation is performed, then model quality improves, but computing resources and time are consumed during validation
Solution Approach 1:
The patent creates perturbed copies of the original training data by introducing random variations. These copied and modified datasets are used to train proxy models for validation purposes, enabling comprehensive model quality assessment without requiring exhaustive analysis of the original model, thus reducing validation time while maintaining reliability.
Solution Approach 2:
The patent changes data parameters by creating perturbed versions of the training data with modified values. This parameter transformation enables the proxy model to learn feature importance patterns efficiently, providing comprehensive validation insights in reduced time compared to analyzing the original model directly.
3Ease of repair
If feature importance analysis is performed, then model debugging improves, but validation complexity increases
Solution Approach 1:
The patent uses a proxy regression model as an intermediary to perform feature importance analysis. This intermediary simplifies the complex task of analyzing the main forecasting model by creating a simpler proxy that captures the essential feature-importance relationships, making debugging easier while managing validation complexity.
Solution Approach 2:
The patent extracts feature importance information from the proxy model trained on perturbed data. By separating the validation analysis from the main model and extracting only the essential feature importance metrics, the system simplifies the debugging process while avoiding the complexity of comprehensive model analysis.
Data Source
AI summary
A device may receive historical time series data and output data associated with a forecasting model and may process the historical time series data and the output data, with a proxy regression model, to determine inference data. The device may create perturbed data from the historical time series data, the output data, and the inference data, and may process the perturbed data, with the proxy regression model, to generate labelled data and to identify top features of the labelled data. The device may process subsets of the top features of the labelled data, with the proxy regression model, to determine feature data identifying an importance of each of the subsets of the top features, and may evaluate the proxy regression model and the feature data to calculate validation data for validating the forecasting model. The device may validate the forecasting model with the validation data.