ML Model Training with Parameter Expanded Data for Sparse Environments
Solution Overview
Problem
Current machine learning models face challenges in generating accurate predictions in sparse data environments, particularly in industries like aerospace where data on various unit prices is limited, leading to incorrect assumptions about demand and elasticity.
Innovation Solution
The system transforms raw historical data into parameter expanded data by aggregating historical values across attribute values, allowing machine learning models to generate predictions on unrepresented price points, thus overcoming data scarcity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If machine learning models are trained using raw historical data in sparse data environments, then the model training process is simple, but the prediction accuracy deteriorates due to incorrect assumptions about demand and elasticity
Solution Approach 1:
The system performs preliminary data transformation by aggregating historical values across attribute values before training the machine learning model. This pre-processing step creates parameter expanded data that enriches the training dataset, allowing the model to learn accurate demand and elasticity relationships even from sparse original data. The aggregation operation is performed in advance to ensure the model receives sufficiently detailed training information.
Solution Approach 2:
The patent transforms one-dimensional raw historical data into multi-dimensional parameter expanded data by aggregating across multiple attribute values. This dimensional transformation expands the feature space, creating additional dimensions of information that enable accurate predictions. The aggregation process creates a multi-dimensional representation of demand patterns that was not present in the original sparse data.
2Measurement precision
If machine learning models are trained using aggregated parameter expanded data, then prediction accuracy on unrepresented price points improves, but the data processing time increases
Solution Approach 1:
The aggregation of historical values into parameter expanded data is performed as a preliminary action before model training. By completing this data transformation in advance, the system prepares enriched training data that can be directly fed to the machine learning model without requiring complex real-time processing. This pre-computation approach reduces the computational burden during the actual prediction phase.
3Quantity of substance
If historical data is aggregated across all attribute values, then sufficient training data is generated for accurate predictions, but the granularity of individual data points is lost
Solution Approach 1:
The system performs aggregation as a preliminary action that consolidates historical values across attribute values to create parameter expanded data. This aggregation is done in a controlled manner that preserves the relationship between different attribute values while increasing the overall quantity of training data. The aggregation operation maintains the structural information needed for accurate predictions.
Solution Approach 2:
The patent merges historical values across multiple attribute values into parameter expanded data structures. This merging process combines multiple data points while preserving their relational information, creating a richer training dataset that maintains granularity through the aggregated structure rather than through individual raw data points.
Data Source
AI summary
Methods, apparatuses, and computer program products for generating predictions using machine learning model(s) are provided. For example, a computer-implemented method may include receiving raw historical data identifying historical value(s) corresponding to attribute value(s) of a set of ordered attribute values and transforming the raw historical data into parameter expanded data by, for each particular attribute value of the set of ordered attribute values, aggregating each of the historical value(s) that corresponds to the particular attribute value and each of the historical value(s) that corresponds to an attribute value that is greater than the particular attribute value. The machine learning model(s) are trained based on the parameter expanded data, which trained machine learning model(s) are used to generate the predictions.