Covariance-Based Data Downsampling for Model Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine learning model training processes are resource-intensive and inefficient, as they often require large datasets with irrelevant data points, leading to increased computational demands and reduced prediction accuracy.
Innovation Solution
A system and method for downsampling training data by learning a covariance function, selecting a subset of data points with high covariance to the prediction data point, and using this subset to train a model, thereby reducing computational resources and improving prediction accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If large datasets are used for model training, then prediction accuracy is improved, but computational resource consumption increases
Solution Approach 1:
The patent extracts only the most relevant data points from the large dataset based on covariance analysis. By calculating the covariance between each data point and the prediction target, the system identifies and extracts the top K data points with highest covariance values, discarding irrelevant data. This extraction principle resolves the contradiction by maintaining prediction accuracy through selective data retention while reducing computational resource consumption by eliminating unnecessary data processing.
Solution Approach 2:
The patent applies local quality by treating different data points differently based on their individual covariance values with the prediction target. Instead of uniformly processing all data points, the system assigns different importance weights to each data point, focusing computational resources on high-covariance regions. This localized approach ensures that computational resources are concentrated on the most informative data points, thereby maintaining accuracy while reducing overall resource consumption.
2Measurement precision
If large datasets are used for model training, then model performance is improved, but training time increases
Solution Approach 1:
The patent performs preliminary covariance calculation and data point ranking before the actual model training process. By pre-identifying and sorting data points based on their covariance values with the prediction target, the system prepares an optimized subset of training data in advance. This preliminary action resolves the contradiction by ensuring that only the most relevant data points are included in the training set, thereby reducing training time while maintaining model performance.
Solution Approach 2:
The patent extracts the top K data points with highest covariance values from the complete dataset before training. This extraction creates a condensed training subset that contains only the most informative samples. By performing this extraction beforehand, the system reduces the training workload and time requirement while preserving the essential information needed for high model performance.
3Quantity of substance
If irrelevant data points are included in training data, then data quantity is increased, but prediction accuracy decreases
Solution Approach 1:
The patent applies local quality by differentiating between relevant and irrelevant data points through covariance analysis. Each data point is evaluated individually based on its covariance value with the prediction target, and only those exceeding a threshold or ranking in the top K are selected. This selective approach ensures that data quantity is optimized to include only high-quality, relevant samples, thereby preventing the degradation of prediction accuracy that would result from including irrelevant data.
Solution Approach 2:
The patent changes the selection parameter from uniform inclusion to covariance-based ranking. By using covariance value as the selection criterion and sorting data points accordingly, the system transforms the training data selection process from a quantity-focused approach to a quality-focused approach. This parameter change ensures that the included data points are those most strongly correlated with the prediction target, thereby maintaining high prediction accuracy even with reduced data quantity.
Data Source
AI summary
Systems, methods, and other embodiments described herein relate to downsampling training data so as to simplify the training of models as well as increase the prediction accuracy of the models. In one embodiment, a method includes training a model on a dataset to learn a covariance function, determining a covariance between a selected data value and the dataset using the covariance function, selecting a subset from the dataset based on the covariance, and predicting one or more potential experiments based on the subset.