Predictability-Driven Training Data Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing size of training data sets for machine learning models poses challenges in processing and storage on devices with limited capacity, efficient communication in distributed environments, and enabling real-time model availability.
Innovation Solution
A technique for compressing training data sets by computing a predictability measure for each data instance, filtering out high predictability instances, and generating a compressed data set that maintains or improves prediction accuracy, using a workflow involving a data set compression module with a simple ML model, predictability computation, and filtering components.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the size of training data sets is increased to improve model performance, then prediction accuracy is improved, but storage capacity and processing complexity increase
Solution Approach 1:
The patent extracts and removes high-predictability data instances from the training set, retaining only low-predictability instances that provide the most valuable learning signals. This extraction approach reduces the overall data quantity while preserving the essential information needed for accurate model training, directly resolving the contradiction between data size and storage processing requirements
Solution Approach 2:
The patent changes the parameter of data selectivity by using predictability measures to identify and filter specific data instances. By transforming the training data selection process into a parameter-driven filtering mechanism based on predictability scores, the system achieves reduced data volume without sacrificing training effectiveness
2Measurement precision
If the size of training data sets is increased to improve model performance, then prediction accuracy is improved, but communication efficiency in distributed environments deteriorates
Solution Approach 1:
The patent extracts only the most informative data instances for distribution across computing systems. By removing high-predictability instances that provide minimal learning value, the system reduces the communication burden in distributed training environments while maintaining the accuracy improvements necessary for effective model training
3Measurement precision
If the size of training data sets is increased to improve model performance, then prediction accuracy is improved, but real-time model availability deteriorates
Solution Approach 1:
The patent extracts a subset of training data that maintains the essential learning signals while significantly reducing the total data volume. This extraction enables faster model training and deployment while preserving prediction accuracy, thereby resolving the contradiction between data quantity and real-time model availability
Solution Approach 2:
The patent performs preliminary filtering of training data to identify and remove high-predictability instances before the actual model training process. This preliminary action reduces the effective training data size, enabling faster training cycles and real-time model deployment without sacrificing the accuracy benefits of comprehensive training
Data Source
AI summary
Techniques for performing predictability-driven compression of training data sets used for machine learning (ML) are provided. In one set of embodiments, a computer system can receive a training data set comprising a plurality of data instances and can train an ML model using the plurality of data instances, the training resulting in a trained version of the ML model. The computer system can further generate prediction metadata for each data instance in the plurality of data instances using the trained version of the ML model and can compute a predictability measure for each data instance based on the prediction metadata, the predictability measure indicating a training value of the data instance. The computer system can then filter one or more data instances from the plurality of data instances based on the computed predictability measures, the filtering resulting in a compressed version of the training data set.