Augmenting Incomplete Training Datasets for Machine Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning systems require large, complete datasets for accurate training, which can hinder their effectiveness in early stages or scenarios with limited data, making it difficult to predict events like machinery failure or loan defaults without sufficient historical data.
Innovation Solution
A method to augment incomplete datasets by extracting temporal and parameter-specific data from complete datasets, creating temporal and parameter-specific augmentations that can be used alongside incomplete datasets for training machine learning models, enabling the generation of probability predictions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a machine learning system waits for complete datasets with definitive outcomes for training, then prediction accuracy is improved, but the time required to deploy the system and start making predictions increases significantly
Solution Approach 1:
The system performs preliminary actions by extracting and storing temporal data and parameter data from complete datasets before training is needed. When incomplete datasets become available, the pre-extracted information from complete datasets is immediately applied to augment the training data, eliminating the need to wait for complete datasets with definitive outcomes. This allows the machine learning system to be deployed earlier while maintaining prediction accuracy.
Solution Approach 2:
The patent applies partial action by using only the necessary components of complete datasets (temporal data and parameter data) rather than waiting for entire complete datasets with definitive outcomes. The system extracts and utilizes specific portions of data that are sufficient for training, allowing deployment before all complete datasets are available, thus reducing time loss while maintaining adequate prediction accuracy.
2Reliability
If a machine learning system uses only complete datasets for training, then prediction reliability is improved, but the quantity of usable training data decreases when data is scarce
Solution Approach 1:
The patent introduces temporal data and parameter data as intermediary elements that bridge incomplete datasets and the training requirements of machine learning systems. These intermediaries are extracted from complete datasets and applied to incomplete datasets, allowing the system to use both complete and incomplete datasets for training. This increases the quantity of usable training data while maintaining prediction reliability through the augmenting effect of the intermediary data.
Solution Approach 2:
The system changes the parameters of incomplete datasets by adding temporal data and parameter data extracted from complete datasets. This parameter transformation allows incomplete datasets to be effectively used for training machine learning models, increasing the quantity of usable training data while maintaining the reliability benefits of complete dataset information through the added parameters.
3Quantity of substance
If temporal data and parameter data are extracted and applied to incomplete datasets, then the quantity of usable training data increases, but the complexity of data processing increases
Solution Approach 1:
The patent segments the data extraction and processing into distinct, manageable components: extracting temporal data from complete datasets, extracting parameter data from complete datasets, and then applying these extracted elements to incomplete datasets. This segmentation of the data processing task reduces overall complexity by breaking down the complex operation into simpler, independent steps that can be executed separately and systematically.
Data Source
AI summary
Systems and methods for augmenting incomplete training dataset for use in a machine learning system are described herein. In an embodiment, a server computer receives a plurality of input training datasets including one or more incomplete input training datasets and one or more complete datasets which contain one or more failure training datasets, the incomplete input training datasets comprising a plurality of parameters. Using the one or more failure training datasets, the server computer generates temporal failure data describing a likelihood of failure of an item as a function of time. Using the one or more complete training datasets, the server computer generates parameter specific likelihoods of failure of an item. The server computer augments the one or more incomplete input training datasets using the temporal failure data and/or the parameter specific likelihoods of failure to create one or more augmented training datasets. The server computer uses the one or more augmented training datasets as input for training a machine learning model that is programmed to generate a probability of failure of a particular item represented by an input dataset.