Aggregating Unique Training Data for Rare Event Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models face challenges in producing accurate predictions when trained with insufficient or randomly generated data, leading to unreliable results, especially for rare events not adequately represented in the training data.
Innovation Solution
The approach involves identifying unique feature sets from multiple datasets, training a special machine learning model to generate intermediate predictions, and ensembling these with predictions from a naïve model to improve the accuracy of the final predicted target variable, thereby addressing sparsity in training data for rare events.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If training data augmentation or synthetic data generation is used to obtain additional training data, then the quantity of training data increases, but the reliability of machine learning model predictions deteriorates because the generated data may not be applicable to real world situations
Solution Approach 1:
The patent copies and aggregates actual training data from multiple datasets rather than generating synthetic data. It identifies and extracts feature sets from other datasets that contain rare events, creating a specialized training dataset that preserves real-world applicability while increasing the quantity of relevant training examples.
Solution Approach 2:
The patent merges training data from multiple different datasets by identifying and extracting feature sets related to rare events. It combines data from the original dataset with data from other datasets that contain similar rare event patterns, creating a unified training dataset that maintains reliability while increasing quantity.
2Quantity of substance
If legacy techniques generate additional training data at random, then the quantity of training data increases, but the applicability to real world situations deteriorates
Solution Approach 1:
The patent applies local quality by selectively extracting and extracting feature sets from specific portions of other datasets that contain rare events. Instead of randomly generating data or using all data uniformly, it focuses on specific local regions (feature sets) within other datasets that are relevant to the rare events of interest, ensuring the generated training data maintains real-world applicability.
3Measurement precision
If more training data is collected and processed, then the accuracy of machine learning models improves, but the computational resources and time required increase
Solution Approach 1:
The patent extracts only the necessary feature sets from multiple datasets that are relevant to rare events. Instead of processing entire datasets or all available data, it selectively extracts and extracts only the feature sets that contain information about rare events, significantly reducing computational requirements while maintaining prediction accuracy for these specific events.
Solution Approach 2:
The patent segments the training data process into identifying rare events, extracting relevant feature sets from multiple datasets, and creating a specialized training dataset. This segmentation allows the system to focus computational resources only on the specific task of training for rare events rather than processing all available data uniformly.
Data Source
AI summary
Methods, systems, and computer-readable media that include processes related to training improved machine learning models using aggregated and unique training data. In one example, a process includes identifying a rare event for which a predicted target variable is to be generated, training a naïve machine learning model to generate the predicted target variable using a particular subset of the training data, and determining that a sparsity of training data exists for the rare event in the particular subset of training data. The process also includes the actions of, in response to determining that a sparsity off training data exists for the rare event in the particular subset of the training data, selecting other subsets of the training data, and identifying feature sets that are associated with the rare event from the selected other subsets of the training data.