Ensemble Model Generation from Multiple Data Sources
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data mining techniques require extensive domain knowledge and data merging, making it cumbersome to generate predictive models from multiple data sources efficiently.
Innovation Solution
A method that generates validation and holdout samples from each data source, merges them into global datasets, determines accuracy values for base models, and creates an ensemble model from selected subsets, allowing for efficient predictive model generation without merging all data sources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all data sources are merged to generate predictive models, then comprehensive analysis is achieved, but system complexity and processing time increase significantly
Solution Approach 1:
The patent divides the data processing task into segments by creating separate base models for each data source rather than merging all data first. Each data source is processed independently to generate its own validation sample, holdout sample, and base model, which reduces the complexity of data merging while maintaining comprehensive analysis capabilities through the ensemble model that combines these segmented results.
2Reliability
If all data sources are merged to generate predictive models, then comprehensive analysis is achieved, but processing time increases significantly
Solution Approach 1:
The patent performs preliminary actions by generating validation samples, holdout samples, and base models for each data source independently before combining them. This preliminary processing allows parallel execution of model generation tasks, reducing overall processing time compared to sequential merging of all data sources followed by model generation.
3Reliability
If domain knowledge is required for data mining applications, then model accuracy improves, but ease of operation decreases
Solution Approach 1:
The system performs self-service by automatically generating base models, evaluating their performance, and constructing ensemble models without requiring users to provide domain knowledge. The framework autonomously handles data preprocessing, model generation, validation, and combination, making the system easy to operate while maintaining high accuracy through its intelligent model selection and ensemble construction mechanisms.
Data Source
AI summary
Techniques are disclosed for generating an ensemble model from multiple data sources. In one embodiment, the ensemble model is generated using a global validation sample, a global holdout sample and base models generated from the multiple data sources. An accuracy value may be determined for each base model, on the basis of the global validation dataset. The ensemble model may be generated from a subset of the base models, where the subset is selected on the basis of the determined accuracy values.