Diverse Ensemble Training via Distinct Targets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
As the number of ensemble members in machine learning systems increases, achieving diversity and incremental improvement becomes increasingly difficult, leading to declining performance gains.
Innovation Solution
Designing a training target for each ensemble member as a distinct function of input, enabling supervised training for diversity using unlabeled data, and employing a joint optimization network to optimize the ensemble's performance by controlling the training of members to support each other.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the number of ensemble members is increased, then the performance improvement is greater, but the diversity becomes increasingly difficult to achieve and incremental improvement quickly declines
Solution Approach 1:
The patent segments the training objective into distinct targets for each ensemble member. Each member is assigned a unique target function that differs from others, enabling individual specialization while maintaining overall ensemble performance. This segmentation allows large ensembles to achieve diversity naturally through differentiated training objectives rather than requiring complex diversity enforcement mechanisms.
Solution Approach 2:
The patent applies local quality by giving each ensemble member its own unique training target tailored to specific aspects of the input data. Instead of uniform training for all members, each member optimizes for its designated target, creating local expertise that contributes to overall ensemble reliability while maintaining simplicity in the training process.
2Reliability
If the number of ensemble members is increased, then the performance improvement is greater, but the incremental improvement quickly declines
Solution Approach 1:
By segmenting training objectives across ensemble members, the patent enables scalable performance improvement. Each member contributes to performance gain through its specialized target, and the modular structure ensures that adding more members continues to yield improvements without the diminishing returns associated with conventional ensemble methods.
Solution Approach 2:
The patent changes the training parameter landscape by introducing distinct target functions for each ensemble member. This parameter transformation enables continuous performance improvement as more members are added, since each new member brings a previously unexplored target dimension, avoiding the saturation point that limits incremental gains in traditional approaches.
Data Source
AI summary
A computer-implemented method of training an ensemble machine learning system comprising a plurality of ensemble members. The method includes selecting a shared objective and an objective for each of the ensemble members. The method further includes training each of the ensemble members according to each objective on a training data set, connecting an output of each of the ensemble members to a joint optimization machine learning system to form a consolidated machine learning system, and training the consolidated machine learning system according to the shared objective and the objective for each of the ensemble members on the training data set. The ensemble members can be the same or different types of machine learning systems. Further, the joint optimization machine learning system can be the same or a different type of machine learning system than the ensemble members.