Learning Device Softmax Conversion Order Inversion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In Gradient Boosting Forest (GBF) for multi-class classification, the softmax conversion process can lead to overestimation of incorrect class probabilities, making precise learning difficult due to the addition of only positive values, which can result in probabilities exceeding 1 and being difficult to correct.
Innovation Solution
A learning device and method that acquires outputs from multiple learners, performs a weighted addition of these outputs using a combination coefficient, and then applies a conversion process like softmax conversion to express probabilities, ensuring the total value remains within a valid range and allows for correction of overestimated probabilities in subsequent rounds.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If softmax conversion is performed on the output of each decision tree before addition, then the probability values are normalized to fall within 0 to 1, but only positive values are added in each round making it difficult to correct overestimated probabilities of incorrect classes
Solution Approach 1:
The patent inverts the conventional order of operations by performing addition of decision tree outputs before applying softmax conversion, rather than converting before addition. This inversion allows the model to add raw output values (which can be negative) and then normalize, enabling correction of overestimated probabilities in subsequent rounds while maintaining proper probability normalization.
2Reliability
If outputs from multiple learners are combined using Gradient Boosting Forest, then the classification performance is improved, but the probability values may exceed 1 and become difficult to correct
Solution Approach 1:
The patent applies softmax conversion as a preliminary action after combining multiple learner outputs but before using the probabilities for final classification decisions. This ensures that probability values are normalized to the valid range (0 to 1) at the appropriate stage, preventing probability values from exceeding 1 while maintaining the benefits of ensemble learning.
Data Source
AI summary
A learning device includes an acquisition unit and a conversion unit. The acquisition unit inputs thereto, for each class, an output of a learner for each class received from another learning device and an output of a learner for each class trained by the own device, and acquires a given output for each class. The conversion unit performs, for each class, a conversion process to express a probability with respect to the given output for each class acquired by the acquisition unit.