Ensemble Predictor Calibration Using Explanation-Based Coefficients
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing predictor calibration techniques are limited to differentiable machine learning models and cannot accommodate non-differentiable explanation scores like the Shapley value, lacking versatility in calibrating prediction bases.
Innovation Solution
A predictor creation device that combines multiple predictors to form an ensemble, using a non-differentiable objective function to determine combination coefficients, allowing calibration for any explanation score and machine learning model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If a normalization term is added to the objective function during model training to calibrate the prediction basis, then the prediction basis characteristic is improved, but the technique can be applied only to differentiable machine learning models such as neural networks, reducing versatility
Solution Approach 1:
The calibration process is segmented into two distinct phases: (1) training phase where predictors are trained using traditional differentiable optimization, and (2) calibration phase where combination coefficients are determined using non-differentiable optimization based on explanation scores. This segmentation allows each phase to use the most appropriate optimization method for its specific requirements.
Solution Approach 2:
The patent introduces combination coefficients as an intermediary layer between individual predictors and the final ensemble prediction. These coefficients are optimized based on explanation scores (such as Shapley values) to calibrate the prediction basis, while the individual predictors themselves remain unchanged and can be any type of machine learning model.
2Productivity
If traditional calibration techniques are used that rely on differentiable objective functions, then optimization can be performed efficiently during training, but non-differentiable explanation scores like Shapley value cannot be used, limiting the types of predictors that can be calibrated
Solution Approach 1:
Individual predictors are trained in advance using efficient differentiable optimization methods. After training, their combination coefficients are determined through a separate calibration process that uses non-differentiable explanation scores. This preliminary action allows each predictor to be optimized independently before ensemble calibration.
Solution Approach 2:
The patent changes the optimization parameter from model weights (which require differentiability) to combination coefficients (which do not). This parameter change allows the use of non-differentiable explanation scores while maintaining calibration efficiency through modern optimization algorithms.
Data Source
AI summary
A predictor creation device including a processor configured to execute a program and a storage device that stores the program acquires a calibration target ensemble predictor obtained by combining a plurality of predictors based on a training data set which is a combination of training data and ground truth data, calculates a prediction basis characteristic related to a feature of the training data for each of the plurality of predictors, acquires an expected prediction basis characteristic related to the feature based on the prediction basis characteristic related to the feature as a result of outputting the prediction basis characteristic related to the calculated feature, determines a combination coefficient of each of the plurality of predictors based on the prediction basis characteristic related to the feature and the expected prediction basis characteristic related to the feature, and calibrates the calibration target ensemble predictor based on the combination coefficient.