Interpretable Differentiable Models for Industrial Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models in manufacturing and process industries face challenges in achieving both interpretability and differentiability, leading to inaccurate predictions with potential real-world consequences, and there is a need for models that can be both differentiable and interpretable to ensure safety and accuracy in industrial applications.
Innovation Solution
The development of 'Indi learning' methods, which generate tree-based models that are both highly interpretable and either locally or globally differentiable, using a novel approach that combines oblique decision trees with cross-validated regression models and local weighting schemes to create models that can predict behavior of real-world systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If high-accuracy machine learning models (such as XGBoost or deep learning models) are used, then predictive accuracy is improved, but interpretability deteriorates as they can only be interpreted indirectly through techniques such as variable importance or Shapley values
Solution Approach 1:
The patent segments the data space using decision trees to create interpretable regions, while within each region uses local regression models to maintain high predictive accuracy. This segmentation allows the model to be both accurate and interpretable by combining global structure with local flexibility.
Solution Approach 2:
The patent applies local quality by using different modeling approaches in different regions of the data space. High-accuracy deep learning models are used where complex patterns exist, while simpler interpretable models are used in regions where data follows clearer patterns, optimizing both accuracy and interpretability locally.
2Measurement precision
If high-accuracy machine learning models (such as XGBoost) are used, then predictive accuracy is improved, but differentiability deteriorates as these models are not globally continuously differentiable
Solution Approach 1:
The patent segments the modeling approach into global decision tree structures and local regression components. The regression components are designed to be differentiable, providing gradients for optimization while maintaining the interpretability of tree structures, thus achieving both accuracy and differentiability.
Solution Approach 2:
The patent introduces an intermediary layer between the non-differentiable tree structures and the optimization process. This intermediary uses differentiable regression models that can provide gradients while being guided by the tree structure, enabling backpropagation and gradient-based optimization.
3Loss of information
If statistical regression models are used, then interpretability and differentiability are improved, but predictive accuracy deteriorates as they may not attain the same level of accuracy as deep learning models
Solution Approach 1:
The patent merges the strengths of interpretable statistical regression models with the predictive power of deep learning models. By combining tree-based segmentation with local regression, it achieves both interpretability and high accuracy that neither approach can achieve alone.
Solution Approach 2:
The patent creates a composite modeling approach that combines different model types (trees and regression) into a unified framework. This composite model leverages the interpretability of trees and the accuracy of regression models to achieve superior performance on both dimensions.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Embodiments create models configured to predict behavior of real-world systems. An example embodiment receives input and output data for a real-world system and, next, subdivides the input and output data received into a plurality of subsets in accordance with a criterion. For each subset of the plurality, a regression model is fit to data of the subset. For each data point in each subset of the plurality of subsets, a respective weight is assigned to the data point for each regression model. In turn, the model configured to predict the behavior of the real-world system is generated by calculating a weighted average of each regression model using the assigned respective weights.