Interpretable Tree-Based Regression for Outlier-Resilient Process Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for training optimal decision trees (ODTs) using mixed-integer linear programming (MILP) are computationally intractable for large datasets, and current loss functions, particularly squared loss, fail to effectively remove outliers, leading to suboptimal generalization and interpretability issues in process control systems.
Innovation Solution
A binary mixed-integer linear programming (MILP)-based training scheme that simultaneously trains a network of decision tree regression models with a novel nonlinear loss function to filter out outliers, ensuring efficient and interpretable predictions by using multivariate hyperplanes and gating functions to refine input data before training downstream trees.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If MILP is used to train ODTs, then global solutions can be determined that are scalable, but the method becomes computationally intractable for large scale data sets
Solution Approach 1:
The patent segments the training process into two stages: a coarse-grained MILP optimization to determine the decision tree structure and splitting rules, and a fine-grained linear regression stage to fit the leaf node values. This segmentation allows the computationally intensive MILP to operate on a simplified representation (tree structure) rather than the full dataset, making it tractable for large-scale data while maintaining global optimality of the tree architecture.
Solution Approach 2:
The patent performs preliminary actions by using MILP to first determine the optimal decision tree structure, splitting rules, and node assignments before performing the actual regression fitting. This preliminary structuring reduces the complexity of subsequent computations, as the tree framework is already established and only the leaf values need to be fitted using simpler linear regression.
2Ease of manufacture
If squared loss is used for training ODTs, then the loss function is simple, but outliers cannot be removed and generalization behavior is limited
Solution Approach 1:
The patent changes the loss function parameter from squared loss to absolute loss (L1 norm). This parameter change transforms the optimization problem from being sensitive to outliers (squared loss penalizes large errors quadratically) to being robust to outliers (absolute loss penalizes errors linearly). The modified loss function is: minimize Σ|x_i - f(x_i)| instead of minimize Σ(x_i - f(x_i))², which effectively downweights the influence of outliers on the model fitting process.
3Measurement precision
If deep neural networks are used for process control, then prediction accuracy is high, but interpretability is lost due to black box hidden layers
Solution Approach 1:
The patent substitutes the neural network mechanism with a decision tree mechanism. Instead of using complex non-linear transformations through hidden layers (neural network approach), the patent employs a tree-structured model with explicit splitting rules and leaf node predictions. This substitution maintains prediction accuracy while providing full interpretability, as the decision tree structure explicitly shows the logic flow, splitting conditions, and prediction rules without any hidden computational layers.
Data Source
AI summary
Aspects of the invention include training an optimal interpretable decision tree for regression using mixed-integer linear programming techniques. A non-limiting example computer-implemented method includes receiving, using a processor, input data that includes time-series data. The method further includes training, using a binary mixed-integer linear program of the processor, an ODT for regression based on the input data. During the training process one or more outliers are filtered out by a linear loss model that minimizes training loss and outlier loss.