Conditional Conformal Prediction Intervals Using Interpretable Error Regions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conformal prediction coverage is challenging in machine learning regression modeling due to significant variation in error distributions, leading to poor performance in heteroscedastic uncertainty scenarios.
Innovation Solution
Computing a conditional conformal prediction interval by identifying interpretable error regions in the input space and using quantile regression to condition the prediction intervals on these regions, allowing for improved coverage without requiring parametric knowledge of the error distribution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conformal prediction intervals are constructed without conditioning on error regions, then the method is simple and does not require parametric knowledge, but the prediction coverage is poor under heteroscedastic uncertainty
Solution Approach 1:
The input space is segmented into multiple error regions based on quantile regression, where each region has relatively homogeneous error characteristics. This segmentation allows the conformal prediction to adapt to local error distributions while maintaining the overall simplicity of the non-parametric approach.
Solution Approach 2:
Instead of applying a uniform conformal prediction interval across the entire input space, the method applies locally-adapted intervals within each error region. This local quality approach improves prediction coverage in heteroscedastic scenarios by accounting for spatial variations in error distributions.
2Reliability
If quantile regression is computed to identify error regions, then prediction coverage improves under heteroscedastic uncertainty, but computational overhead increases
Solution Approach 1:
Quantile regression is computed in advance during the calibration phase to identify error regions and their characteristics. This preliminary action allows the main conformal prediction to proceed efficiently by simply looking up pre-computed region assignments and quantile values, rather than performing complex computations during prediction time.
Solution Approach 2:
The computationally intensive quantile regression computation is extracted from the online prediction process and performed offline during calibration. This separation allows the online prediction to be fast while still benefiting from the improved coverage provided by region-aware conformal intervals.
3Productivity
If constant-width conformal prediction intervals are used, then the method is computationally efficient, but the intervals do not adapt to varying uncertainty across different input regions
Solution Approach 1:
The conformal prediction intervals are made dynamic by conditioning them on the identified error regions. Each region has its own quantile value that reflects the local uncertainty characteristics, allowing the intervals to adapt to heteroscedastic patterns while maintaining computational efficiency through pre-computation.
Data Source
AI summary
An apparatus for computing a conditional conformal prediction interval for a machine learning point prediction regression model and calibration point predictions forming a distribution of an error around the point prediction regression model in an input space. The apparatus includes a conformal regions circuit configured to compute a quantile regression of the error to compute an approximation of a quantile of the error. The conformal regions circuit is further configured to identify a set of regions in the input space where the distribution within each region in the set of regions is interpretably constant. In one embodiment, the apparatus also includes a conformal prediction circuit configured to compute the conditional conformal prediction interval for the point prediction regression model conditioned on the identified set of regions and the corresponding computed quantile of the error for each region in the set of regions.