OOD Calibration via Geometric Feature Transformation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning models, such as deep neural networks, face challenges when encountering out-of-distribution (OOD) data, leading to unreliable predictions in both safety-critical and non-safety-critical applications, as they are typically trained in well-controlled environments and fail to generalize to unseen data types.
Innovation Solution
The implementation of a geometric transformation within the machine learning model using two parametric instance-dependent scalar functions, which enhances the model's sensitivity and includes a scoring function to detect OOD data by characterizing covariate and concept shifts, allowing for improved identification and interpretation of out-of-distribution conditions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a machine learning model is optimized on training data through empirical risk minimization, then the model performs very well on test data from the same distribution, but the model fails to generalize to out-of-distribution data
Solution Approach 1:
The patent segments the prediction space by introducing a geometric transformation that separates in-distribution and out-of-distribution data regions. The transformation decomposes the feature space into distinct segments, allowing the model to handle different data types differently. This is achieved through the geometric transformation of features using parametric instance-dependent scalar functions, which creates separated decision boundaries for OOD and ID data.
Solution Approach 2:
The patent introduces an additional dimensional aspect to the prediction by incorporating a scoring function that operates in a separate dimension from the standard classification output. This scoring function evaluates out-of-distribution likelihood independently, adding a new dimension to the model's decision-making capability. The geometric transformation also effectively adds dimensional separation by transforming features into a space where OOD and ID data are more distinguishable.
2Measurement precision
If the model is trained in a well-controlled environment with closed-world assumption, then the model achieves high accuracy on training data, but the model produces unreliable predictions when encountering unseen data types
Solution Approach 1:
The patent applies preliminary action by introducing the geometric transformation and scoring function during the training phase, rather than adding them only during deployment. The model is pre-trained with the capability to detect OOD data, and the geometric transformation parameters are learned alongside the main classification task. This preliminary preparation ensures the model has built-in mechanisms to handle distribution shifts before encountering real-world data.
Solution Approach 2:
The geometric transformation acts as an intermediary between the raw features and the final prediction. Instead of directly mapping features to class probabilities, the transformation first processes features through a geometric operation that accounts for distribution characteristics. This intermediary step mediates between the training distribution and test distribution, smoothing the transition and improving reliability on unseen data types.
3Device complexity
If standard machine learning models are used without geometric transformation, then the model structure remains simple, but the model cannot detect out-of-distribution data or characterize covariate and concept shifts
Solution Approach 1:
The patent changes key parameters of the model by introducing parametric instance-dependent scalar functions in the geometric transformation. These parameters are learned during training and adapt to different data instances. The transformation uses parameters that control the geometric operation (such as scaling, rotation, or projection factors), allowing the model to flexibly adjust its behavior based on the input characteristics while maintaining a relatively simple overall structure.
Data Source
AI summary
A method includes providing, using at least one processing device of an electronic device, input data to a machine learning model. The method also includes extracting, using the at least one processing device, features of the input data. The method further includes performing, using the at least one processing device, a geometric transformation of the features, where the geometric transformation is based on first and second parametric instance-dependent scalar functions. In addition, the method includes producing, using the at least one processing device, a predictive probability distribution based on the transformed features.