Continuous-Output Distance Metric Learning for KNN Regression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The known Large Margin Nearest Neighbour (LMNN) model is limited to supervised learning models that predict discrete output values and is not applicable for regression problems with continuous output values, limiting the accuracy of non-parametric models like the KNN algorithm.
Innovation Solution
Adapt the LMNN model to learn a distance metric for input spaces with continuous output values by determining a transform that correlates the distance between input data points with the difference in their associated continuous outputs, applying this transform to the input space, and learning a distance metric from the transformed space.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the known LMNN model is used for supervised learning, then discrete output predictions are improved, but it cannot be applied to continuous output regression problems
Solution Approach 1:
The patent adapts the LMNN model by changing the output parameter type from discrete to continuous. This is achieved by modifying the distance metric learning objective to work with continuous output values instead of discrete class labels, allowing the model to handle regression problems while maintaining its metric learning capabilities
Solution Approach 2:
The patent creates a universal distance metric learning framework that can handle both discrete classification and continuous regression problems. By formulating the learning objective in a way that accommodates both output types, the model gains multi-functionality and can be applied across different supervised learning scenarios
2Ease of operation
If standard distance metrics are used in KNN algorithm, then computational simplicity is maintained, but prediction accuracy for continuous outputs deteriorates
Solution Approach 1:
The patent applies preliminary action by learning an optimal distance metric from the training data before performing KNN predictions. This pre-processing step transforms the input space to optimize the distance metric, enabling more accurate predictions while keeping the actual KNN computation simple and efficient
Solution Approach 2:
The patent replaces the standard mechanical distance calculation (Euclidean distance) with a learned distance metric that adapts to the specific data distribution. This substitution maintains computational simplicity while significantly improving prediction accuracy by using a data-driven distance measure instead of a fixed mathematical formula
Data Source
AI summary
A computer-implemented method for learning a distance metric, D, for an input space according to a large margin nearest neighbour model adapted for outputs having a continuous value. The method comprising determining a transform, L, for the input space, wherein the transform, L, is configured to correlate the distance between input data points in the input space with the difference between the values of continuous outputs associated with the input data points; applying the transform, L, to the input space; and learning the distance metric, D, from the transformed input space.