Neighborhood Mixture Model for Non-Linear Regression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Identifying non-linear dependencies between variables in high-dimensional data is challenging due to tightly coupled features, continuous target variables, and the sensitivity of local geometric structures to the choice of metric in non-linear regression models.
Innovation Solution
A neighborhood mixture model (NMM) is used for simultaneous metric learning and variable selection, which creates a univariate neighborhood probability map, computes pairwise distances, and performs quadratic programming optimization to re-weight data for non-linear regression, allowing for local geometric constraints and feature selection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional metric learning methods are used in high-dimensional non-linear regression, then computational complexity increases and accuracy decreases, but the patent achieves both improved accuracy and reduced complexity through neighborhood mixture models
Solution Approach 1:
The patent segments the high-dimensional feature space by introducing a neighborhood mixture model that decomposes the complex metric learning problem into multiple local neighborhood relationships. Each neighborhood is characterized by a local metric, avoiding the need to compute and optimize a single global metric across all high-dimensional features, thereby reducing computational complexity while maintaining prediction accuracy.
Solution Approach 2:
The patent applies local quality by allowing different regions of the feature space to have different metric properties through the neighborhood mixture model. Each component of the mixture model learns a local metric that is optimized for its specific neighborhood, enabling accurate local geometric structure capture without the computational burden of a global metric in high-dimensional space.
2Measurement precision
If local geometric constraints are enforced in non-linear regression, then measurement precision improves, but the sensitivity to metric choice becomes a problem
Solution Approach 1:
The patent makes the metric dynamic by using a mixture model where the metric choice is not fixed but adapts to each data point based on its neighborhood membership. The model dynamically selects and combines multiple local metrics according to the probability distribution of neighborhoods, allowing the metric to adapt to local geometric structures without being overly sensitive to any single metric choice.
Solution Approach 2:
The patent creates a composite metric structure by combining multiple local metrics into a neighborhood mixture model. Each metric in the mixture serves as a 'component material' that captures different aspects of the local geometry, and their weighted combination creates a robust composite metric that is less sensitive to the limitations of any individual metric while maintaining high measurement precision.
Data Source
AI summary
A computer-implemented method for simultaneous metric learning and variable selection in non-linear regression is presented. The computer-implemented method includes introducing a dataset and a target variable, creating a univariate neighborhood probability map for each reference point of the dataset, and determining a pairwise distance between each reference point and other points within the dataset. The computer-implemented method further includes computing a Hessian matrix of a quadratic programming (QP) problem, performing optimization of the QP problem, re-weighing data derived from the optimization of the QP problem, and performing non-linear regression on the re-weighed data.


