Neighborhood Mixture Model for Non-Linear Regression

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Identifying non-linear dependencies between variables in high-dimensional data is challenging due to tightly coupled features, continuous target variables, and the sensitivity of local geometric structures to the choice of metric in non-linear regression models.

Innovation Solution

A neighborhood mixture model (NMM) is used for simultaneous metric learning and variable selection, which creates a univariate neighborhood probability map, computes pairwise distances, and performs quadratic programming optimization to re-weight data for non-linear regression, allowing for local geometric constraints and feature selection.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If traditional metric learning methods are used in high-dimensional non-linear regression, then computational complexity increases and accuracy decreases, but the patent achieves both improved accuracy and reduced complexity through neighborhood mixture models

Engineering Contradiction:
Improveaccuracy of non-linear predictorsVSAvoidcomputational complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent segments the high-dimensional feature space by introducing a neighborhood mixture model that decomposes the complex metric learning problem into multiple local neighborhood relationships. Each neighborhood is characterized by a local metric, avoiding the need to compute and optimize a single global metric across all high-dimensional features, thereby reducing computational complexity while maintaining prediction accuracy.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent applies local quality by allowing different regions of the feature space to have different metric properties through the neighborhood mixture model. Each component of the mixture model learns a local metric that is optimized for its specific neighborhood, enabling accurate local geometric structure capture without the computational burden of a global metric in high-dimensional space.

Inventive Principle:
Principle #3Local quality

2Measurement precision

If local geometric constraints are enforced in non-linear regression, then measurement precision improves, but the sensitivity to metric choice becomes a problem

Engineering Contradiction:
Improveaccuracy of non-linear predictorsVSAvoidsensitivity to metric choice
Core Design Contradiction:
Measurement precisionVSAdaptability or versatility

Solution Approach 1:

The patent makes the metric dynamic by using a mixture model where the metric choice is not fixed but adapts to each data point based on its neighborhood membership. The model dynamically selects and combines multiple local metrics according to the probability distribution of neighborhoods, allowing the metric to adapt to local geometric structures without being overly sensitive to any single metric choice.

Inventive Principle:
Principle #15Dynamics

Solution Approach 2:

The patent creates a composite metric structure by combining multiple local metrics into a neighborhood mixture model. Each metric in the mixture serves as a 'component material' that captures different aspects of the local geometry, and their weighted combination creates a robust composite metric that is less sensitive to the limitations of any individual metric while maintaining high measurement precision.

Inventive Principle:
Principle #40Composite materials

Data Source

PatentUS11281990B2Mining non-linear dependencies via a neighborhood mixture model
Publication Date: 2022.03.22 NEC CORP
  • US11281990B2 patent drawing
  • US11281990B2 patent drawing
  • US11281990B2 patent drawing

AI summary

A computer-implemented method for simultaneous metric learning and variable selection in non-linear regression is presented. The computer-implemented method includes introducing a dataset and a target variable, creating a univariate neighborhood probability map for each reference point of the dataset, and determining a pairwise distance between each reference point and other points within the dataset. The computer-implemented method further includes computing a Hessian matrix of a quadratic programming (QP) problem, performing optimization of the QP problem, re-weighing data derived from the optimization of the QP problem, and performing non-linear regression on the re-weighed data.