Input Convex Neural Network for Bregman Divergence Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for learning Bregman divergence are inefficient due to high run-time costs and representational capacity, as they rely on direct representations and max affine approximations, which restrict the ability to capture asymmetric distance measures and triangle inequality violations in tasks like visual similarity determination.
Innovation Solution
The method employs an Input Convex Neural Network (ICNN) to learn a convex generating function φ directly, using double backpropagation to compute empirical Bregman divergence efficiently and ensure the function's convexity, thereby overcoming the limitations of traditional approaches.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If direct representation of Bregman divergence is used, then measurement precision is improved, but device complexity and run-time cost increase significantly
Solution Approach 1:
The patent replaces the direct computational representation of Bregman divergence with a neural network-based generative function φ. Instead of directly computing divergence using complex formulas, the system uses a neural network to generate the function φ, which then computes divergence through differentiable operations. This substitution reduces representational complexity while maintaining measurement precision through the learned function φ.
2Ease of operation
If fixed distance measure (e.g., Euclidean distance) is used, then ease of operation is improved, but adaptability to asymmetric tasks deteriorates
Solution Approach 1:
The patent transforms the static fixed distance measure into a dynamic learned distance metric. Instead of using a predetermined Euclidean distance, the system employs a neural network that learns the generating function φ dynamically from data. This learned function can adapt to capture asymmetric relationships in the data, providing both computational simplicity through the learned metric and adaptability to various task-specific distance patterns.
Solution Approach 2:
The patent changes the parameters of the distance measure from fixed values to learned parameters through the neural network. The generating function φ has parameters that are optimized during training to capture the specific asymmetric relationships in the data. This allows the distance measure to adapt its parameters based on the task requirements while maintaining ease of computation through the learned function.
3Productivity
If embedding network projects inputs to optimize fixed distance measure, then productivity is improved, but ability to handle tasks where projection is not possible deteriorates
Solution Approach 1:
The patent creates a universal framework that can handle both symmetric and asymmetric distance measures through the same neural network-based approach. The generating function φ learned by the neural network can be applied to various tasks including those where traditional projection methods fail, such as asymmetric similarity determination and tasks violating triangle inequality. This universal approach maintains productivity while expanding adaptability across different task types.
Data Source
AI summary
A method, system, and computer program product for configuring a computer for data similarity determination using Bregman divergence may include storing a data set having plural data pairs with one or more data points corresponding to one or more features and generating a trained input convex neural network (ICNN) using the data set, the ICNN having one or more parameters. Training the ICNN may include extracting one or more features for each piece of data in the first data pair, generating an empirical Bregman divergence for the first data pair, and computing one or more gradients between the one or more features within the first data pair using known target distances and the computed empirical Bregman divergence.