Convolution Exponential Transformation for Invertible Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing generative models for digital signal processing, particularly in image processing, face challenges with invertible functions and exact likelihood computation, especially when applied to convolutions, where only a few linear normalizing flows can be directly applied.
Innovation Solution
The introduction of a convolution exponential transformation using a matrix exponential to construct a linear transformation parametrized by a convolution kernel, allowing for efficient forward and inverse computation with the same time complexity and exact determinant computation in linear time, enabling bijective mapping in artificial neural networks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If linear normalizing flows are applied to convolutions, then exact likelihood computation is enabled, but only a few linear flows can be directly applied to convolutions
Solution Approach 1:
The patent transforms the convolution operation into a matrix exponential form parametrized by a convolution kernel. This parameter transformation enables exact likelihood computation while maintaining versatility for convolutional operations. The matrix exponential construction allows the model to compute determinants in linear time while supporting general convolutional transformations.
2Reliability
If invertible functions are used for density modeling, then bijective mapping is achieved, but forward and inverse computation have different time complexities
Solution Approach 1:
The patent employs asymmetric construction where the forward transformation uses a matrix exponential with convolution kernel, and the inverse transformation leverages the property that the inverse of exp(M) is exp(-M). This asymmetric approach ensures both forward and inverse computations have the same time complexity, achieving computational symmetry for bijective mapping.
3Ease of manufacture
If convolutional transformations are made invertible, then density evaluation becomes tractable, but the transformation complexity increases
Solution Approach 1:
The patent substitutes the mechanical convolution operation with a matrix exponential transformation. This substitution replaces the traditional convolutional layer with an invertible transformation that maintains the same functional behavior while enabling exact determinant computation. The matrix exponential form simplifies the computation of Jacobian determinants to linear time complexity.
Data Source
AI summary
A device for and a computer implemented method of digital signal processing. The method includes providing a first set of data, mapping the first set of data with to a second set of data, and determining an output of the digital signal processing depending on the second set of data. The second set of data is determined depending on a sum of a finite series of terms. At least one term of the series is determined depending on a result of a convolution of the first set of data with a kernel and at least one term of the series is determined depending on the first set of data and independent of the kernel.