Oscillator-Based Vector Convolution for Low-Power Dot Product Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for computing convolutions between vectors are computationally costly, especially for large data sets, and struggle to provide a closed-form analytic function for the degree-of-match (DoM) that is differentiable, limiting the success of pattern recognition and machine learning algorithms.
Innovation Solution
A method using a cluster of weakly coupled oscillators to compute the dot product of two vectors by calculating the DoM between the vectors and the zero vector, employing piecewise continuous and differentiable functions to estimate the magnitude square of the difference, which is then combined to approximate the dot product.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional computational methods are used to compute convolutions between vectors, then computation accuracy is maintained, but computational cost and power consumption increase significantly
Solution Approach 1:
The patent replaces traditional digital computational systems with a physical oscillator-based system. Weakly coupled oscillators naturally perform convolution operations through their physical dynamics, substituting mechanical/computational processing with physical phenomena. This substitution dramatically reduces computational cost and power consumption while maintaining the ability to compute convolutions between arbitrary vectors.
Solution Approach 2:
The oscillator system performs convolution operations autonomously through its natural physical dynamics. The weakly coupled oscillators self-organize to compute the convolution without requiring external computational intervention for each operation. This self-service capability enables continuous, energy-efficient processing of convolution operations.
2Productivity
If oscillator clusters are used to compute degree-of-match, then computational cost is reduced, but the inability to provide a closed-form differentiable function limits machine learning algorithm success
Solution Approach 1:
The patent pre-characterizes the relationship between oscillator cluster outputs and Lp norms by collecting training data and fitting closed-form differentiable functions offline. This preliminary action creates lookup tables or function approximations that can be used during runtime, enabling gradient descent training compatibility without sacrificing the computational efficiency of the oscillator system.
Solution Approach 2:
The patent introduces closed-form differentiable functions as intermediaries between the oscillator cluster outputs and the machine learning training process. These functions serve as a bridge, translating the oscillator's physical output into a form compatible with gradient descent algorithms, thereby enabling the use of standard machine learning training techniques with the oscillator-based computational system.
3Measurement precision
If more oscillators are used to increase processing capability, then computation accuracy improves, but device complexity and area increase
Solution Approach 1:
The patent changes the coupling parameters and initial conditions of the oscillators to optimize the convolution computation. By carefully selecting coupling strengths, oscillator frequencies, and initial phases, the system achieves accurate convolution results with fewer oscillators. This parameter optimization reduces device complexity while maintaining computation accuracy.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach significantly reduces computational cost and power consumption, achieving faster processing speeds and improved performance in deep learning networks, with the potential for large improvements in size, weight, area, and power (SWAP) efficiency.
Implementation Method 1
The degree-of-match is computed from the difference between the two vectors, and is based on the dynamics of spontaneous synchronization among the coupled oscillators. The concept is that if the vectors have similar values such that the match is high and the differences are small, then the oscillators synchronize in frequency and phase relatively quicker.
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
A method to perform convolutions between arbitrary vectors includes estimating a first degree of match for a difference between a first vector having a plurality of first elements and a second vector having a plurality of second elements using a first cluster of coupled oscillators, estimating a second degree of match for the first vector using a second cluster of coupled oscillators, estimating a third degree of match for the second vector using a third cluster of coupled oscillators, deriving a first squared L2 norm from the first degree of match, deriving a second squared L2 norm from the second degree of match, deriving a third squared L2 norm from the third degree of match, adding the second squared L2 norm and the third squared L2 norm, and subtracting the first squared L2 norm to form a sum, and dividing the sum by two.