Oscillatory Recurrent Gated Neural Integrator Circuits for Signal Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Long Short-Term Memory (LSTM) networks are not well-suited for processing oscillatory signals with time-varying amplitudes, frequencies, and phases, and they are difficult to design due to their complex functionality, while also being energy-inefficient compared to analog circuitry.
Innovation Solution
The development of Oscillatory Recurrent Gated Neural Integrator Circuits (ORGANICs), which utilize an analog electrical circuit to compute output responses based on input and recurrent drives, modulatory responses, and readouts, allowing for efficient processing of oscillating signals and offering energy-efficient design.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If LSTMs are used to process oscillatory signals, then long-term dependencies can be represented, but the signals cannot be processed effectively because LSTMs approximate oscillatory signals with piecewise constant functions
Solution Approach 1:
The patent changes the fundamental parameters of the neural network by introducing complex-valued weights and oscillatory basis functions instead of real-valued weights and piecewise constant activations. This allows the network to naturally represent and process oscillatory signals with time-varying amplitudes, frequencies, and phases while maintaining the ability to capture long-term dependencies.
2Reliability
If LSTMs are designed to handle long-term dependencies, then they can represent complex temporal patterns, but they become difficult to design due to complex functionality
Solution Approach 1:
The patent segments the complex LSTM functionality into simpler components by using a fixed oscillatory basis function set combined with learnable complex weights. This decomposition separates the temporal modeling (handled by the oscillatory basis functions) from the parameter learning (handled by the complex weights), making the system easier to design and more interpretable while maintaining long-term dependency representation.
3Productivity
If LSTMs are implemented as digital recurrent neural networks, then they can process sequential data, but they are energy-inefficient compared to analog circuitry
Solution Approach 1:
The patent substitutes the digital computational mechanism with an analog electrical circuit implementation. The complex-valued computations are performed using physical electrical components (resistors, capacitors, operational amplifiers) that naturally handle complex arithmetic through their electrical properties, thereby achieving energy efficiency comparable to analog systems while maintaining the full computational capability of the recurrent neural network.
Data Source
AI summary
A method for computation with recurrent neural networks includes receiving an input drive and a recurrent drive, producing at least one modulatory response; computing at least one output response, each output response including a sum of: (1) the input drive multiplied by a function of at least one of the at least one modulatory response, each input drive including a function of at least one input, and (2) the recurrent drive multiplied by a function of at least one of the at least one modulatory response, each recurrent drive including a function of the at least one output response, each modulatory response including a function of at least one of (i) the at least one input, (ii) the at least one output response, or (iii) at least one first offset, and computing a readout of the at least one output response.


