Neural Network Signal Processing Using Prospective Coding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In neural networks with slow components, the response lag in each layer leads to decreased inference speed and disrupts learning, as timing mismatches between instructive signals and neural activity occur, particularly in deep networks.
Innovation Solution
A novel signal processing framework that enables fast computation and learning by allowing neurons to predict their future state, effectively making local information processing instantaneous, and incorporating prospective coding to alleviate delays and noise in analogue circuits.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If neurons with finite response time are used in hierarchical neural networks, then the network can be implemented in physical substrates, but the inference speed decreases with network depth due to accumulated response lags
Solution Approach 1:
The patent applies preliminary action by computing the future state of neurons ahead of time using prospective coding. Instead of waiting for neurons to naturally respond after receiving inputs, the system calculates what the neuron states will be at future time points and uses these pre-computed values immediately. This eliminates the waiting period inherent in physical neuron response times, allowing deep networks to maintain fast inference speeds while remaining implementable in physical substrates.
2Reliability
If relaxation phases are introduced to handle timing mismatches, then learning can be maintained, but the overall processing time increases significantly
Solution Approach 1:
The prospective coding mechanism performs preliminary computation of future neuron states, allowing the system to bypass relaxation phases entirely. By knowing what neuron states will be at future time points, the system can immediately use these values for learning computations without waiting for the physical relaxation process to complete, thus maintaining learning capability while eliminating the time loss associated with relaxation phases.
3Reliability
If long stimulus presentation times with small learning rates are used, then timing mismatches are reduced, but learning becomes inherently slow
Solution Approach 1:
Prospective coding computes future neuron states in advance, allowing the system to use large learning rates with short stimulus presentation times. The pre-computed future states provide the necessary timing synchronization without requiring prolonged stimulus exposure, thereby maintaining both timing accuracy and fast learning speed simultaneously.
4Reliability
If phased plasticity is implemented to be active only after relaxation periods, then learning can occur, but it is challenging to implement in asynchronous distributed systems
Solution Approach 1:
By pre-computing future neuron states through prospective coding, the system eliminates the need for coordinated relaxation periods before learning can occur. This allows phased plasticity to be implemented continuously without requiring global synchronization or waiting for relaxation completion, significantly reducing implementation complexity in asynchronous distributed systems while maintaining learning functionality.
Data Source
AI summary
The present invention concerns a method of processing signals in a neural network comprising a set of neurons interconnected by a set of synapses, with each neuron comprising a soma and a set of apical and/or basal dendrites. According to an embodiment, the method comprises: updating at least one synaptic input signal configured to be received at input nodes of one or more neurons; updating the corresponding basal potential of at least one of the basal dendrites; updating the corresponding apical potential of at least one of the apical dendrites; updating a potential differential for at least one neuron by using at least the updated apical and basal potentials; updating the somatic potential of at least one of the somas by using at least the corresponding updated potential differentials; updating the prospective potential of at least one neuron by using at least the corresponding updated somatic potentials and potential differentials; and generating a neuronal output signal for at least one neuron by using the corresponding updated prospective potentials. The method may further comprise updating prospective and/or membrane time constants, as well as synaptic weights for a subset of neurons using a subset of the above-mentioned variables.