Neuron Error Value Generation for Feedback Neural Networks

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional artificial neural networks face limitations in constructing networks with complex signal feedback, as back-propagation algorithms restrict networks to feed-forward configurations, making it difficult to mimic the complex feedback mechanisms observed in biological neural networks.

Innovation Solution

A learning method that allows neural networks to produce neuron-level error values without relying on explicit error propagation, enabling the construction of networks with arbitrarily complex signal feedback paths, and allowing for both feed-forward and feedback configurations.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If back-propagation algorithms are used for training neural networks, then weight adjustment can be performed, but the network topology is restricted to feed-forward configurations only

Engineering Contradiction:

Improvenetwork topology flexibilityVSAvoidnetwork configuration complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent applies feedback by enabling post-synaptic neurons to send error values back to pre-synaptic neurons through the same connections used for signal transmission. This feedback mechanism allows the network to learn from output errors and adjust weights in pre-synaptic neurons, thereby supporting complex feedback topologies while maintaining the ability to perform weight adjustment.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The patent makes the network connections universal by allowing them to serve dual purposes: transmitting signals in the forward direction and propagating error values in the backward direction. This multi-functionality eliminates the need for separate feedback pathways and enables the same network structure to support both feed-forward and feedback configurations.

Inventive Principle:

Principle #6Universality (Multi-functionality)

2Reliability

If complex signal feedback paths are introduced to mimic biological neural networks, then biological fidelity improves, but the ability to propagate error values becomes difficult

Engineering Contradiction:

Improvebiological neural network fidelityVSAvoiderror value propagation difficulty

Core Design Contradiction:

ReliabilityVSDifficulty of detecting and measuring

Solution Approach 1:

The patent implements feedback by allowing post-synaptic neurons to propagate error values back to pre-synaptic neurons through the existing signal transmission connections. This enables complex feedback paths to coexist with error propagation capabilities, maintaining biological fidelity while preserving the ability to detect and measure error values through the feedback mechanism.

Inventive Principle:

Principle #23Feedback

3Ease of operation

If feed-forward network configuration is used, then error propagation is simplified, but the network cannot effectively handle complex feedback mechanisms

Engineering Contradiction:

Improveerror propagation simplicityVSAvoidfeedback mechanism capability

Core Design Contradiction:

Ease of operationVSAdaptability or versatility

Solution Approach 1:

The patent extends the feed-forward architecture by adding feedback capability where post-synaptic neurons send error values back to pre-synaptic neurons. This allows the network to maintain the simplicity of feed-forward error propagation while gaining the versatility to handle complex feedback mechanisms through the same weight adjustment process.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The patent makes the network operations universal by enabling connections to function in both feed-forward and feedback directions. This multi-functionality allows the network to handle complex feedback mechanisms without requiring separate operational modes, combining the simplicity of feed-forward processing with the adaptability of feedback handling.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS7814038B1Feedback-tolerant method and device producing weight-adjustment factors for pre-synaptic neurons in artificial neural networks

Publication Date: 2010.10.12 REPICI DOMINIC JOHN

AI summary

In an artificial neural network a method and neuron device that produce weight-adjustment factors, also called error values (116), for pre-synaptic neurons (302a . . . 302c) that are used to adjust the values of connection weights (106 . . . 106n) in neurons (100) used in artificial neural networks (ANNs). The amount of influence a pre-synaptic neuron has had over a post-synaptic neuron is calculated during signal propagation in the post-synaptic neuron (422a . . . 422n) and accumulated for the pre-synaptic neuron (426) for each post-synaptic neuron to which the pre-synaptic neuron's output is connected (428). Influence values calculated for use by pre-synaptic neurons may further be modified by the post-synaptic neuron's output value (102) (option 424), and its error value (116) (option 1110).