Hardware Neural Network Training Circuit
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current hardware implementations of artificial neural networks (ANNs) face inefficiencies in training due to the need for offline software training, which is slow and prevents reaching expected computational efficiency, as they lack full hardware implementation of training algorithms like back propagation.
Innovation Solution
A method for hardware-implemented training of feedforward ANNs that involves generating output signals, measuring cost values, varying synaptic weights, and determining desired weight changes based on comparisons, allowing for iterative updates without external computing systems, using a cost function measurable on the hardware level to guide weight adjustments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If offline software training is used for hardware-implemented ANNs, then training can be performed with conventional computing systems, but computational efficiency is reduced and training speed is slow
Solution Approach 1:
The patent replaces software-based training algorithms with hardware-based training circuits that implement the same training functionality using electronic components. The training unit is implemented as hardware circuitry that can perform weight updates and cost function calculations directly in the neural network processor, eliminating the need for external software training systems and achieving parallel, high-speed training operations.
2Productivity
If intermittent ANN operation is used for training, then external computing systems can update weights, but computational efficiency cannot reach expected orders of magnitude
Solution Approach 1:
The patent enables continuous training operations by integrating the training unit within the neural network processor itself. The training can occur continuously during normal operation or in dedicated training modes without requiring the network to be halted or transferred to external systems. This continuous operation maintains computational efficiency and eliminates idle time associated with intermittent training approaches.
3Productivity
If full hardware implementation of training algorithms is implemented, then computational efficiency increases, but device complexity increases
Solution Approach 1:
The patent merges the training functionality with the inference functionality by integrating the training unit into the same hardware processor that executes neural network operations. This consolidation allows the system to perform both training and inference using a unified hardware architecture, reducing overall system complexity compared to having separate hardware systems for training and inference while maintaining high training efficiency.
Data Source
AI summary
A method for hardware-implemented training of a feedforward artificial neural network is provided. The method comprises: generating a first output signal by processing an input signal with the network, wherein a cost quantity assumes a first cost value; measuring the first cost value; defining a group of at least one synaptic weight of the network for variation; varying each weight of the group by a predefined weight difference; after the variation, generating a second output signal from the input signal to measure a second cost value; comparing the first and second cost values; and determining, based on the comparison, a desired weight change for each weight of the group such that the cost function does not increase if the respective desired weight changes are added to the weights of the group. The desired weight change is based on the weight difference times −1, 0, or +1.