Dual Neural Network Operation Apparatus for Parallel Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional neuromorphic processors require halting the normal operation process to perform learning, which limits their efficiency in parallel operation and learning processes due to the need for separate forward and backward propagation processes.
Innovation Solution
An operation apparatus comprising a first and second neural network, an evaluation circuit, and a coefficient-updating circuit that alternates between two modes, allowing simultaneous operation and learning by updating coefficients in one network based on the error evaluation of the other, enabling parallel operation and learning processes without halting the system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the conventional neural network halts the normal operation process to perform learning, then the learning process can be performed with an external processor, but the productivity is reduced due to the inability to perform operation and learning in parallel
Solution Approach 1:
The system is divided into two separate neural networks: a first neural network dedicated to normal operation and a second neural network dedicated to learning processes. This segmentation allows both networks to operate independently and simultaneously, eliminating the need to halt operation for learning while maintaining separate functional responsibilities for each network
Solution Approach 2:
A second neural network is created as a copy of the first neural network with the same layer structure. This copy is used specifically for learning processes, allowing the original first neural network to continue its normal operation without interruption while the duplicate handles learning computations
2Productivity
If the neural network performs operation and learning in parallel, then the productivity is improved, but the device complexity increases due to the need for two separate neural networks and mode switching mechanisms
Solution Approach 1:
Both the first and second neural networks share the same layer structure and can perform both operation and learning functions, though they specialize in different primary roles. This universality reduces the need for entirely separate architectures and allows flexible mode switching between operation and learning tasks
Solution Approach 2:
The system implements dynamic mode switching between the first and second neural networks, where the controller can alternately switch between operation mode (using first network) and learning mode (using second network), allowing the system to adapt its configuration based on current operational requirements
Data Source
AI summary
According to an embodiment, an operation apparatus includes a first neural network, a second neural network, an evaluation circuit, and a coefficient-updating circuit. The first neural network performs an operation in a first mode. The second neural network performs an operation in a second mode and has a same layer structure as the first neural network. The evaluation circuit evaluates an error of the operation of the first neural network in the first mode and evaluates an error of the operation of the second neural network in the second mode. The coefficient-updating circuit updates, in the first mode, coefficients set for the second neural network based on an evaluating result of the error of the operation of the first neural network, and updates, in the second mode, coefficients set for the first neural network based on an evaluating result of the error of the operation of the second neural network.