Neural Network Training Using Analog Circuit Blocks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network training methods face challenges with precision and energy efficiency, as general-purpose processors consume high power and analog circuit elements wear out quickly due to frequent conductance changes, requiring more iterations to converge on a trained state.
Innovation Solution
A system that switches from using a processor and memory for initial training iterations to an analog circuit element functional block for later iterations, leveraging higher precision and reduced power consumption while minimizing wear on analog components.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If general-purpose processors are used for neural network training, then training precision is improved, but energy consumption increases
Solution Approach 1:
The patent segments the training process into two distinct phases: initial training iterations performed on general-purpose processors (CPU/GPU) and subsequent iterations performed on neuromorphic processors. This segmentation allows each processor type to operate in its optimal performance regime, with the CPU/GPU handling the precision-critical initial phase and the energy-efficient neuromorphic processor handling the later phase, thereby resolving the contradiction between training precision and energy consumption.
Solution Approach 2:
The patent implements a dynamic switching mechanism that transitions the training workload from general-purpose processors to neuromorphic processors based on the training iteration count. This dynamic approach allows the system to adapt its computational resources throughout the training process, maintaining high precision when needed while minimizing energy consumption in later iterations, thus resolving the contradiction between precision and energy efficiency.
2Use of energy by moving object
If analog circuit elements are used for training iterations, then energy efficiency is improved, but component reliability deteriorates
Solution Approach 1:
The patent applies preliminary action by completing the initial training iterations on general-purpose processors before transitioning to neuromorphic processors. This preliminary phase establishes a foundation that reduces the number of subsequent iterations needed, thereby limiting the cumulative wear on analog circuit elements while still achieving energy efficiency in the later phases, effectively resolving the contradiction between energy efficiency and component reliability.
Solution Approach 2:
The patent changes the operational parameters of the training system by switching between different processor types at specific iteration milestones. This parameter change strategy allows the system to leverage the high energy efficiency of analog circuit elements while constraining their usage to a limited number of iterations, thus maintaining component reliability while achieving overall energy efficiency.
3Measurement precision
If more training iterations are performed, then training precision is improved, but training time increases
Solution Approach 1:
The patent employs a hybrid computational approach where general-purpose processors and neuromorphic processors work together in sequence, with each processor type optimized for specific phases of the training process. This copying of the training workload across different computational platforms allows the system to achieve high training precision through multiple iterations while minimizing total training time by leveraging the speed advantages of neuromorphic processing in later iterations.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach reduces the number of iterations needed to reach a trained state, conserves electrical power, and avoids memory access bottlenecks, resulting in improved performance and extended component lifespan.
Implementation Method 1
analog circuit element functional block that performs computational operations for neural network training. Although using the analog circuit element functional block can operate using less electrical power and/or faster than the processor, the analog circuit element functional block may lack the precision of the processor
Data Source
AI summary
A system is described that performs training operations for a neural network, the system including an analog circuit element functional block with an array of analog circuit elements, and a controller. The controller monitors error values computed using an output from each of one or more initial iterations of a neural network training operation, the one or more initial iterations being performed using neural network data acquired from the memory. When one or more error values are less than a threshold, the controller uses the neural network data from the memory to configure the analog circuit element functional block to perform remaining iterations of the neural network training operation. The controller then causes the analog circuit element functional block to perform the remaining iterations.