Resistive Processing Unit for Neural Network Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing artificial neural network (ANN) architectures face challenges in balancing power consumption, training speed, and efficiency, particularly due to the resource-intensive nature of offline learning techniques used in crosspoint devices, which hinder the optimization of training speed and efficiency.
Innovation Solution
The implementation of a two-terminal resistive processing unit (RPU) with a non-linear active region that performs both data storage and processing operations, enabling local data storage and processing within the RPU itself, thereby accelerating online neural network training and other methodologies like matrix inversion and decomposition.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If offline learning techniques are used in crosspoint devices to limit power consumption, then power consumption is reduced, but training speed and training efficiency deteriorate
Solution Approach 1:
The patent merges data storage and data processing functions into a single two-terminal resistive processing unit. The active region serves both as storage (maintaining conduction states representing weights) and as processing element (performing multiply-accumulate operations through voltage application and current measurement), eliminating the need for separate storage and processing components while enabling online learning capabilities
Solution Approach 2:
The resistive processing unit is designed as a universal component that can simultaneously perform multiple functions: storing weight values in its conduction state, performing multiplication operations through voltage-controlled conductance changes, executing accumulation operations through parallel current summation, and supporting both online and offline learning modes. This multi-functionality resolves the contradiction by allowing the same device to operate in high-speed online learning mode when needed while maintaining low-power offline learning capability
2Use of energy by moving object
If offline learning techniques are used in crosspoint devices to limit power consumption, then power consumption is reduced, but training efficiency deteriorates
Solution Approach 1:
The patent merges data storage and data processing functions into a single two-terminal resistive processing unit. The active region serves both as storage (maintaining conduction states representing weights) and as processing element (performing multiply-accumulate operations through voltage application and current measurement), eliminating the need for separate storage and processing components while enabling online learning capabilities
Solution Approach 2:
The resistive processing unit performs processing operations locally within itself without requiring external processing elements. The active region directly modulates its own conduction state based on applied voltages and automatically performs the multiply-accumulate operations through its inherent electrical characteristics, making the device self-sufficient and eliminating the need for resource-intensive external training infrastructure
3Use of energy by moving object
If simple crosspoint devices are designed to keep power consumption within acceptable range, then power consumption is controlled, but device complexity increases to enable both storage and processing
Solution Approach 1:
The patent merges data storage and data processing functions into a single two-terminal resistive processing unit. The active region serves both as storage (maintaining conduction states representing weights) and as processing element (performing multiply-accumulate operations through voltage application and current measurement), eliminating the need for separate storage and processing components while enabling online learning capabilities
Solution Approach 2:
The resistive processing unit is designed as a universal component that can simultaneously perform multiple functions: storing weight values in its conduction state, performing multiplication operations through voltage-controlled conductance changes, executing accumulation operations through parallel current summation, and supporting both online and offline learning modes. This multi-functionality resolves the contradiction by allowing the same device to operate in high-speed online learning mode when needed while maintaining low-power offline learning capability
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach reduces power consumption while enhancing the speed and efficiency of ANN training, allowing for broader applications and improved overall performance by eliminating the need for external processing and storage elements.
Implementation Method 1
The active region has a conduction state that identifies a weight of a training methodology applied to the RPU
Implementation Method 2
The active region is configured to effect a non-linear change in a conduction state of the active region based on at least one first encoded signal applied to the first terminal and at least one second encoded signal applied to the second terminal
Data Source
AI summary
Embodiments are directed to a two-terminal resistive processing unit (RPU) having a first terminal, a second terminal and an active region. The active region effects a non-linear change in a conduction state of the active region based on at least one first encoded signal applied to the first terminal and at least one second encoded signal applied to the second terminal. The active region is configured to locally perform a data storage operation of a training methodology based at least in part on the non-linear change in the conduction state. The active region is further configured to locally perform a data processing operation of the training methodology based at least in part on the non-linear change in the conduction state.