Multi-element Synapse for Neural Network Weight Updating
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network learning technologies face limitations in energy efficiency and computational complexity due to the von Neumann architecture, and resistive memory systems require excessive time and resources for weight updating, especially when representing multiple weight bits and applying parallel updating methods.
Innovation Solution
A neural network learning apparatus and method using multiple-element synapses, where a first synaptic unit with higher conductance values is used for early training and a second synaptic unit with higher precision is used for later training, allowing selective updating based on learning progress and accuracy evaluation to accelerate neuromorphic hardware learning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple cells are utilized for a synapse to represent more weight bits through resistive memory, then the precision of weight representation is improved, but the device complexity and operational overhead increase
Solution Approach 1:
The synaptic unit is segmented into multiple synapse arrays (first synapse array, second synapse array, etc.), where each array contains multiple synapses. This segmentation allows the system to represent more weight bits by distributing them across multiple simpler synapses rather than using a single complex high-precision synapse, thereby reducing operational overhead while maintaining precision.
Solution Approach 2:
The patent implements a hierarchical nested structure where synapses are nested within synapse arrays, which are nested within synaptic units. Multiple synaptic units can be further nested or organized in banks. This nesting enables efficient parallel updating at different hierarchical levels, improving both precision and reducing complexity through structured organization.
2Productivity
If parallel updating method is applied to resistive memory array, then the learning speed is improved, but the time and resources required for weight updating increase due to device selection and calculation overhead
Solution Approach 1:
The patent pre-organizes the synaptic unit into multiple synapse arrays with synapses arranged in specific patterns (e.g., column-wise or row-wise distribution of weight bits). This preliminary structuring enables direct parallel updating operations without requiring complex device selection and calculation during the learning process, thus maintaining high learning speed while reducing updating overhead.
Solution Approach 2:
The patent enables dynamic selection and updating of specific synapse arrays based on learning requirements. Different synapse arrays can be selectively updated in parallel depending on which weight bits need modification, allowing the system to adaptively optimize the updating process and avoid unnecessary operations on already-correct weights.
3Use of energy by moving object
If resistive memory is used to store conductance values for weights, then the energy efficiency is improved, but the limited number of conductance states restricts the precision of weight representation
Solution Approach 1:
Weight bits are segmented and distributed across multiple synapses within synapse arrays. Each synapse stores a portion of the weight bits using its conductance state. By combining the information from multiple synapses, the system achieves high precision weight representation while each individual synapse only requires limited conductance states, maintaining energy efficiency.
Solution Approach 2:
Different synapses within a synapse array can have different conductance values corresponding to different weight bit values. The patent enables selective updating of specific synapses based on learning requirements, allowing each local synapse to optimize its conductance state independently while contributing to the overall high-precision weight representation of the synaptic unit.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enables high-accuracy neural network learning by selectively using resistive elements with different precision levels, reducing the time and resources required for weight updating and improving training efficiency, applicable to various AI systems like autonomous driving and image processing.
Implementation Method 1
A first synaptic unit including a plurality of first resistive elements to update a weight of a neural network based on a first precision; and a second synaptic unit including a plurality of second resistive elements to update the weight of the neural network with a precision higher than the first precision
Data Source
AI summary
Disclosed are an apparatus and a method for neural network learning using a synapse based on multiple elements. A neural network learning apparatus using a synapse based on multiple elements according to an exemplary embodiment of the present disclosure includes a first synaptic unit including a plurality of first resistive elements to update a weight of a neural network based on a first precision and a second synaptic unit including a plurality of second resistive elements to update the weight of the neural network with a precision higher than the first precision.