Distributed Memory Bit Clusters for Low-Latency Neural Network Weights
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional techniques for network memory operations in artificial neural networks result in significant power consumption and increased latency due to off-chip memory access for weight and bias values, requiring high standby voltage and inefficient data loading.
Innovation Solution
Utilizing non-volatile distributed memory with configuration bit clusters, where only subsets of configuration bit clusters are activated and mapped to specific network layers, allowing instantaneous on-chip access and reducing power consumption by eliminating standby voltage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If off-chip memory is used to store weight values and bias values, then storage capacity is sufficient, but power consumption increases and latency increases
Solution Approach 1:
The memory system is segmented into multiple banks, each bank containing configuration bit clusters that can be independently activated. This segmentation allows only the necessary memory banks to be powered on during operation, reducing overall power consumption while maintaining sufficient storage capacity through the combined banks.
Solution Approach 2:
The memory architecture transitions from a static fully-powered memory system to a dynamic system where memory banks and configuration bit clusters can be selectively activated and deactivated based on operational needs, enabling power consumption to adapt to actual usage patterns.
2Quantity of substance
If off-chip memory is used to store weight values and bias values, then storage capacity is sufficient, but latency increases
Solution Approach 1:
The memory architecture implements a nested structure with configuration bit clusters at the finest granularity level, contained within memory banks, which are in turn contained within the overall non-volatile distributed memory system. This nested organization enables hierarchical access control where smaller, faster configuration bit clusters provide quick access to frequently needed parameters while larger banks provide sufficient total storage capacity.
Solution Approach 2:
Configuration bit clusters are pre-organized and mapped to specific network layers before operation. This preliminary organization allows the system to directly access the required configuration data without sequential searching, significantly reducing access latency while maintaining the storage capacity of the full memory system.
3Ease of operation
If stand-by voltage is applied to memory, then memory remains ready for access, but power consumption increases
Solution Approach 1:
Instead of continuous stand-by voltage, the system employs periodic activation of configuration bit clusters based on operational demands. Memory banks are activated only when their associated network layers require access, eliminating continuous power consumption while ensuring readiness when needed through on-demand activation.
Solution Approach 2:
The memory system eliminates the need for external stand-by voltage management by using non-volatile memory technology that inherently retains data without power. Configuration bit clusters are activated only when service is required, and the system self-manages power states without continuous external control voltage.
4Speed
If configuration bit clusters are activated for network layer access, then access speed improves, but power consumption increases
Solution Approach 1:
Different portions of the memory system (configuration bit clusters and banks) are activated with different power states based on local operational requirements. Only the specific configuration bit clusters needed for the current network layer operation are activated at full power, while other clusters remain in low-power states, providing fast access where needed without powering the entire system.
Data Source
AI summary
Systems and techniques include identifying a network layer for performing a memory operation, identifying a subset of a plurality of configuration bit clusters of a non-volatile distributed memory that are mapped to the identified network layer using a cluster mapping, in response to identifying the subset of the plurality of configuration bit clusters, activating the subset of the plurality of configuration bit clusters, loading network component data from the subset of the plurality of configuration bit clusters into a local buffer, and applying the network component data to the network layer for performing the memory operation.