Neural Network Memory Management for Sparse Connectivity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Artificial neural networks with sparse connectivity face inefficiencies in memory management due to the storage of many zero values in synaptic weight memory, leading to increased memory footprint and power consumption.
Innovation Solution
A compute device with a memory management system that separates synaptic weights into connection, pointer, and weight memories, storing only nonzero weights in the weight memory, reducing memory footprint by using connection memory to identify nonzero connections and pointer memory to access corresponding weights.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If zero values are stored in synaptic weight memory for sparse connectivity neural networks, then complete weight information is preserved, but memory footprint increases
Solution Approach 1:
The patent extracts only the nonzero weights from the complete weight matrix and stores them separately in a compact weight storage structure. The connectivity information (including zero-value positions) is stored in a connection storage structure. This separation allows the system to preserve complete weight information while significantly reducing the memory footprint by excluding redundant zero values from the weight storage.
Solution Approach 2:
The patent segments the weight storage into two distinct components: a connection storage structure that holds connectivity information (including zero-value indicators) and a compact weight storage structure that holds only nonzero weights. This segmentation allows independent optimization of each component, achieving reduced memory usage while maintaining complete information through their coordinated use.
2Use of energy by stationary object
If all connection weights are stored separately in synaptic weight memory, then weight access is straightforward, but power consumption increases due to storing many zero values
Solution Approach 1:
The patent extracts only the essential nonzero weight information from the complete weight set and stores it in a compact structure. By removing redundant zero values, the memory footprint and associated power consumption are significantly reduced. The connection storage structure tracks which weights are nonzero, enabling efficient access without storing unnecessary data.
Solution Approach 2:
The patent introduces an intermediary connection storage structure that acts as an index or map between neuron pairs and their corresponding nonzero weights. This intermediary enables efficient weight access by first checking connectivity information and then retrieving the actual weight values, reducing both memory usage and power consumption while managing the increased complexity through structured organization.
3Productivity
If sparse connectivity is represented with full weight storage, then memory access is simple, but memory efficiency decreases
Solution Approach 1:
The patent segments weight storage into connection information and compact weight values, improving memory efficiency by eliminating redundant zero storage. The segmented structure organizes data by connectivity patterns, allowing efficient retrieval by first checking connection validity and then accessing the corresponding compact weight, maintaining operational simplicity through this two-stage access pattern.
Solution Approach 2:
The patent performs preliminary organization of weight data by pre-computing and storing connectivity information that identifies nonzero weight positions. This preliminary action enables subsequent weight access operations to skip over zero values entirely, improving memory efficiency while maintaining simple access through pre-organized lookup structures that guide the retrieval process.
Data Source
AI summary
Technologies for memory management of a neural network include a compute device to read a memory of the compute device to access connectivity data associated with a neuron of the neural network, determine a memory address at which weights corresponding with the one or more network connections are stored, and access the corresponding weights from a memory location corresponding with the memory address. The connectivity data is indicative of one or more network connections from the neuron.