Run-Length Encoded Addressing for Sparse Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neuromorphic computers face inefficiencies in storing connectivity information due to large numbers of neurons and sparse connections, leading to inefficient storage and dynamic growth of connectivity tables.
Innovation Solution
A space-efficient addressing scheme is implemented by representing node connections as run-length encoded differences between full network addresses, exploiting sparsity and locality, which reduces the number of bits required to store address information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If full 64-bit network addresses are stored for each connection in a lookup table, then address precision and uniqueness are maintained, but memory usage becomes excessively large for very large sparse networks
Solution Approach 1:
The patent extracts and stores only the relevant portion of the address information - specifically, the difference between the source node address and destination node address, rather than storing complete 64-bit addresses. This extraction of essential information reduces storage requirements while maintaining address precision through the run-length encoded difference representation.
Solution Approach 2:
The patent changes the parameter representation from full 64-bit addresses to run-length encoded differences, which significantly reduces the number of bits required to represent address information. This parameter transformation exploits the sparsity and locality of connections to achieve compression from 64 bits to approximately 25 bits per address.
2Adaptability or versatility
If connectivity tables are designed to accommodate maximum potential connections, then all possible connections can be stored, but storage efficiency deteriorates due to the sparse nature of actual connections
Solution Approach 1:
The patent implements a dynamic addressing scheme where the storage representation adapts to the actual connectivity patterns of the network. The run-length encoded difference format allows the system to efficiently represent varying degrees of sparsity and locality, making the storage structure dynamically efficient rather than statically oversized.
Solution Approach 2:
The patent uses partial action by storing only the necessary difference information rather than complete addresses. This partial representation is sufficient to maintain address precision and enable lookup operations while avoiding the excessive storage requirements of storing full addresses for all potential connections.
3Adaptability or versatility
If the network learns and forms new synapse connections dynamically, then network adaptability improves, but storage representation complexity increases due to dynamic growth of connectivity tables
Solution Approach 1:
The run-length encoded difference representation provides a self-adapting storage mechanism that automatically adjusts to new connections formed during network learning. As new synapses are created, the same compression scheme applies, maintaining efficiency without requiring manual reconfiguration or complex management structures.
Data Source
AI summary
A connectivity look up structure is maintained for a network that comprises a plurality of nodes, each node is connectable to one or more other nodes, and nodes that are connected tend to be local to one another in the network, and the number of node connections in the network tends to be sparse in relation to the number of potential node connections in the network. The connectivity look up structure stores, for a given node, an address of each other node that is connected to the given node, wherein the stored address for the other node is represented as a run-length encoded difference between a full network address of the given node and a full network address of the other node.