Neural Network Connectivity via Event-Driven LFSR Computation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network systems face challenges in scaling connectivity due to high memory requirements, power consumption, and latency issues, particularly when modeling large biological neural networks, as traditional methods store connectivity and parameters in memory, which becomes inefficient and unscalable.
Innovation Solution
The system calculates neural connectivity and parameters on-demand using a Linear Feedback Shift Register (LFSR) and caching mechanisms, allowing for efficient re-computation of connectivity and other parameters in an event-driven manner, reducing the need for extensive memory storage and improving computational efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If connectivity and parameters are stored in memory for large-scale neural networks, then neural network connectivity can be maintained, but memory requirements and power consumption increase drastically
Solution Approach 1:
The system pre-calculates and stores only essential seed values in a small lookup table before operation. During runtime, all connectivity information is generated on-demand from these seeds using LFSR algorithms, eliminating the need to store complete connectivity matrices while maintaining full network topology capability
Solution Approach 2:
The invention extracts only the minimal necessary information (seed values) from the complete connectivity data and stores it in a compact lookup table. The remaining connectivity information is derived computationally during operation, separating essential stored data from derivable data to reduce memory footprint and power consumption
2Quantity of substance
If connectivity and parameters are stored in memory for large-scale neural networks, then neural network connectivity can be maintained, but access latency increases
Solution Approach 1:
Essential seed values are pre-computed and stored in a small lookup table before operation begins. During runtime, connectivity information is generated instantaneously from these seeds using simple LFSR bit-manipulation operations, eliminating the need for slow memory accesses to large connectivity matrices
Solution Approach 2:
The invention replaces memory access operations with computational operations. Instead of retrieving connectivity data from slow memory storage, the system uses fast in-memory LFSR algorithms to generate connectivity information on-demand, substituting mechanical memory access with electronic computation
3Adaptability or versatility
If crossbar structure is used for neural network connectivity, then all-to-all connectivity between neurons is achieved, but scalability to biological connectivity levels is limited
Solution Approach 1:
The system uses universal LFSR algorithms that can generate any connectivity pattern (sparse, dense, random, structured) from seed values stored in a lookup table. The same computational mechanism handles all connectivity types, providing universal adaptability without requiring different hardware structures for different connectivity regimes
Solution Approach 2:
The invention transitions from spatial hardware connectivity (physical crossbar wires) to computational connectivity (algorithmic generation). Instead of scaling physical interconnect dimensions, the system scales computational capability, using software-based LFSR generation to achieve biological-scale connectivity that cannot be realized with physical crossbars
4Productivity
If hierarchical connectivity approach is used to scale neural networks, then connectivity to localized groups is improved, but connectivity and bandwidth to non-local nodes is reduced
Solution Approach 1:
The system dynamically generates connectivity patterns based on runtime requirements rather than using fixed hierarchical structures. The LFSR-based generation can adapt connectivity patterns on-the-fly, providing both local and global connectivity as needed without being constrained by predetermined hierarchical limitations
Data Source
AI summary
Systems and methods achieving scalable and efficient connectivity in neural algorithms by re-calculating network connectivity in an event-driven way are disclosed. The disclosed solution eliminates the storing of a massive amount of data relating to connectivity used in traditional methods. In one embodiment, a deterministic LFSR is used to quickly, efficiently, and cheaply re-calculate these connections on the fly. An alternative embodiment caches some or all of the LFSR seed values in memory to avoid sequencing the LFSR through all states needed to compute targets for a particular active neuron. Additionally, connections may be calculated in a way that generates neural networks with connections that are uniformly or normally (Gaussian) distributed.