Neural Network Signal Processing Module With Column-Wise Memory Organization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network hardware architectures face inefficiencies in using silicon for weight and data storage, and struggle with scalability in terms of the number of neurons and synapses, leading to suboptimal performance and energy consumption.
Innovation Solution
A signal processing module with a memory organization that stores data column-wise over multiple addresses, allowing for efficient data transfer and routing, and featuring a routing unit that can route data to multiple computation units, along with a memory virtualization unit for reorganization and independent access, enabling flexible and efficient neuronal computations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is stored in conventional memory organization, then memory capacity is sufficient, but data transfer efficiency to computation units is low
Solution Approach 1:
The patent transforms the conventional row-wise memory organization into a column-wise organization, effectively changing the dimensional access pattern. This allows data to be transferred in parallel columns to multiple computation units simultaneously, dramatically improving data transfer efficiency from sequential to parallel operations.
Solution Approach 2:
The memory is divided into multiple columnar segments that can be accessed and transferred independently to different computation units. This segmentation enables parallel data transfer paths, where each column can be routed to a specific computation unit without interfering with other columns, thus improving overall productivity.
2Adaptability or versatility
If hardware architecture is specialized for specific neural networks, then performance for those networks is high, but adaptability to other networks is reduced
Solution Approach 1:
The patent designs a universal memory and routing architecture that can accommodate different neural network configurations through programmable routing units and flexible data organization. The same hardware structure can be reconfigured via software to implement various neural network types (CNN, RNN, transformer) without requiring dedicated hardware for each network type, thus achieving multi-functionality.
Solution Approach 2:
The routing unit and memory allocation are designed to be dynamically reconfigurable based on the specific neural network being executed. The system can adapt its data flow paths and memory access patterns in real-time according to the computational requirements of different network architectures, maintaining high performance across diverse applications.
3Adaptability or versatility
If the number of neurons and synapses is increased, then neural network capability is improved, but hardware scalability is limited
Solution Approach 1:
The patent employs a hierarchical and modular architecture where computation units can be nested within larger processing blocks, which in turn are nested within the full neural network processor. This nested structure allows for scalable expansion from small to large networks by simply replicating and nesting modular units, making it easier to scale from fewer to many neurons and synapses without proportionally increasing overall system complexity.
4Speed
If memory is organized for high-speed access, then computation speed is improved, but energy consumption increases
Solution Approach 1:
The patent combines multiple data transfer operations into single parallel columnar transfers, reducing the total number of memory access cycles required. By merging data retrieval for multiple computation units into unified column-wise transfers, the system achieves high-speed access while minimizing the cumulative energy cost of repeated individual accesses.
Data Source
AI summary
A signal processing module comprises at least one operational unit incorporating computation units, input and output interfaces able to be linked to a bus and a memory storing data destined for the computation units, the memory being organized so that each data word is stored column-wise over several addresses according to an order dependent on the application, a column having a width of one bit, the words being transferred in series to the computation units.