Von-Neumann Neural Network Memory Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing von-Neumann neural network implementations face performance and power consumption overheads due to frequent remote memory and IO communications, and non-von-Neumann implementations struggle to build a flexible ecosystem for various learning algorithms.
Innovation Solution
A neural network processing system utilizing von-Neumann cores with a shared memory architecture that includes event address memory, weight data storage areas, and accumulation memory, allowing for noncoherent remote memory writes and efficient event-driven communication between nodes, thereby reducing memory communication traffic and supporting flexible learning algorithms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If von-Neumann implementations use frequent remote memory and IO communications to exchange event messages, then neural network processing can be performed, but performance overhead and power consumption increase significantly
Solution Approach 1:
The system segments the memory system into multiple types (weight data storage areas, accumulation memory, event address memory) organized in a tree structure. This segmentation allows different types of data to be stored in optimized locations, reducing the need for frequent remote memory communications and lowering power consumption while maintaining processing capability.
Solution Approach 2:
The patent introduces a hierarchical memory architecture that adds a dimensional organization layer beyond traditional flat memory structures. By organizing memories in a tree structure with multiple levels and types, the system reduces communication overhead through localized access patterns, thereby reducing power consumption while maintaining neural network processing performance.
2Productivity
If von-Neumann implementations use frequent remote memory and IO communications to exchange event messages, then neural network processing can be performed, but performance overhead increases significantly
Solution Approach 1:
The memory system is segmented into specialized areas (weight data storage, accumulation memory, event address memory) that can be accessed independently. This segmentation enables parallel processing operations and reduces the need for sequential remote memory communications, thereby reducing performance overhead while maintaining neural network processing capability.
Solution Approach 2:
The event address memory acts as an intermediary structure that facilitates efficient communication between different parts of the system. By using this intermediate memory layer, the system reduces the need for direct frequent remote memory and IO communications, lowering performance overhead while maintaining processing capability.
3Loss of energy
If non-von-Neumann implementations are used to reduce memory communication overhead, then power consumption and performance overhead are reduced, but difficulty in building a flexible ecosystem for various learning algorithms increases
Solution Approach 1:
The von-Neumann core is designed with universal functionality to execute various learning algorithms while maintaining efficient memory access patterns. The core can perform both traditional von-Neumann operations and event-driven neural network processing, providing adaptability for different algorithms while reducing power consumption through optimized memory access.
Solution Approach 2:
The system changes the operational parameters of the von-Neumann architecture by implementing event-driven processing modes and specialized memory access patterns. This allows the same hardware to achieve lower power consumption while maintaining the flexibility to support various learning algorithms through software configuration rather than hardware reconfiguration.
Data Source
AI summary
A neural network processing system includes one source node having a source memory and a source core, and one destination node having a destination memory and a destination core, the source core and the destination core being von-Neumann cores, the destination memory including weight data storage areas for storing weight data corresponding to each node, an accumulation memory for accumulating the weight data, and an event address memory, the destination core identifying the weight data storage area and accumulating the weight data to store the accumulated weight data in the accumulation memory, the source memory including a data set having first information for identifying the destination node and second information for identifying the weight data storage area, and the source core reading the data set and sending the second information in the data set to the destination node to conduct remote memory write.