Neural Network Event Bundling for Lower Memory Bandwidth Load
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network processor systems face inefficiencies due to the high energy consumption and memory bandwidth usage associated with neuron state reads and writes for every activation event, particularly in the transmission of coordinate values and identifiers for target neural network layers.
Innovation Solution
A processor system with a message exchange network that bundles activation event data with common coordinate values, reducing the amount of data transmitted and minimizing memory access by packing multiple events into a single message, thereby optimizing the message exchange network load and memory access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If every activation event is processed individually with separate coordinate transmission, then processing accuracy is maintained, but message exchange network load and memory bandwidth consumption increase significantly
Solution Approach 1:
The patent combines multiple activation events into a single bundled message when they share common coordinate values. Instead of transmitting each event separately, the system groups events with identical coordinates and transmits them once, significantly reducing the number of messages on the network while preserving processing accuracy through proper event enumeration in the bundled structure.
Solution Approach 2:
The bundled message structure serves multiple functions simultaneously: it transmits coordinate information once for multiple events, maintains event identification through enumeration, and enables efficient processing by the receiving core. This multi-functional approach resolves the contradiction by making the message structure more versatile rather than simply reducing traffic.
2Reliability
If neuron state is read and written to memory for every activation event, then complete state updates are ensured, but energy consumption and memory bandwidth usage increase
Solution Approach 1:
The system performs preliminary bundling of activation events before memory access. By grouping multiple events that will update the same neuron state into a single processed unit, the receiving processor core can perform memory reads and writes fewer times, maintaining state update completeness while reducing the frequency of memory operations and associated energy consumption.
Solution Approach 2:
The patent changes the parameter of event processing from individual events to bundled events. This parameter change transforms the memory access pattern from multiple separate reads/writes to fewer consolidated accesses, directly reducing energy consumption while the bundled structure ensures all necessary state updates are still performed.
3Measurement precision
If coordinate values and layer identifiers are transmitted for every activation event, then target neuron identification is accurate, but data transmission volume and processing overhead increase
Solution Approach 1:
The patent merges coordinate values and layer identifiers into a single bundled message structure that is transmitted once for multiple events with common coordinates. This eliminates redundant transmission of coordinate data and layer identifiers while maintaining accurate target neuron identification through the bundled event enumeration mechanism.
Solution Approach 2:
The system extracts common coordinate values and layer identifier information from individual event transmissions and separates them into a shared bundled structure. By taking out the redundant common information and transmitting it once, the data transmission volume is reduced while the extracted information remains available for accurate target neuron identification through the bundled event list.
Data Source
AI summary
A processor system is disclosed herein that comprises a plurality of processor cores. The processor system is configured to execute a neural network having at least a first neural network layer and a second neural network layer. A first of the processor cores is configured to transmit a plurality of activation event data in a packed message. The packed message comprises a common indication for a source of the plurality of activation event data in the first neural network layer.