Neuron Core Synchronization via Batch Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network processors face significant challenges due to the need for storing all internal states of neural network elements in memory, leading to high storage requirements and restricted application mapping options.
Innovation Solution
The proposed event-based neural network processor organizes neural network elements into source and current batches, utilizing a temporary storage space dynamically allocated to current batches, which allows for moderate memory requirements and efficient processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all internal states of neural network elements are stored in memory, then the neuron states can be maintained and updated, but the storage requirements become excessively high and application mapping options are restricted
Solution Approach 1:
The patent segments the neural network layers into multiple batches, where each batch is processed independently with its own temporary storage space. This segmentation allows the system to handle large-scale neural networks by dividing them into manageable portions, reducing the peak memory requirements while maintaining the ability to store and update neuron states for each batch separately
Solution Approach 2:
The patent implements dynamic allocation of temporary storage space to batches based on their processing needs. The storage capacity is dynamically adjusted to accommodate the specific requirements of each batch being processed, allowing efficient memory utilization without permanently allocating excessive storage resources for the entire neural network
2Quantity of substance
If temporary storage space is dynamically allocated to multiple batches, then memory requirements are reduced, but synchronization between processor cores becomes complex
Solution Approach 1:
The patent implements a feedback mechanism using end-of-batch (EOB) notification messages that signal when a batch has completed processing. This feedback allows processor cores to coordinate their operations, ensuring that synchronization occurs only when necessary (at batch boundaries) rather than requiring continuous complex synchronization protocols throughout the processing of each batch
Solution Approach 2:
The patent employs periodic batch processing where synchronization occurs at regular intervals corresponding to batch completions. Instead of requiring continuous synchronization between cores, the system uses periodic synchronization points marked by EOB messages, simplifying the synchronization mechanism while maintaining data consistency across batches
3Productivity
If neuron states are updated in response to event messages, then the neural network processing is event-driven and efficient, but an avalanche of event messages can occur
Solution Approach 1:
The patent segments the neural network processing into discrete batches, which limits the propagation scope of event messages. By confining event-driven processing within batch boundaries and using EOB messages to delimit batch completions, the system prevents event messages from cascading indefinitely across the entire network, thereby controlling the event message rate while maintaining event-driven efficiency within each batch
Data Source
AI summary
Examples relate to event-based neural network processing. A neural network includes a source neural network layer with source batches and a current neural network layer with current batches. A section in a temporary storage space is temporarily allocated to a number of the current batches. For each current batch in the number of current batches, one or more event messages are received from one or more respective source batches associated with the current batch. The section in the temporary storage space is used to update neuron state data of respective neural network elements in the current batch. Based on compliance with an emission condition and an activation function of a neural network element in the current batch, an event message is issued for the neural network element to a destination. The section in the temporary storage space is released and an end of batch notification is issued.