Neuro-synaptic Circuitry Loop Buffer for Parallel Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Digital neuromorphic processors face inefficiencies due to trade-offs between flexibility and power consumption, particularly during peak neuron activity, leading to increased latency and loss of accuracy, and are limited by the need for repetitive instruction fetching and separate memory management for neuron states and synaptic weights.
Innovation Solution
A neuro-synaptic processing circuitry with a data memory, neuron processing elements (NPEs) that execute SIMD instructions, a general-purpose CPU for control, and a loop buffer that manages micro-code kernels for parallel and time-multiplexed operations, allowing flexible neural network emulation without CPU intervention and reducing power consumption by disabling unused NPEs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a general-purpose processor is used to perform neuromorphic operations, then flexibility to support various neural networks is maintained, but the processor becomes overloaded during peak neuron activity resulting in increased latency and reduced efficiency
Solution Approach 1:
The system segments processing tasks by separating control functions (handled by the general-purpose processor) from execution functions (handled by dedicated NPEs). The loop buffer segments and queues micro-code kernels for iterative execution, allowing the general-purpose processor to initiate tasks without being blocked by their execution, thereby resolving the overload issue while maintaining flexibility.
Solution Approach 2:
The loop buffer acts as an intermediary between the general-purpose processor and NPEs. It receives micro-code kernels from the processor, manages their iterative execution, and coordinates data flow between memory and processing elements. This mediator enables asynchronous operation where the processor can continue preparing new tasks while previous tasks are being executed, eliminating the bottleneck.
2Productivity
If hardware acceleration is implemented for specific neuromorphic operations, then processing efficiency is improved, but flexibility is limited to only the operations supported by the hardware acceleration
Solution Approach 1:
The NPEs are designed as universal processing elements that can execute various types of neuromorphic operations through programmable micro-code kernels stored in the loop buffer. Instead of dedicated hardware for each operation type, the system uses multi-functional NPEs that can be configured via software to perform different operations, thereby maintaining flexibility while achieving hardware-level execution efficiency.
Solution Approach 2:
The system dynamically configures NPE behavior through micro-code kernels loaded into the loop buffer. The processing functionality is not fixed but can be adjusted by loading different micro-code sequences, allowing the hardware to adapt to various neural network operations while maintaining high-speed execution. This dynamic reconfigurability bridges the gap between fixed hardware efficiency and software flexibility.
3Measurement precision
If a general-purpose processor fetches and executes instructions for each neuron update, then accurate processing is achieved, but energy consumption increases due to repetitive instruction fetching
Solution Approach 1:
The system prepares micro-code kernels in advance and stores them in the loop buffer before execution is needed. These pre-prepared instruction sequences contain all necessary operations for neuron updates, eliminating the need for real-time instruction fetching during execution. The general-purpose processor loads these kernels beforehand, and the NPEs execute them iteratively without requiring continuous processor intervention, thereby reducing energy consumption while maintaining accuracy.
Solution Approach 2:
The loop buffer enables continuous execution of micro-code kernels by NPEs without interruption for instruction fetching. Once a kernel is loaded, the NPEs can iterate through it continuously, performing neuron updates without pausing to retrieve new instructions from the general-purpose processor. This continuous execution mode eliminates the energy-wasting fetch-decode-execute cycle while preserving processing accuracy through the pre-validated micro-code sequences.
4Device complexity
If separate memory management is used for neuron states and synaptic weights, then data organization is simplified, but the system requires separate memory ports and increased complexity in data flow management
Solution Approach 1:
The system merges the memory management for neuron states and synaptic weights into a unified data memory structure accessed through a single memory port. The loop buffer manages a unified address space that can reference both neuron states and synaptic weights, eliminating the need for separate memory ports and reducing the complexity of memory management. This unified approach simplifies the data flow while maintaining efficient access to all required data through centralized control.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Example embodiments describe a neuro-synaptic processing circuitry (100) for performing neuro-synaptic operations based on synaptic weights (131) and neuron states (132) and comprises i) a data memory (130) for storing the synaptic weights and neuron states; the data memory having a first memory port (135) for loading and storing data (136) from and to the data memory; ii) a plurality of neuron processing elements, NPEs, (160) configurable to execute NPE instructions (124) in parallel according to a single instruction, multiple data, SIMD, instruction set; wherein the NPEs have access to respective portions (137) of the memory port; the SIMD instruction set comprising instructions for loading and storing the synaptic weights and neuron states from and to the memory port, and for performing the neuro-synaptic operations; iii) a general-purpose central processing unit, GP-CPU, (140) configured to execute program code; iv) a loop buffer (120) having a register-based memory (123); an address calculation unit (121); and a program counter (122).