Shared Scratchpad Vector Reduction Without Core Synchronization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network computation systems face inefficiencies in performing vector reductions due to the need for synchronization and resource reservation across multiple processor cores, leading to performance and power overheads, especially in distributed systems.
Innovation Solution
Implementing a hardware circuit with a shared scratchpad memory that allows concurrent vector accumulations using direct memory access (DMA) and an operator unit, enabling atomic reductions without synchronization, and leveraging software-managed addressing to write results directly into shared memory.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If synchronization and resource reservation are used across multiple processor cores for vector reductions, then correctness of computation is ensured, but performance and power overhead increase
Solution Approach 1:
The patent introduces a shared scratchpad memory as an intermediary structure between multiple processor cores. This scratchpad memory enables atomic read-modify-write operations that allow concurrent vector reductions without requiring traditional synchronization mechanisms. The scratchpad acts as a mediator that coordinates access from multiple cores simultaneously, eliminating the need for locks or barriers while ensuring computational correctness through hardware-supported atomic operations.
2Reliability
If synchronization and resource reservation are used across multiple processor cores for vector reductions, then correctness of computation is ensured, but power consumption increases
Solution Approach 1:
The shared scratchpad memory serves as a mediator that eliminates the need for power-intensive synchronization operations. By providing hardware-supported atomic read-modify-write capabilities, the scratchpad allows multiple cores to perform vector reductions concurrently without activating power-consuming synchronization logic, thereby significantly reducing overall power consumption while maintaining computational correctness.
3Adaptability or versatility
If traditional vector reduction methods are used with multiple processor cores, then computation can be distributed, but the number of operations and time required increase
Solution Approach 1:
The patent enables continuous useful action by allowing multiple processor cores to perform vector reduction operations concurrently on the shared scratchpad memory without interruption for synchronization. The atomic read-modify-write operations ensure that each core can continuously accumulate values into the shared memory location simultaneously, eliminating idle time and maximizing utilization of all processing resources throughout the computation.
4Adaptability or versatility
If traditional vector reduction methods are used with multiple processor cores, then computation can be distributed, but operation count increases
Solution Approach 1:
The patent merges the distribution of computation across multiple cores with a unified accumulation location in the shared scratchpad memory. Instead of requiring separate reduction operations for each core that must then be combined, the system allows all cores to simultaneously accumulate their partial results into the same memory location through atomic operations, effectively merging the distributed computation results in hardware and reducing the total operation count.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Methods, systems, and apparatus, including computer-readable media, are described for performing vector reductions using a shared scratchpad memory of a hardware circuit having processor cores that communicate with the shared memory. For each of the processor cores, a respective vector of values is generated based on computations performed at the processor core. The shared memory receives the respective vectors of values from respective resources of the processor cores using a direct memory access (DMA) data path of the shared memory. The shared memory performs an accumulation operation on the respective vectors of values using an operator unit coupled to the shared memory. The operator unit is configured to accumulate values based on arithmetic operations encoded at the operator unit. A result vector is generated based on performing the accumulation operation using the respective vectors of values.