Hierarchical Memory for Parallel Arithmetic in Deep Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning processors require specialized compilers to efficiently execute instructions, as their architectures differ from general-purpose processors, leading to suboptimal performance due to inefficient instruction execution and data transfer.
Innovation Solution
A compiler device with an interface for receiving instruction sequences, hierarchical memory for data transfer, and parallel arithmetic operators to merge instructions that can be executed in parallel, reducing resource conflicts and improving processing performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If instructions are executed sequentially in deep learning processors, then resource conflicts are avoided, but processing performance deteriorates due to inefficient instruction execution
Solution Approach 1:
The patent merges multiple independent instructions into a single combined instruction that can be executed in parallel by multiple arithmetic operators. This allows instructions that previously had to be executed sequentially to now run simultaneously, improving processing performance while managing complexity through unified instruction structure
Solution Approach 2:
The patent segments the instruction execution process into hierarchical memory levels (zeroth memories connected to first memories, first memories connected to second memories), allowing different segments of data to be accessed from different memory levels simultaneously, enabling parallel execution without resource conflicts
2Speed
If data transfer between memories is performed frequently, then data availability for parallel computation is improved, but processing time increases due to transfer overhead
Solution Approach 1:
The patent implements preliminary data transfer from zeroth memories to first memories before parallel computation begins. This preliminary action ensures that data is already available in the first memories when parallel computation starts, eliminating the need for frequent data transfers during computation and reducing overall processing time
Solution Approach 2:
The patent creates a nested hierarchical memory structure where first memories are nested within the data transfer path between zeroth and second memories. This nested arrangement allows data to be pre-loaded into first memories while simultaneously allowing further data transfer to second memories, enabling overlapping operations that reduce total transfer time
Data Source
AI summary
An accelerator includes an interface configured to receive an instruction sequence including a plurality of instructions; a hierarchical memory configured to perform data transfer between a plurality of zeroth memories and a plurality of first memories according to a data transfer instruction specifically for data transfer between the plurality of zeroth memories and the plurality of first memories included in the instruction sequence received by the interface, the hierarchical memory including the plurality of zeroth memories, the plurality of first memories, and one or more second memories, each of the one or more second memories being connected to corresponding first memories among the plurality of first memories, and each of the plurality of first memories being connected to corresponding zeroth memories among the plurality of zeroth memories; and a plurality of arithmetic operators configured to operate in parallel by using the hierarchical memory.