Instruction Generation for Hierarchical Block Data Transfers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep learning processors, such as GPUs, face inefficiencies due to architectural differences from general-purpose processors, necessitating specialized instruction generation to optimize performance.
Innovation Solution
An instruction generating method that includes generating data transfer instructions for an arithmetic processing device with hierarchical blocks, allowing efficient data transfers between execution sections and blocks, using unicast and multicast instructions optimized by dynamic programming to minimize instruction count.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data transfers are performed between execution sections and blocks in a hierarchical structure, then data transfer efficiency is improved, but the number of instructions required increases
Solution Approach 1:
The patent combines multiple data transfer operations into a single instruction by introducing a multicast instruction that can transfer data from one source to multiple destinations simultaneously. This merging of operations reduces the total instruction count while maintaining efficient data transfer across the hierarchical block structure.
Solution Approach 2:
The multicast instruction serves multiple functions: it can perform one-to-one data transfer like a unicast instruction, and simultaneously perform one-to-many data transfers. This multi-functionality allows a single instruction type to handle various data transfer scenarios, reducing the overall instruction count needed in the system.
2Speed
If specialized instructions are generated for deep learning processors, then calculation speed is improved, but instruction generation complexity increases
Solution Approach 1:
The patent segments data transfer operations into distinct categories (first through fourth data transfers) based on source and destination types. This segmentation allows the instruction generator to systematically handle different transfer scenarios using standardized patterns, reducing generation complexity while optimizing performance for each segment.
Solution Approach 2:
The patent changes the parameters of data transfer instructions by introducing multicast capabilities that modify the destination parameter from a single block to multiple blocks. This parameter change enables more efficient instruction generation for deep learning workloads without requiring completely new instruction formats.
Data Source
AI summary
With respect to a method of generating an instruction to be executed by an arithmetic processing device including first blocks, each of the first blocks including execution sections, the method includes generating, by at least one processor, at least one data transfer instruction that causes the arithmetic processing device to perform at least one of first data transfers, second data transfers, third data transfers, or fourth data transfers. Transfer sources of the first data transfers are execution sections, transfer destinations of the first data transfers are execution sections, transfer sources of the second data transfers are first blocks, transfer destinations of the second data transfers are first blocks, transfer sources of the third data transfers are first blocks, transfer destinations of the third data transfers are execution sections, transfer sources of the fourth data transfers are execution sections, and transfer destinations of the fourth data transfers are first blocks.