Cooperative Compiler Framework for Scratchpad Memory Allocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current scratchpad memory allocation algorithms for heterogeneous devices in neural network computing are wasteful due to the lack of reuse of allocated SPM locations across different compilers, leading to inefficient use of limited resources.
Innovation Solution
A method and system that unify records of the same objects across different compilation states to enable cooperative allocation of scratchpad memory (SPM) to subgraphs in neural network models, optimizing SPM allocation at compile time using a global optimization manager.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conservative SPM allocation algorithms are used to avoid data hazards across different compilers, then timing predictability and data safety are ensured, but SPM resource utilization deteriorates due to lack of reuse
Solution Approach 1:
The patent merges the allocation decisions of multiple target-specific compilers into a unified global allocation framework. The global optimization manager consolidates SPM allocation across different compilers and target devices, allowing shared reuse of SPM locations while maintaining timing guarantees through coordinated management of data hazards.
Solution Approach 2:
The patent creates a universal SPM allocation mechanism that serves multiple compilers and target devices simultaneously. The global optimization manager acts as a multi-functional coordinator that manages SPM resources for heterogeneous devices, enabling the same SPM locations to be reused by different compilers under unified control.
2Adaptability or versatility
If multiple target-specific compilers are used to compile subgraphs for heterogeneous devices, then device-specific optimization is achieved, but SPM allocation efficiency deteriorates due to lack of coordination
Solution Approach 1:
The patent segments the compilation process into two distinct layers: target-specific compilers that handle device-specific optimization for individual subgraphs, and a global optimization manager that handles cross-compiler coordination. This segmentation allows each component to focus on its specialized function while the global manager ensures overall efficiency.
Solution Approach 2:
The global optimization manager serves as an intermediary between multiple target-specific compilers. It receives allocation requests from various compilers, coordinates their SPM usage to enable reuse, and resolves potential data hazards. This intermediary role maintains device-specific optimization while improving overall allocation efficiency.
3Reliability
If SPM locations are allocated separately by each compiler without reuse, then data hazards are avoided, but resource waste increases
Solution Approach 1:
The global optimization manager implements a feedback mechanism where allocation decisions from one compiler are communicated to subsequent compilers. This feedback allows the system to make informed reuse decisions, allocating SPM locations to multiple compilers only when safe, thereby reducing waste while maintaining data safety through continuous coordination.
Data Source
AI summary
A system allocates scratchpad memory (SPM) to heterogeneous devices for neural network computing. The system executes the operations of a global optimization manager. The global optimization manager receives compilation states from compilers, which compile corresponding subgraphs of a neural network model into corresponding subcommands that run on the heterogeneous devices. The global optimization manager unifies records of a same object across different ones of the compilation states, and allocates the SPM to the subgraphs according to the unified records of the compilation states.