Neural Network Code Generation via Hardware-Aware Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural network operations face inefficiencies in data loading and processing due to varying device performance and tiling methods, which affect the dimensional data division and dataflow order, leading to suboptimal performance and resource utilization.
Innovation Solution
A processor-implemented method and apparatus that generate a target mapping model based on hardware information and neural network structure, optimizing the mapping model by calculating mapping parameters, memory access size, and pruning inadequate models to maximize performance and resource utilization, thereby generating an optimized code for neural network operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If different tiling methods are used to divide dimensional data, then device performance and resource utilization may improve, but the complexity of determining optimal tiling increases
Solution Approach 1:
The system changes parameters such as tiling dimensions, dataflow configurations, and mapping strategies to optimize neural network operation performance. By systematically varying these parameters and evaluating their impact on device performance, the system identifies optimal configurations without manually designing complex tiling methods.
Solution Approach 2:
The code generation apparatus automatically generates optimized codes by self-evaluating different tiling methods and mapping models. The system uses performance prediction models to assess various configurations and selects the best one autonomously, eliminating the need for external manual optimization expertise.
2Productivity
If data is divided and loaded using different tiling methods, then resource utilization may improve, but the number of data reload operations increases
Solution Approach 1:
The system performs preliminary actions by pre-calculating optimal tiling configurations and mapping models before executing neural network operations. By determining the best data division strategy in advance using performance prediction, the system minimizes redundant data reload operations during actual execution, thus reducing time loss.
Solution Approach 2:
The system uses feedback from performance prediction models to evaluate different tiling methods and their impact on data reload frequency. By continuously assessing the relationship between tiling configurations and reload operations, the system selects configurations that maximize resource utilization while minimizing unnecessary data reloading.
3Productivity
If mapping models are optimized based on hardware information and neural network structure, then operation performance improves, but the complexity of model selection increases
Solution Approach 1:
The code generation apparatus autonomously performs model selection by evaluating multiple mapping models against hardware information and neural network structure. The system uses built-in performance prediction capabilities to automatically identify the optimal mapping model without requiring external intervention or complex manual selection processes.
Solution Approach 2:
The system introduces an intermediate performance prediction model that mediates between hardware information, neural network structure, and mapping model selection. This intermediary component simplifies the selection process by providing performance estimates for different mapping models, enabling automated selection without direct complex analysis of all possible configurations.
Data Source
AI summary
A method and an apparatus for generating a code for a neural network operation are disclosed. The method includes receiving information on hardware configured to perform a neural network operation of the neural network, generating, using a processor, a target mapping model mapping the neural network operation on processing elements available to perform the neural network operation based on the information and a structure of the neural network, and generating a code to configure the hardware to perform the neural network operation based on the target mapping model.