Neural Network Execution Time Estimation in Multi-Core Accelerators
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for estimating the execution time of neural networks in multi-core accelerators fail to consider memory access time and communication overhead between cores, leading to inaccurate performance estimation and performance degradation due to communication contention.
Innovation Solution
A processor-implemented method that generates trace information for each core, including operation timing and communication overhead, and calculates execution time using a weighted node graph and network on chip simulator, reflecting hardware characteristics and compiler optimizations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional execution time estimation methods are used for neural networks in multi-core accelerators, then the estimation process is simple, but the accuracy of execution time estimation is poor because memory access time and communication overhead between cores are not considered
Solution Approach 1:
The patent applies preliminary action by pre-calculating and storing communication overhead values and memory access time values in lookup tables before actual execution time estimation. The trace information generation unit pre-generates execution traces that include communication and memory access characteristics, which are then used during estimation without requiring complex real-time calculations. This resolves the contradiction by preparing data in advance to improve accuracy while avoiding complex runtime computations.
Solution Approach 2:
The patent introduces trace information as an intermediary element that captures communication overhead and memory access characteristics. Instead of directly calculating complex multi-core interactions, the system generates trace information that serves as a mediator between the neural network execution and the execution time estimation. This trace information includes communication patterns and memory access data, allowing accurate estimation without requiring complex direct analysis of all core interactions.
2Measurement precision
If communication overhead between cores is considered in execution time estimation, then the performance prediction accuracy is improved, but the computational complexity of the estimation increases
Solution Approach 1:
The patent uses copying by creating trace information that replicates the communication patterns and execution characteristics of the actual neural network run. Instead of performing complex analytical calculations to estimate communication overhead, the system copies the actual communication behavior into trace data structures that can be processed more efficiently. This allows accurate performance prediction by analyzing copied execution traces rather than computing communication overhead from first principles.
Solution Approach 2:
The patent employs cheap short-living objects by using lightweight trace data structures that can be quickly generated and processed. The trace information is created as a temporary, disposable representation of execution characteristics that can be rapidly processed for estimation purposes and then discarded. This approach provides accurate performance prediction without requiring heavy computational resources, as the trace objects are simple and ephemeral.
3Productivity
If memory access time is included in the execution time calculation, then the bottleneck identification capability is improved, but the complexity of the estimation model increases
Solution Approach 1:
The patent applies local quality by focusing the estimation model on specific local characteristics of memory access patterns rather than attempting to model all memory operations uniformly. The trace information generation unit identifies and highlights critical memory access points that contribute to bottlenecks, applying detailed analysis only where needed. This allows improved bottleneck identification capability while avoiding the complexity of uniformly analyzing all memory operations in the neural network.
Data Source
AI summary
A method and apparatus for estimating execution time of a neural network are provided, the method of estimating execution time of a neural network in a multi-core accelerator, the method including generating trace information including operation timing information for each core of the multi-core accelerator, and calculating the execution time of the neural network reflecting communication overhead between cores of the multi-core accelerator and memory access time for each core of the cores, based on the trace information.