Neural Network Processor Debugging Circuit for Bottleneck Identification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Complex artificial neural networks face challenges in identifying bottlenecks due to their massive computation requirements and intricate software and hardware configurations, making it difficult to optimize performance effectively.
Innovation Solution
Implementing debugging circuits that generate notification packets for each instruction's execution time across various processing engines, allowing for the identification and optimization of bottlenecks by modifying hardware resources and reordering operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Difficulty of detecting and measuring
If debugging circuits are added to generate notification packets for each instruction, then bottleneck identification capability is improved, but device complexity increases
Solution Approach 1:
The patent introduces notification packets as intermediary data structures that carry timing information between processing engines and the bottleneck identification module. These packets serve as a mediator that enables detailed performance monitoring without requiring complex direct instrumentation of each processing element, thus improving detectability while controlling complexity growth.
Solution Approach 2:
The patent creates simplified copies of execution timing information through notification packets that replicate essential performance data without duplicating the entire processing engine complexity. By copying only the necessary timing and identification fields into notification packets, the system achieves comprehensive bottleneck analysis while maintaining manageable device complexity.
2Measurement precision
If notification packets are generated for each instruction execution, then measurement precision is improved, but loss of time increases
Solution Approach 1:
The patent implements partial action by generating notification packets selectively for instructions that are likely to contain bottlenecks, rather than for every single instruction in the neural network. This approach maintains high measurement precision for critical operations while reducing the overall time overhead by avoiding unnecessary packet generation for routine instructions.
Solution Approach 2:
The patent performs preliminary action by pre-configuring which instructions and processing engines should generate notification packets based on static analysis of the neural network architecture and execution patterns. This pre-filtering approach ensures that measurement precision is maintained for critical path operations while minimizing time overhead by excluding non-critical instructions from detailed timing analysis.
Data Source
AI summary
Disclosed herein are techniques for debugging the performance of a neural network. In one embodiment, a neural network processor includes a processing engine, a debugging circuit coupled to the processing engine, and an interface to a memory device. The processing engine is configured to execute instructions for implementing a neural network. The debugging circuit is configurable to determine, for each instruction in a set of instructions, a first timestamp indicating a start time of executing the instruction and a second timestamp indicating an end time of executing the instruction by the processing engine. The interface is configured to save the first timestamp and the second timestamp for each instruction in the set of instructions into the memory device. The debugging circuit can be configured to different debug levels. The neural network processor can include multiple debugging circuits for multiple processing engines that operate in parallel.