Neural Network Layer-by-Layer Debugging via Tensor Interception
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Debugging neural networks on special-purpose processors, such as those with systolic arrays, is challenging due to inaccessible intermediary results, precision-based summation errors, and the complexity of massive computations, making it difficult to identify defective data processing units and sort through millions of computations for erroneous output values.
Innovation Solution
A layer-by-layer approach is adopted to shorten the neural network, allowing tensor outputs at lower-level intermediate representations to be accessible, by simulating execution to generate reference tensors, repeatedly shortening the network, and then expanding it to identify mismatch locations, thereby minimizing adverse effects on the compiled code.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Difficulty of detecting and measuring
If traditional debugging techniques are used on special-purpose processors, then breakpoint examination can be performed, but intermediary results remain inaccessible and precision-based summation errors cannot be detected
Solution Approach 1:
The patent introduces a debugger program as an intermediary between the special-purpose processor and the user. This debugger program intercepts and examines intermediary results (tensors) during neural network execution, making previously inaccessible data available for analysis without altering the underlying processor architecture.
Solution Approach 2:
The patent creates copies of intermediary tensors at various execution stages and presents them to the user through the debugger program. By copying tensor data from different layers and execution points, the system enables examination of intermediate computation results without requiring modifications to the original processor or execution flow.
2Productivity
If the neural network is executed on special-purpose processors with systolic arrays, then computation performance is improved, but identifying defective data processing units becomes difficult due to massive parallel computations
Solution Approach 1:
The patent segments the neural network execution into discrete layers and individual tensor operations. By dividing the massive parallel computation into layer-by-layer processing stages, the debugger can examine intermediary results at each stage, making it possible to identify which specific layer or processing unit produces erroneous outputs without analyzing all millions of computations simultaneously.
Solution Approach 2:
The debugger program implements feedback by comparing intermediary tensor results against expected values and notifying the user of mismatches. This feedback mechanism allows rapid identification of defective data processing units by highlighting where computations deviate from expected behavior, rather than requiring manual analysis of all computation results.
3Ease of operation
If the neural network is shortened to enable debugging, then intermediary results become accessible, but the compute workload is reduced and performance is affected
Solution Approach 1:
The patent applies partial action by enabling debugging capabilities for specific layers or tensor operations rather than requiring the entire neural network to be shortened or modified. The debugger program can selectively intercept and examine intermediary results at chosen execution points, providing debugging functionality without reducing the overall compute workload or performance of the complete neural network.
Data Source
AI summary
Techniques are disclosed for debugging a neural network execution on a target processor. A reference processor may generate a plurality of first reference tensors for the neural network. The neural network may be repeatedly reduced to produce a plurality of lengths. For each of the lengths, a compiler converts the neural network into first machine instructions, the target processor executes the first machine instructions to generate a first device tensor, and the debugger program determines whether the first device tensor matches a first reference tensor. A shortest length is identified for which the first device tensor does not match the first reference tensor. Tensor output is enabled for a lower-level intermediate representation of the shortest neural network, and the neural network is converted into second machine instructions, which are executed by the target processor to generate a second device tensor.