Tensor Order-of-Magnitude Analysis for ML Computation Verification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for comparing tensor data generated by different hardware executing machine learning models are inadequate, particularly for large and complex models, as they often rely on final outputs or subsets like Top1 or Top5 values, which do not sufficiently verify the accuracy of ML computations and may mask errors that propagate through layers.
Innovation Solution
A new approach that calculates and graphs relative errors against the order of magnitude of tensors, discarding large-order tensors to focus on smaller tensors with significant deviations, allowing identification of issues like bugs or precision differences between target and reference systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional methods compare only final outputs or Top1/Top5 values, then verification process is simple and fast, but accuracy of ML computation verification is insufficient and errors may be masked
Solution Approach 1:
The patent segments the verification process by dividing tensors into different order of magnitude groups. Instead of treating all tensor values uniformly, the system segments them into significant values (|x| ≤ threshold) and less significant values (|x| > threshold), applying different verification strategies to each segment. This allows thorough verification of critical values while efficiently handling less critical ones, resolving the contradiction between verification accuracy and process complexity.
Solution Approach 2:
The patent introduces order of magnitude as a new parameter to differentiate verification depth. By changing the verification parameter from uniform treatment to magnitude-based stratification, the system achieves higher measurement precision for critical tensor values while managing overall complexity through parameter-driven differentiation. The threshold parameter controls the balance between verification thoroughness and computational cost.
2Measurement precision
If all tensor values are compared to verify computation accuracy, then verification accuracy is high, but processing time and computational resources increase significantly
Solution Approach 1:
The patent applies partial action by selectively verifying only the significant portion of tensor values (those with |x| ≤ threshold) in detail, while applying coarser verification or skipping verification for less significant values (|x| > threshold). This partial verification approach achieves sufficient verification accuracy for critical values without the prohibitive cost of verifying all values with equal thoroughness, directly addressing the time-accuracy tradeoff.
Solution Approach 2:
The patent uses the threshold parameter to control verification depth and resource allocation. By adjusting the threshold, the system dynamically balances verification accuracy against processing time. Smaller thresholds verify more values in detail (higher accuracy, more time), while larger thresholds verify fewer values (lower accuracy, less time), providing a controllable tradeoff mechanism.
3Reliability
If large tensors are included in verification, then comprehensive coverage is achieved, but difficulty of detecting and measuring errors increases
Solution Approach 1:
The patent segments the tensor verification task by order of magnitude, creating distinct groups of significant and less significant values. This segmentation reduces the difficulty of error detection by focusing detailed analysis on the smaller, more critical values where errors are more detectable and meaningful, while applying simplified detection to larger values that contribute less to overall accuracy.
Solution Approach 2:
The patent applies local quality by using different verification thresholds and strategies for different ranges of tensor values. Instead of uniform verification, the system applies stricter, more detailed verification to significant values (|x| ≤ threshold) and more lenient verification to less significant values (|x| > threshold), matching the verification effort to the local importance of each value range.
Data Source
AI summary
A system includes a machine learning (ML) accelerator running a first code generated by a first compiler that generates a first plurality of tensors associated with one or more ML operations of a ML model. The system includes a processor that receives the first and the second plurality of tensors associated with the ML model. The second plurality of tensors is generated by a second code generated by a second compiler running on a hardware executing the one or more ML operations of the ML model. The processor generates a plurality of relative errors associated with the first and second plurality of tensors. The processor calculates an order of magnitude associated with the first plurality of tensors and generates a graph associated with the plurality of relative errors and the calculated order of magnitude associated with the first plurality of tensors. The graph is rendered.