AI Tensor Verification and Error Classification Across Compilers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for comparing and validating machine learning (ML) model performance across different hardware systems are inadequate, particularly for large and complex ML models, as they often rely on final output verification which may not detect underlying errors or mismatches, and do not provide insights into the causes of discrepancies.
Innovation Solution
A method is introduced to compare tensors generated by a target system with those from a reference system by calculating relative errors and order of magnitude, using ML models to classify errors and identify their causes, focusing on tensors with smaller order of magnitude to detect significant issues.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If conventional final output comparison is used for verification, then verification simplicity is maintained, but measurement precision and ability to detect underlying errors deteriorates
Solution Approach 1:
The patent segments the verification process into multiple stages: intermediate tensor verification at different computation stages, layer-by-layer comparison, and final output verification. This segmentation enables detection of errors at their source rather than only at the final output, thereby improving measurement precision while maintaining operational feasibility through systematic breakdown of the verification task
Solution Approach 2:
The patent applies partial verification by selectively comparing only critical tensors and intermediate results that have significant impact on final output accuracy, rather than verifying every single tensor. This approach maintains verification simplicity while improving error detection capability by focusing computational resources on the most informative comparisons
2Measurement precision
If layer-by-layer tensor comparison is performed, then measurement precision and error detection capability are improved, but device complexity and computational overhead increases
Solution Approach 1:
The patent applies local quality by differentiating verification intensity across different layers and tensor types. Critical layers with higher impact on final accuracy receive more rigorous verification, while less critical layers use simplified comparison methods. This selective approach improves overall verification accuracy without uniformly increasing system complexity across all components
Solution Approach 2:
The patent changes verification parameters dynamically based on layer characteristics, tensor dimensions, and computational complexity. Verification thresholds, sampling rates, and comparison granularity are adjusted according to local requirements, enabling high precision where needed while reducing complexity in less critical areas
3Measurement precision
If all tensor values are compared for verification, then measurement precision is improved, but loss of time and computational resources increases
Solution Approach 1:
The patent implements partial verification by comparing only the most informative tensor elements rather than all values. This includes focusing on tensors with larger magnitudes, critical intermediate results, and samples strategically selected to maximize detection probability. This approach maintains high verification completeness while dramatically reducing the time and computational resources required
Solution Approach 2:
The patent performs preliminary filtering and prioritization of tensors before comparison, identifying which tensors are most likely to contain errors based on their position in the computation graph, their impact on final output, and statistical properties. This preliminary action enables focused verification that achieves high measurement precision with reduced time investment
4Productivity
If conventional verification methods are used, then productivity is maintained, but reliability and ability to identify error sources deteriorates
Solution Approach 1:
The patent implements feedback mechanisms where verification results from intermediate tensors are used to guide subsequent verification steps. When errors are detected at intermediate stages, the system provides feedback to adjust verification focus, prioritize problematic areas, and identify error sources more reliably. This feedback loop improves reliability without significantly impacting productivity through intelligent resource allocation
Data Source
AI summary
A system includes a machine learning (ML) accelerator running a first code generated by a first compiler that generates a first plurality of tensors associated with one or more ML operations of a ML model. The system includes a processor that receives the first and the second plurality of tensors associated with the ML model. The second plurality of tensors is generated by a second code generated by a second compiler running on a hardware executing the one or more ML operations of the ML model. The processor generates a plurality of relative errors associated with the first and second plurality of tensors. The processor calculates an order of magnitude associated with the first plurality of tensors. The processor extracts features from the plurality of relative errors and the plurality of order of magnitude values and generates the error classification model based on the one or more features.