ML Compiler Sub-graph Validation via Dual-Execution Comparison
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current validation and debugging processes for machine learning (ML) network models are inefficient and incomplete, as they are performed in isolation at different stages of the compilation and transformation process, lacking comprehensive end-to-end validation and requiring extensive continuous integration, which is time-consuming and resource-intensive due to the vast validation space of emerging ML network models with varying tensor shapes, layouts, and quantization requirements.
Innovation Solution
A holistic approach is proposed that partitions ML network models into well-defined sub-graphs, generating internal representations for each unit, and compiling them into two sets of low-level instructions for execution on both ML-specific hardware and a secondary device, allowing for comprehensive validation and debugging through comparison of output results, thereby reducing validation time and resource requirements while ensuring thorough coverage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If validation and debugging are performed in isolation at different stages of the compilation process using unit tests, then the validation process is simpler to implement, but the validation coverage is incomplete and requires extensive continuous integration
Solution Approach 1:
The patent segments the ML network model into well-defined sub-graphs with specific entry and exit points. Each sub-graph represents a discrete computational unit that can be independently validated. This segmentation allows the validation system to break down the complex end-to-end validation into manageable pieces while maintaining comprehensive coverage, resolving the contradiction between simple implementation and complete validation.
2Reliability
If comprehensive end-to-end validation is performed for the entire ML network model, then the validation coverage is complete, but the validation time and resource requirements increase significantly
Solution Approach 1:
By dividing the ML network model into sub-graphs, the patent enables parallel validation of multiple segments simultaneously. Each sub-graph can be validated independently and in parallel with others, significantly reducing the total validation time compared to sequential end-to-end validation, while maintaining comprehensive coverage through the collective validation of all sub-graphs.
Solution Approach 2:
The patent validates sub-graphs individually rather than requiring complete end-to-end validation for the entire model. This partial action approach allows validation to be performed on specific segments without executing the full compilation and validation pipeline, reducing resource requirements and validation time while still achieving comprehensive coverage across all model components.
3Ease of operation
If unit tests are simplified to generate reference results with ease, then the ease of operation is improved, but the measurement precision of validation results decreases
Solution Approach 1:
The patent introduces intermediate representation (IR) of sub-graphs as a mediator between the high-level ML model and low-level hardware instructions. This IR serves as a precise intermediate form that preserves computational semantics while enabling systematic validation. The IR allows for accurate reference result generation without excessive simplification, maintaining measurement precision while improving ease of operation through standardized representation.
Data Source
AI summary
A system to support validation and debugging of compiled low-level instructions for a machine learning (ML) network model on an ML-specific hardware. A compiler identifies well-defined boundaries in the ML network model based on primitives used to generate low-level instructions for the hardware. The ML network model is partitioned into units/layers/sub-graphs based on the plurality of well-defined boundaries. The compiler then generates an internal representation for each of the units wherein the internal representation is mapped to components in the hardware. Each of the units is compiled into a first set to be executed on the ML-specific hardware and a second set to be executed on a second computing device. The output results from executing the two sets of low-level instructions are compared to validate the first set of low-level instructions. If the outputs do not match fully, the first set of low-level instructions is debugged and recompiled.