Neural Network Reduction for Data-Dependent Compiler Bug Isolation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Identifying and reproducing compiler errors in neural networks is challenging due to the time-consuming and labor-intensive process of pinpointing error regions, especially when errors are data-dependent and involve sensitive information, making it difficult to reproduce exact bugs using fake models with random data.
Innovation Solution
A method involving a reference compiler and user compiler to rearrange weights and tensors, followed by network partitioning into sub-networks, and further reduction by a data reducer to isolate and reproduce compiling bugs efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Object-affected harmful factors
If fake models with random weights and random data are used instead of original network models, then sensitive information is protected, but the ability to reproduce exact errors is lost because errors are data dependent
Solution Approach 1:
The patent creates a simplified copy of the original neural network model that preserves the error-causing structure and data dependencies. Instead of using the full original model with sensitive data, a reduced copy is generated that maintains the critical error-reproducing characteristics while removing unnecessary complexity and sensitive information.
2Measurement precision
If the original network model with hundreds to thousands of layers is used to identify error regions, then exact error location can be found, but the process becomes time-consuming and labor-intensive
Solution Approach 1:
The patent segments the large neural network model into smaller, manageable sub-networks or modules. By dividing the complex model with hundreds to thousands of layers into smaller segments, the error identification process can focus on specific regions independently, dramatically reducing the time and effort required while maintaining precision in locating error-causing layers.
Solution Approach 2:
The patent extracts and removes irrelevant or redundant parts of the neural network model that do not contribute to the error. By taking out unnecessary layers, operations, or data paths from the original model, the reduced model retains only the essential error-causing components, making bug identification faster and more efficient.
3Productivity
If the neural network is reduced into smaller sub-networks, then bug identification becomes more efficient, but the complexity of the compilation and partitioning process increases
Solution Approach 1:
The patent performs preliminary analysis and preparation steps before partitioning the network model. This includes pre-identifying potential error-causing regions, pre-processing the model structure, and pre-planning the partitioning strategy. By doing this preliminary work upfront, the actual bug identification process becomes more efficient, and the complexity of partitioning is managed systematically rather than ad hoc.
Data Source
AI summary
A method for reducing a neural network includes compiling the neural network by a reference compiler to rearrange reference weights, manipulating a reference tensor inputted to the neural network to output reference tensors, compiling the neural network by a user compiler to rearrange user weights, manipulating the reference tensor inputted to the neural network to output a user tensor, if a reference tensor of a last layer of the neural network is inconsistent with the user tensor, then a network reducer sorting and partitioning the neural network into a plurality of sub-networks each containing at least one layer. If the user tensor is inconsistent with a corresponding reference tensor, and the network reducer is unable to further partition the sub-network, then output the sub-network to a data reducer. The data reducer simplifies the reference tensor inputted to the corresponding sub-network and simplifies corresponding user weights.