Reducible Neural Network Blocks for Efficient NPU Execution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural networks, particularly those based on MobileNetV2 architecture, face inefficiencies when executed on neural processing units (NPUs), leading to reduced performance and mitigated benefits despite their efficiency on general-purpose processors.
Innovation Solution
A neural network architecture that includes reducible blocks, which are transformed from a full format used for training to a reduced format for execution, optimizing operations for efficient execution on NPUs while maintaining accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional inverted bottleneck blocks are used in neural networks, then the network can maintain a compact size, but execution efficiency on NPUs deteriorates due to suboptimal resource utilization
Solution Approach 1:
The inverted bottleneck block is segmented into two separate 3×3 convolution operations instead of maintaining the expanded bottleneck structure. This segmentation allows each convolution to be executed independently and efficiently on NPU hardware, improving resource utilization and execution efficiency while maintaining the compact block size.
Solution Approach 2:
The block structure parameters are changed from the traditional inverted bottleneck configuration (with expansion and compression layers) to a simplified structure using two 3×3 convolutions. This parameter change optimizes the computational operations for NPU execution without increasing the overall model size.
2Measurement precision
If complex operations are used in final layers to maintain accuracy, then model performance is improved, but resource utilization on NPUs deteriorates
Solution Approach 1:
The final layer operations are segmented into multiple 3×3 convolution operations rather than using complex pooled fully-connected layers. This segmentation maintains the accuracy needed for classification while enabling efficient execution on NPU hardware by utilizing the processor's strengths in handling convolutional operations.
Solution Approach 2:
Instead of using traditional pooled fully-connected layers for final classification (which are computationally expensive on NPUs), the approach inverts the strategy by using multiple 3×3 convolutions. This inversion leverages the NPU's optimized convolutional processing to achieve both high accuracy and efficient resource utilization.
3Measurement precision
If model size is increased to improve accuracy, then classification performance is enhanced, but execution efficiency on NPUs deteriorates due to increased computational costs
Solution Approach 1:
The model architecture parameters are changed to use multiple 3×3 convolution operations throughout, including in the final layers. This parameter change enables the model to maintain high classification accuracy without increasing overall model size, while simultaneously improving execution efficiency on NPU hardware by utilizing operations that map well to the processor's architecture.
Data Source
AI summary
Disclosed is a novel neural network architecture and methods for generating neural network-based models from such architecture. A first version of the neural network, that is used for training purposes, includes one or more blocks in a first format that can then be replaced with corresponding blocks in a second format for execution. An executable model can thus be provided comprising a second version of the neural network including the one or more blocks in the second format. This then allows the training to be performed in a first, e.g. expanded format, but with a second, e.g. reduced, format model then provided for execution.