Transformer Parameter Alignment for Faster SoC Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The Vision Transformer, when incorporated into a system on a chip (SoC) in edge devices like in-vehicle ECUs, experiences slow inference due to parameters not satisfying specified conditions.
Innovation Solution
An information processing apparatus that includes a determiner to check if neural network parameters meet specified conditions and a modifier to adjust these parameters to multiples of specific values, ensuring they align with the SoC requirements, thereby enhancing inference speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If the Vision Transformer uses original parameters, then model accuracy is maintained, but inference speed is slow on SoC
Solution Approach 1:
The system performs preliminary determination of whether parameters satisfy specified conditions before inference execution. The determiner checks parameters in advance, and the modifier pre-adjusts them if needed, avoiding runtime performance issues and ensuring optimal inference speed from the start
Solution Approach 2:
The modifier changes neural network parameters to satisfy specified conditions (such as being multiples of specific values) when the original parameters do not meet these conditions. This parameter adjustment enables faster inference on SoC while maintaining model functionality
2Productivity
If parameters are modified to satisfy SoC conditions, then inference speed improves, but model performance may deteriorate
Solution Approach 1:
The system incorporates a determiner that checks whether parameters satisfy specified conditions and a modifier that adjusts them accordingly. This feedback mechanism ensures parameters are appropriately aligned with SoC requirements while maintaining model performance through controlled modification
Solution Approach 2:
The modifier selectively adjusts parameters to satisfy specified conditions (such as being multiples of specific values) only when necessary, rather than blindly modifying all parameters. This selective parameter change approach improves inference throughput while preserving model accuracy
Data Source
AI summary
An information processing apparatus includes: an NN input component that acquires a neural network including a Transformer; a determiner that determines whether one or more parameters used for the neural network satisfy a specified condition; and a modifier that modifies the one or more parameters to cause the one or more parameters to satisfy the specified condition, when the one or more parameters are determined not to satisfy the specified condition.