NPU Quantization for Voice Chip Processing Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current voice chips face processing burdens due to reduced on-chip memory and internal high-bandwidth memory, leading to decreased data loading speed and insufficient computing power for AI tasks, particularly in dual-core architectures where one core handles all voice processing.
Innovation Solution
A neural network processing unit (NPU) with a quantizing unit and an operation unit is introduced, which quantizes float-type input data, performs matrix-vector or convolution operations, and applies inverse quantization to reduce processing burdens and enhance efficiency by offloading calculations from the core.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If one core in a dual-core architecture is used for voice processing, then the hardware structure is simple, but the processing burden on the core becomes huge
Solution Approach 1:
The patent divides the processing system into two independent parts: a neural network processing unit (NPU) dedicated to voice processing and a general-purpose core for control logic. This segmentation allows the voice processing burden to be isolated from the main control core, enabling parallel processing and eliminating the bottleneck of single-core processing.
2Measurement precision
If float type data is processed directly, then the calculation precision is high, but the processing speed and efficiency are reduced
Solution Approach 1:
The patent changes the data representation parameter from float type to quantized integer type for internal neural network calculations. This parameter transformation reduces the computational complexity and increases processing speed while maintaining sufficient precision through controlled quantization. The inverse quantization step restores the original float precision for output.
3Productivity
If quantization is applied to neural network calculations, then the processing efficiency is improved, but the implementation complexity increases
Solution Approach 1:
The patent extracts the quantization and inverse quantization operations as dedicated functional units within the NPU architecture. By separating these operations from the main processing pipeline and implementing them as specialized hardware blocks, the system achieves efficient quantized processing without overwhelming implementation complexity in the main control logic.
Data Source
AI summary
A neural network processing method, a neural network processing unit (NPU) and a processing device are provided. The method includes: obtaining by a quantizing unit in the NPU float type input data, quantizing the float type input data to obtain quantized input data, and providing the quantized input data to an operation unit; performing by the operation unit of the NPU a matrix-vector operation and/or a convolution operation to the quantized input data to obtain an operation result of the quantized input data; and performing by the quantizing unit inverse quantization to the operation result output by the operation unit to obtain an inverse quantization result.