Communication Model Quantization for Low-Overhead Terminal AI
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI models in communication systems face challenges in being small enough for terminal scenarios while maintaining high throughput, due to high memory footprints and computational demands, which are not adequately addressed by current quantization methods.
Innovation Solution
Implementing model quantization configuration information exchange between communication devices to reduce AI model size and improve inference efficiency by converting high-precision models to lower precision formats like INT8, using quantization aware training or post-training quantization, and optimizing parameter precision.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If AI models are used in terminal scenarios, then high throughput and intelligence are achieved, but model size and memory footprint become too large for terminal devices
Solution Approach 1:
The patent applies parameter changes by converting model parameters from high-precision floating-point format to low-precision integer format (quantization). This changes the numerical representation parameters of the model, reducing the bit width required to store each parameter while maintaining acceptable inference accuracy, thus resolving the contradiction between model size and throughput
2Measurement precision
If high-precision AI models are deployed, then inference accuracy is maintained, but memory consumption and power usage increase
Solution Approach 1:
The patent changes the precision parameter of model weights and activations from high-precision floating-point to low-precision integer formats. This parameter transformation reduces memory bandwidth requirements and computational energy consumption while maintaining inference accuracy within acceptable ranges through quantization-aware training or post-training quantization techniques
3Measurement precision
If full-precision model parameters are transmitted over the air interface, then model accuracy is preserved, but transmission overhead increases
Solution Approach 1:
The patent changes the representation parameter of model parameters from full-precision floating-point to compressed low-precision integer formats before transmission. This parameter transformation significantly reduces the number of bits required to represent each parameter, thereby reducing air interface transmission overhead while maintaining sufficient model precision for terminal inference tasks
Data Source
Figure 1~2
Figure 3~4B
Figure 5~6
AI summary
The present application relates to a communication method and device. The method comprises: a first communication device receives model quantization configuration information from a second communication device; and the first communication device executes a model quantization related operation according to the model quantization configuration information. In embodiments of the present application, the model quantization configuration information can indicate a model quantization related operation, and model quantization is realized.