数据处理方法、电子设备、存储介质和程序产品
By introducing learnable compensation parameters in low-bit quantization training and combining them with historical data, the scaling factor is dynamically corrected, which solves the quantization error and training instability problems caused by relying on historical maximum values in existing technologies, and improves the stability and accuracy of the model.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHANGHAI BIREN TECH CO LTD
- Filing Date
- 2025-11-21
- Publication Date
- 2026-07-17
AI Technical Summary
Existing technologies rely solely on historical maximum values to estimate scaling factors, leading to increased quantization errors and unstable model training during low-bit quantization training, especially with decreased accuracy when outliers occur.
A learnable compensation parameter is introduced, which is updated by the loss value of the model training. This parameter is then combined with a scaling factor based on historical data to dynamically adjust the scaling factor to adapt to the current data distribution. The final scaling factor is determined by a linear summation operation.
It improves the accuracy of quantization, reduces quantization error, enhances the stability and final accuracy of model training, and adapts to dynamic changes in data distribution and the impact of outliers.
Smart Images
Figure CN121166074B_ABST