Model inference method and apparatus, electronic device, and storage medium
By converting the real number model into a quaternion model and performing quantization-aware training and fine-tuning, the problem of insufficient model expressive power under extremely low bit quantization is solved, and the model accuracy is restored and the inference speed is improved.
CN122154803APending Publication Date: 2026-06-05PEKING UNIV
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- PEKING UNIV
- Filing Date
- 2026-01-26
- Publication Date
- 2026-06-05
AI Technical Summary
Technical Problem
Existing technologies have limited model expressive power when using extremely low bit quantization, making it difficult to fully compensate for the accuracy loss introduced by quantization, resulting in poor model inference performance.
Method used
The pre-trained real number model is losslessly and equivalently converted into a quaternion model, and then fine-tuned through quaternion matrix multiplication and quantization-aware training, replacing the sign flipping and component permutation operations for inference.
Benefits of technology
It restores or surpasses the accuracy of full-precision models with extremely low bit quantization, significantly improves inference speed and reduces power consumption, and achieves efficient model compression.
✦ Generated by Eureka AI based on patent content.
Smart Images
Figure CN122154803A_ABST
Abstract
The application provides a model reasoning method and device, electronic equipment and storage medium, and relates to the technical field of artificial intelligence, comprising: converting lossless equivalence of real number matrix multiplication in a pre-trained real number model into quaternion matrix multiplication to obtain a quaternion model; quantizing quaternion weights of the quaternion model; performing quantization-aware training fine-tuning on the quantized quaternion model; and performing multiplication-free reasoning based on the quantization-aware training fine-tuned quaternion model. The application can achieve extremely high model compression rate while maintaining or even improving model accuracy, significantly accelerating model reasoning speed, and improving model reasoning performance.
Need to check novelty before this filing date? Find Prior Art