Residual Quantization for Neural Networks

By adopting a hybrid precision data format and a quantized precision floating-point format on neural network accelerator, the problem of high computational cost in training and using neural networks in the prior art is solved, and more efficient computing performance and accuracy are achieved.

CN113168563BActive Publication Date: 2025-03-28MICROSOFT TECHNOLOGY LICENSING LLC
2 Cites 0 Cited by

Patent Information

Application Number
CN201980081880.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2018-12-14
Filing Date
2019-12-03
Publication Date
2025-03-28
Estimated Expiration
2039-12-03

AI Technical Summary

Technical Problem

The prior art has high computational costs when training and using neural networks, resulting in difficulty in performing feature extraction in real time on a general-purpose processor.

Method used

The neural network is trained in a hybrid precision data format, and the accelerator is configured to perform tensor operations on a given layer, converting the input tensor in the normal precision floating point format into a quantized precision floating point format, and using residual tensors when necessary to improve output accuracy.

Benefits of technology

Through mixed precision training, computational complexity and storage requirements are reduced, computational performance and energy efficiency of neural network accelerators are improved, accuracy is improved in classification tasks, and convergence time of training is reduced.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

A method and apparatus for providing simulation of operations of quantized precision in a neural network are disclosed. In some examples, operations of quantized precision are performed in a block floating point format in which the values ​​of the tensors share a common exponent. Techniques for selecting higher or lower precision can be used based on various input indices. When converted to a quantized tensor, a residual tensor is generated. In one embodiment, an error value associated with the conversion from a normal precision floating point number to a quantized tensor is used to determine whether to use a residual tensor in a dot product calculation. Using a residual tensor can improve the precision of the output from a node. The choice of whether to use a residual tensor depends on various input indices, including an error value, a layer number, an exponent value, a layer type, and the like.
Need to check novelty before this filing date? Find Prior Art