Intelligent precision scaling and rounding adaptation

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for implementing a runtime adaptive precision scaling system in one or more hardware accelerators in a machine learning subsystem. One of the methods includes processing, by a machine-learning subsystem, a workload using full-precision operations. While processing the workload using the full-precision operations, an error value between processing the workload using the full-precision operations and processing the workload using reduced-precision operations is computed. In response to determining that the error value satisfies a threshold, the machine learning subsystem switches to processing the workload using the reduced-precision operations.

Images