Scheduler for removing redundancy in quantized multiplications on inference accelerators

Abstract

Methods and systems, including computer-readable media, are described for a multiplication scheduler for quantized inference (MSQI) computations. The multiplication scheduler is implemented as a MSQI hardware unit on a compute tile / unit of a special-purpose hardware integrated circuit, such as an inference accelerator or tensor processing unit (TPU). The multiplication scheduler is designed to optimize multiplication operations in hardware ML inference accelerators by minimizing or eliminating redundant computations (e.g., multiplications) on prevalent quantized formats.

Images