Precision compensation circuit for sparse neural network computation

Abstract

A tensor processing circuit with precision compensation is disclosed for neural network computations between weight and activation tensors. The circuit comprises a tensor cache to store the weight tensor and a tensor mask cache to store a tensor mask associated with the activation tensor. The tensor mask contains data about high- and low-magnitude features of the activation tensor, with the low-magnitude features encoded to reduce memory usage while accepting a precision trade-off. A computation circuit processes the neural network operation based on the weight tensor, tensor mask, and activation tensor, generating an intermediate tensor. A precision compensation circuit then adjusts this intermediate tensor to correct for precision losses due to the sparsification by utilizing data from the weight tensor and tensor mask. Finally, an accumulation circuit performs summation operations on the precision-adjusted intermediate tensor with data from an accumulate cache, generating an output tensor for the neural network computation.

Images