Graphics processor, constraint decoding method, and storage medium

Abstract

The present disclosure provides a graphics processor, a constraint decoding method and a storage medium. The graphics processor comprises: an on-chip memory storing a mask; the mask is used to indicate whether to output a corresponding token; a vector arithmetic logic unit configured to perform a probability calculation task of a current token; a load storage unit in communication connection with the on-chip memory and the vector arithmetic logic unit, configured to read the mask from the on-chip memory according to a read address within a time window in which the vector arithmetic logic unit performs the probability calculation task of the current token; and the vector arithmetic logic unit is further configured to filter the result of the probability calculation task of the current token by using the mask to obtain a target token. The present disclosure can eliminate PCIe communication delay, and further use the time-consuming token calculation to cover up the delay of reading the mask, improve the GPU utilization, and ensure the continuity of inference.

Images