1 results about "Register based" patented technology

The application discloses a large language model weight compression method, an inference operation method, a system and a medium, and belongs to the technical field of large language model weight compression and inference operation optimization. The method comprises the following steps: acquiring a weight matrix of a large language model and constructing an exponential high-frequency window and a benchmark value, classifying and encoding each weight data into a fixed-length code word and splitting the fixed-length code word into an independent bitmap, synchronously generating a compact value stream and a rollback value stream, and packing the compact value stream and the rollback value stream into weight compression data; loading the weight compression data, generating a state mask and a channel prefix mask in a register based on the bitmap, calculating a reading offset by using a population count instruction, reading a compressed data segment from a corresponding data stream, reconstructing weight data in the register in combination with the code word, and finally directly inputting the weight data into a matrix multiplication unit for operation; therefore, by implementing the application, the problem of efficiency reduction caused by control flow divergence and redundant memory access in GPU inference of existing variable-length encoding can be solved, and the model inference efficiency under lossless compression can be improved.