Model training and reasoning method, device, equipment and storage medium based on compression weight

Abstract

The present application discloses a model training and reasoning method, device, equipment and storage medium based on compressed weights, relates to the technical field of model training and reasoning, and discloses a model training and reasoning method based on compressed weights, including: obtaining an initial weight matrix of a preset model; obtaining an initial weight matrix block according to the initial weight matrix, and obtaining an initial weight according to the initial weight matrix block; obtaining a weight exponent bit and a weight mantissa bit according to the initial weight, and obtaining a mantissa retention value according to the weight exponent bit; compressing the initial weight according to the weight exponent bit, the mantissa retention value and the weight mantissa bit to obtain a compressed weight; when forward propagation or reverse propagation of a preset model is detected, decompressing the compressed weight to obtain a target weight, and training and reasoning the preset model according to the target weight. By compressing and decompressing the weights by the exponent bit and the mantissa retention value of the model weights, it is possible to reduce memory usage while maintaining model performance.

Images