Data quantification method and device, computer readable storage medium and chip

By grouping, quantizing, and storing the data blocks of a large language model into a continuous data structure, the problem of excessive storage and computational overhead of large language models is solved, achieving more efficient memory utilization and computational performance.

CN122180968APending Publication Date: 2026-06-09SUNMMIO SCIENCE & TECHNOLOGY (BEIJING) CO LTD
0 Cites 0 Cited by

Patent Information

Application Number
CN202680000112.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-01-29
Publication Date
2026-06-09

AI Technical Summary

Technical Problem

Large language models suffer from excessive storage and computational overhead during inference, especially in matrix multiplication operations, which leads to excessive memory and bandwidth requirements.

Method used

By dividing the data block to be quantized into multiple data groups according to a preset grouping method, and performing quantization calculations on each data group according to the target quantization format, quantized data and metadata are obtained and stored as a continuous data structure, reducing the storage and inference calculation overhead of the model.

Benefits of technology

It significantly reduces the storage and inference computation overhead of large language models, reduces the demand for memory capacity and bandwidth, and improves inference efficiency.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The invention discloses a data quantization method and device, a computer readable storage medium and a chip, and the data quantization method comprises the steps: obtaining a to-be-quantized data block, and dividing the to-be-quantized data block into one or more to-be-quantized data groups according to a preset grouping method; performing quantization calculation on each to-be-quantized data group according to a target quantization format to obtain quantized data corresponding to data in each to-be-quantized data group and quantized metadata corresponding to each to-be-quantized data group, the quantized metadata being used for performing inverse quantization on the quantized data; and storing the quantized metadata corresponding to the to-be-quantized data block and the quantized data as a continuous data structure body.
Need to check novelty before this filing date? Find Prior Art