Data quantization method and device, computer readable storage medium, chip

Abstract

A data quantization method and device, a computer readable storage medium and a chip, the data quantization method comprising: obtaining a data block to be quantized, dividing the data block to be quantized into one or more data groups to be quantized according to a preset grouping method; performing quantization calculation on each data group to be quantized according to a target quantization format, to obtain quantization data corresponding to data in each data group to be quantized and quantization metadata corresponding to each data group to be quantized, wherein the quantization metadata is used for dequantization of the quantization data; and storing the quantization metadata corresponding to the data block to be quantized and the quantization data as a continuous data structure.

Description

Technical Field

[0001] This article relates to, but is not limited to, the field of computer technology, and in particular to a data quantization method and apparatus, a computer-readable storage medium, and a chip. Background Technology

[0002] Large Language Models (LLMs) are a type of deep learning-based artificial intelligence model that is trained on large amounts of text data and is able to understand, generate, and reason about natural language.

[0003] Currently, most large language models in the industry are based on the Transformer model. Matrix multiplication is the core operation for attention calculation when performing inference using a Transformer-based large language model. For example, during attention calculation, the query vector, key vector, and value vector can be calculated using the following formula (1):

[0004] Result[M][N]=Activation[M][L]×Weight[L][N] (1).

[0005] Wherein, the Result matrix is ​​the calculation result matrix, the Activation matrix is ​​the input activation value matrix, the Weight matrix is ​​the model weight parameter matrix, M is the batch size or sequence length, L is the feature dimension, and N is the output feature dimension. Summary of the Invention

[0006] The following is an overview of the subject matter described in detail herein. This overview is not intended to limit the scope of the claims.

[0007] This disclosure provides a data quantization method, including:

[0008] Obtain the data block to be quantized, and divide the data block to be quantized into one or more data groups to be quantized according to a preset grouping method;

[0009] According to the target quantization format, each of the data groups to be quantized is quantized to obtain the quantized data corresponding to the data in each data group to be quantized and the quantization metadata corresponding to each data group to be quantized. The quantization metadata is used to dequantize the quantized data.

[0010] The quantization metadata corresponding to the data block to be quantized and the quantization data are stored as a continuous data structure.

[0011] This disclosure also provides a data quantization apparatus, including a memory; and a processor connected to the memory, the memory being used to store instructions, the processor being configured to perform steps of the data quantization method as described in any embodiment of this disclosure based on the instructions stored in the memory.

[0012] This disclosure also provides a chip including a data quantization device as described in any embodiment of this disclosure.

[0013] This disclosure also provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the data quantization method as described in any embodiment of this disclosure.

[0014] This disclosure also provides a computer program product that, when run on a computer, causes the computer to perform a data quantization method as described in any embodiment of this disclosure.

[0015] The data quantization method, apparatus, computer-readable storage medium, and chip of this disclosure acquire a data block to be quantized, divide the data block into one or more data groups to be quantized according to a preset grouping method, and perform quantization calculations on each data group to be quantized according to a target quantization format to obtain quantized data corresponding to the data in each data group to be quantized and quantized metadata corresponding to each data group to be quantized. The quantized metadata and quantized data corresponding to the data block to be quantized are stored as a continuous data structure, which can significantly reduce the storage and inference calculation overhead of large language models, thereby reducing the model's requirements for memory capacity and bandwidth.

[0016] Other features and advantages of this application will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the application. Other advantages of this application can be realized and obtained by means of the solutions described in the description and the accompanying drawings. Attached Figure Description

[0017] The accompanying drawings are used to provide an understanding of the technical solutions of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the technical solutions of this disclosure and do not constitute a limitation on the technical solutions of this disclosure.

[0018] Figure 1 This is a flowchart illustrating a data quantization method as an exemplary embodiment of the present disclosure.

[0019] Figure 2 This is a schematic diagram of a data quantization process as an exemplary embodiment of the present disclosure.

[0020] Figure 3A This is a schematic diagram of the storage structure for quantized data and quantized metadata of uint4.

[0021] Figure 3B This is a schematic diagram of the storage structure for uint8 quantized data and quantized metadata.

[0022] Figure 4 A schematic diagram of a data dequantization process provided for an exemplary embodiment of this disclosure.

[0023] Figure 5 This is a schematic diagram of the structure of a data quantization device as an exemplary embodiment of the present disclosure. Detailed Implementation

[0024] This application describes several embodiments, but these descriptions are exemplary and not restrictive, and it will be apparent to those skilled in the art that many more embodiments and implementations are possible within the scope of the embodiments described herein. Although many possible combinations of features are shown in the drawings and discussed in the detailed description, many other combinations of the disclosed features are also possible. Unless specifically limited, any feature or element of any embodiment may be used in combination with, or may replace, any feature or element of any other embodiment.

[0025] This application includes and contemplates combinations of features and elements known to those skilled in the art. The embodiments, features, and elements disclosed in this application may also be combined with any conventional features or elements to form a unique inventive scheme as defined by the claims. Any feature or element of any embodiment may also be combined with features or elements from other inventive schemes to form another unique inventive scheme as defined by the claims. Therefore, it should be understood that any feature shown and / or discussed in this application may be implemented individually or in any suitable combination. Therefore, the embodiments are not limited except by the limitations imposed by the appended claims and their equivalents. Furthermore, various modifications and changes may be made within the scope of the appended claims.

[0026] Furthermore, in describing representative embodiments, the specification may have presented methods and / or processes as a specific sequence of steps. However, the method or process should not be limited to the specific order of steps described herein, to the extent that it does not depend on such a specific order. As will be understood by those skilled in the art, other sequences of steps are also possible. Therefore, the specific order of steps set forth in the specification should not be construed as a limitation of the claims. Moreover, the claims concerning the method and / or process should not be limited to the steps performed in the written order, and those skilled in the art will readily understand that these orders can be varied and still remain within the spirit and scope of the embodiments of this application.

[0027] Unless otherwise defined, the technical or scientific terms used in the embodiments of this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms "first," "second," and similar terms used in the embodiments of this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" indicate that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, but do not exclude other elements or objects.

[0028] like Figure 1 As shown in the embodiments of this disclosure, a data quantization method is provided, including:

[0029] Step 101: Obtain the data block to be quantized and divide it into one or more data groups to be quantized according to the preset grouping method;

[0030] Step 102: Perform quantization calculations on each data group to be quantized according to the target quantization format to obtain the quantized data corresponding to the data in each data group to be quantized and the quantization metadata corresponding to each data group to be quantized. The quantization metadata is used to dequantize the quantized data.

[0031] Step 103: Store the quantization metadata and quantization data corresponding to the data block to be quantized as a continuous data structure.

[0032] The data quantization method of this disclosure involves acquiring a data block to be quantized, dividing the data block into one or more data groups according to a preset grouping method, and performing quantization calculations on each data group according to the target quantization format to obtain quantized data corresponding to the data in each data group and quantization metadata corresponding to each data group. The quantization metadata and quantization data corresponding to the data block to be quantized are stored as a continuous data structure, which can significantly reduce the storage and inference computation overhead of large language models, thereby reducing the model's requirements for memory capacity and bandwidth.

[0033] In some exemplary embodiments, the data block to be quantized can be a weight parameter matrix of a large language model, such as a query matrix W_Q, a key matrix W_K, or a value matrix W_V. However, this disclosure does not limit this.

[0034] During model inference, the weight parameter matrix consists of fixed parameters, while the activation value matrix is ​​a dynamically calculated intermediate result. Therefore, quantization techniques can be applied to the fixed weight parameter matrix to convert it from high-precision floating-point numbers (such as FP32) to low-precision formats (such as INT8, INT4, FP8, etc.), thereby reducing model storage and computational overhead and improving inference efficiency.

[0035] In some exemplary embodiments, the data block to be quantized can be a 32*32 or 64*64 data matrix. However, this disclosure does not limit it in this regard.

[0036] The data quantization method of this disclosure divides the weight parameter matrix of a large language model into multiple data blocks to be quantized. Each data block to be quantized is a small piece of data within the weight parameter matrix, reducing the memory usage during the quantization process. Furthermore, by designing each data block to be quantized to a size of 32*32 or 64*64, this disclosure better preserves the local feature information of the weight matrix during quantization, achieving a better balance between accuracy loss and compression ratio compared to larger or smaller blocks.

[0037] In this embodiment of the disclosure, the preset grouping method can be any one of the following: grouping by row, grouping by column, and grouping by block.

[0038] For example, taking a 32*32 data matrix as an example, when grouping by row, a data block to be quantized is divided into 32 data groups, each data group including one row of data in the data block; when grouping by column, a data block to be quantized is divided into 32 data groups, each data group including one column of data in the data block; when grouping by block, the data block to be quantized is directly treated as one data group (that is, one data block to be quantized is divided into 1 data group).

[0039] The data quantization method of this disclosure can be applied to various types of target quantization formats. For example, the target quantization format can be any of the following: unsigned integer quantization, signed integer quantization, general floating-point quantization, and micro-scaling floating-point quantization, etc.

[0040] Unsigned integer quantization refers to the process of converting floating-point data (e.g., fp32) to unsigned integer data (e.g., uint8) fp32. Signed integer quantization refers to the process of converting floating-point data (e.g., fp32) to signed integer data (e.g., int8). General floating-point quantization refers to the process of converting high-precision floating-point numbers (e.g., fp32) to low-precision floating-point numbers (e.g., fp16). Microscaling floating-point quantization refers to the process of converting high-precision floating-point numbers (e.g., fp32) to microscaling data format floating-point numbers (e.g., mxfp8). Microscaling data formats are efficient data representation methods designed specifically for deep learning, aiming to reduce computational and storage costs by reducing data bit width while maintaining model accuracy. It consists of three core components: scaling factor, elements, and scaling block size. For details, please refer to the paper "Microscaling Data Formats for Deep Learning" published on October 6, 2024 by researchers such as Bita Darvish Rouhani and Ritchie Zhao.

[0041] For example, in step 102, when the target quantization format is unsigned integer quantization, quantization calculation is performed on each group of data to be quantized, including:

[0042] For each set of data to be quantified, perform the following operations:

[0043] Determine the maximum value x of the data to be quantized in the data set to be quantized. max and minimum value x min And the maximum value q corresponding to the target quantization format. max and minimum value q min ;

[0044] Calculate the scaling factor s and the median z using the following formulas: s = (x max -x min ) / (q max -q min z = -x min / s+q min ;

[0045] Each data point in the dataset to be quantized is quantized according to the following formula:

[0046] For unsigned integer quantization, for each set of data to be quantized (e.g., a block, row, or column of the weight parameter matrix to be quantized), find the maximum value x of the data to be quantized. maxand minimum value x min And the maximum value q corresponding to the target quantization format. max and minimum value q min For example, for int4 symmetric quantization, q max and q min q are 7 and -7 respectively, for int8 symmetric quantization. max and q min 127 and -127 respectively, for uint4 asymmetric quantization, q max and q min q are 0 and 15 respectively, for uint4 asymmetric quantization. max and q min They are 0 and 255 respectively; however, to calculate the scaling factor s and the median z: s = (x max -x min ) / (q max -q min z = -x min / s+q min And quantize each data point in the dataset according to the scaling factor s and the median z: Here, operator [x] rounds the floating-point number to the nearest integer value; the function clamp(x; n, p) outputs the integer x. int The values ​​are limited to [n, p]. The above quantization process can quantize the data to be quantized in each data set, such as m floating-point numbers, into m integers, one floating-point number s, and one floating-point number z.

[0047] For example, suppose we perform asymmetric quantization (uint4) on the four numbers [-0.6, 0.1, 0.4, 0.9] (this is just an example to illustrate the quantization process; in reality, a set of data to be quantized may contain 32 or 64 values, etc.). max =2 4 -1 = 15, q min =0, x max -x min =0.9-(-0.6)=1.5, s=1.5 / (15-0)=0.1, then z=(0.6 / 0.1+0)=6, the corresponding data after quantization of -0.6 is: (-0.6 / 0.1+6)=0, the data after quantization of 0.1 is: (0.1 / 0.1+6)=7, the data after quantization of 0.4 is: (0.4 / 0.1+6)=10, and the data after quantization of 0.9 is: (0.9 / 0.1+6)=15.

[0048] In large language model inference, matrix calculations are required using a weight parameter matrix. This involves first dequantizing the quantized data to convert the weight coefficients into high-precision floating-point numbers before performing matrix calculations. The dequantization formula is: weight = s·weight int -z, weight is the dequantized value, weight int This represents the quantized value. This saves memory and bandwidth while enabling higher matrix computation density.

[0049] The following uses a 32*32 data block to be quantized as an example to illustrate the quantization calculation process of this embodiment in detail:

[0050] First, the data block to be quantized is loaded into memory. In this embodiment, it is assumed that the data block to be quantized is a 32×32 data matrix as the smallest unit.

[0051] Next, row quantization, column quantization, or whole-block quantization is performed on the 32×32 data block. Before the quantization calculation, the maximum value q corresponding to the target quantization format is determined. max and minimum value q min For example, when uint4, q min =0 and q max =15; when uint8, q min =0 and q max =255; when int4, q min =-8 and q max =7; when int8, q min =-128 and q max =127.

[0052] like Figure 2 As shown, for row quantization, the quantization calculation process includes:

[0053] (1) Find the maximum value x in each row of the 32×32 data matrix. max and minimum value x min ;

[0054] (2) Calculate the scaling factor s and median z for each row:

[0055] (3) Calculate the reciprocal of s, sr;

[0056] (4) Calculate [x·sr+z] for each element in each row based on its own s and z values, and convert the result from floating point to integer;

[0057] (5) Limit the result to [q] min ,q max ]Inside.

[0058] For column quantization, the quantization calculation process includes:

[0059] (1) Find the maximum value x in each column of a 32×32 data matrix. max and minimum value x min ;

[0060] (2) Calculate the scaling factor s and median z for each column:

[0061] (3) Calculate the reciprocal of s, sr;

[0062] (4) Calculate [x·sr+z] for each element in each column based on its own s and z values, and convert the result from floating point to integer;

[0063] (5) Limit the result to [q] min ,q max ]Inside.

[0064] For block quantization, the quantization calculation process includes:

[0065] (1) Find the maximum value x in each row of the 32×32 data matrix. max1 and minimum value x min ;

[0066] (2) Among the 32 maximum values ​​x max1 And 32 minimum values ​​x min1 Find the maximum value x in the middle. max and minimum value x min ;

[0067] (3) Calculate the scaling factor s and median z of the entire block:

[0068] (4) Calculate the reciprocal of s, sr;

[0069] (5) Calculate [x·sr+z] for all elements based on the s and z values, and convert the result from floating point to integer;

[0070] (6) Limit the result to [q] min ,q max ]Inside.

[0071] For row quantization and column quantization, 32 scaling factors *s*, 32 medians *z*, and 1024 quantized data points are formed into a continuous data structure. For block quantization, to ensure compatibility with row and column quantization, a scaling factor *s* and a median *z* can be copied into 32 identical scaling factors *s* and 32 identical medians *z*. That is, the data structure corresponding to block quantization is the same as the data structure corresponding to row or column quantization. For example, *uint4* corresponds to the data structure after block quantization of the data to be quantized, as shown below. Figure 3AAs shown, uint8 corresponds to the quantized data structure of a block of data to be quantized, as follows: Figure 3B As shown.

[0072] In this embodiment of the disclosure, when the target quantization format is unsigned integer quantization, the quantization metadata includes a scaling factor and a median. The continuous data structure includes a scaling factor array, a median array, and a quantization data matrix. The scaling factor array stores the scaling factors corresponding to one or more data groups to be quantized, the median array stores the medians corresponding to one or more data groups to be quantized, and the quantization data matrix stores the quantized data corresponding to the data blocks to be quantized. For unsigned integer quantization, the scaling factor s can be stored as a floating-point number in fp8 e8m0 format, while the quantized data can be stored as a floating-point number in fpx (e.g., fp8 / 6 / 4, etc.) format.

[0073] In this embodiment of the disclosure, for block quantization, i.e., when the preset grouping method is block grouping, the scaling factor array may include m identical scaling factors, and the median array may include m identical medians, where m is the number of data rows or columns of the data block to be quantized. For example, m = 32.

[0074] like Figure 4 As shown, during large model inference, the device loads the quantized data structure into memory and performs the quantization according to the following formula: weight = s·weight int -z performs dequantization to form a 32×32 floating-point matrix, which is part of the weight parameter matrix. Figure 4 In the middle, the multiplier is used to implement the relationship between s and weight. int The multiplication operation is performed, and the subtractor is used to implement s·weight. int The multiplication operation between z and wi[i][j] is the element in the i-th row and j-th column of the quantized matrix, and Weight[i][j] is the element in the i-th row and j-th column of the dequantized matrix.

[0075] For signed integer quantization, general floating-point quantization, and micro-scaling floating-point quantization, the median z can be considered as 0. That is, it is not necessary to calculate the median z; only the scaling factor s needs to be calculated.

[0076] For example, in step 102, when the target quantization format is signed integer quantization, quantization calculation is performed on each group of data to be quantized, including:

[0077] For each set of data to be quantified, perform the following operations:

[0078] Determine the maximum value x of the data to be quantized in the data set to be quantized. max and minimum value x minAnd the maximum value q corresponding to the target quantization format. max and minimum value q min ;

[0079] Calculate the scaling factor s using the following formula: s = (x max -x min ) / (q max -qmin);

[0080] Each data point in the dataset to be quantized is quantized according to the following formula:

[0081] For example, in step 102, when the target quantization format is general floating-point quantization, quantization calculation is performed on each group of data to be quantized, including:

[0082] For each set of data to be quantified, perform the following operations:

[0083] Determine the absolute value x of the data with the largest absolute value in the data set to be quantified. amax And the range of values ​​corresponding to the target quantization format;

[0084] Calculate the scaling factor s using the following formula: s = x amax / range;

[0085] Each data point in the dataset to be quantized is quantized according to the following formula:

[0086] BF16 (Brain Floating Point) is a 16-bit floating-point format designed specifically for deep learning. Proposed by Google, it is widely used in AI accelerators such as TPUs. It sacrifices some precision (7 mantissas) to achieve the same numerical range (8 exponents) as FP32, thus effectively avoiding numerical overflow problems during training. This refers to converting the value corresponding to bf16 to the value corresponding to fpx (such as fp8 / 6 / 4). For unsigned integer quantization, signed integer quantization, and general floating-point quantization, the scaling factor s can be stored as a floating-point number in BF16 format, while the quantized data can be stored as a floating-point number in the corresponding target quantization format, such as uintx (such as uint4), intx (such as int4), or fpx (such as fp8 / 6 / 4).

[0087] In this embodiment of the disclosure, x amaxThis determines the dynamic range of the data. The range corresponding to the target quantization format is the range of the floating-point format to be quantized. For example, for fp8 e4m3 fn, the range is 448.0; for fp8e5m2, the range is 57344.0; for fp6 e3m2, the range is 28.0; for fp6 e2m3, the range is 7.5; and for fp4 e2m1, the range is 6.0.

[0088] For example, in step 102, quantization calculations are performed on each group of data to be quantized, including:

[0089] For each set of data to be quantified, perform the following operations:

[0090] Determine the absolute value x of the data with the largest absolute value in the data set to be quantified. amax And the range of values ​​corresponding to the target quantization format;

[0091] Calculate the scaling factor s using the following formula: s = xamax / range;

[0092] Each data point in the dataset to be quantized is quantized according to the following formula: in, Indicates will Converting from a high-precision data format to a low-precision data format, such as from fp32 / fp16 to unit8 / uint4 / int8 / int4 (i.e., integer quantization), or from fp32 / bf16 to fp8 / fp6 / fp4 (i.e., floating-point quantization), etc.

[0093] For example, in step 102, when the target quantization format is micro-scaling floating-point quantization, quantization calculations are performed on each group of data to be quantized, including:

[0094] For each set of data to be quantified, perform the following operations:

[0095] Determine the maximum absolute value x of the data to be quantized in the data set to be quantized. amax , will x amax Convert the value to fp8 e8m0 format, and use the converted value as the scaling factor s corresponding to the data group to be quantized;

[0096] Each data point in the dataset to be quantized is quantized according to the following formula:

[0097] Among them, fp8 e8m0 format is an 8-bit floating-point number format with a value range of [-2]. 7 ,2 7The value is defined as [-128, 128]. It consists of 1 sign bit, 8 exponent bits, and 0 mantissa bits, therefore it has no fractional part and all values ​​are integers. In this embodiment, x... amax The value converted to fp8 e8m0 format is used as a scaling factor to dynamically adjust the numerical range of low-precision data (such as fp8 / 6 / 4, etc.) to prevent overflow or underflow issues in AI calculations. That is, for micro-scaling floating-point quantization, the scaling factor s can be stored as a floating-point number in fp8 e8m0 format, while the quantized data can be stored as a floating-point number in fpx (such as fp8 / 6 / 4, etc.) format.

[0098] In this embodiment of the disclosure, when the target quantization format is any one of signed integer quantization, general floating-point quantization, and micro-scaling floating-point quantization, the quantization metadata includes scaling factors, and the continuous data structure includes a scaling factor array and a quantization data matrix; wherein, the scaling factor array is used to store the scaling factors corresponding to one or more data groups to be quantized, and the quantization data matrix is ​​used to store the quantized data corresponding to the data blocks to be quantized. During inverse quantization calculation, the inverse quantization calculation formula is: weight = s·weight q Where, weight is the dequantized value, weight q This is the quantized value.

[0099] In this embodiment of the disclosure, the continuous data structure can be represented as:

[0100] Struct quantBlock

[0101] {

[0102] Byte reserve[M]; / / Stores s and z

[0103] Type element

[32]

[32] ; / / Stores quantized data

[0104] }

[0105] For example, for symmetric row quantization of int4, the above data structure becomes:

[0106] Struct quantBlock_int4_sym

[0107] {

[0108] Bf16s

[32] ; / / Stores s, without z, each s corresponds to a row of data elements.

[0109] Int4 element

[32]

[32] ; / / Stores quantized data

[0110] }

[0111] For symmetric row quantization of uint8, the above data structure becomes:

[0112] Struct quantBlock_uint8_asym

[0113] {

[0114] Bf16 s

[32] ; / / Stores s, each s corresponds to a column of data elements

[0115] Bf16 z

[32] ; / / Stores z, each z corresponds to a column of data elements.

[0116] Uint8 element

[32]

[32] ; / / Stores quantized data

[0117] }

[0118] For block quantization of fp6e3m2, the above data structure becomes:

[0119] Struct quantBlock_fp6e3m2

[0120] {

[0121] Bf16 s

[32] ; / / Stores s, each s is the same, corresponding to all data elements

[0122] fp6e3m2element

[32]

[32] ; / / Stores quantized data

[0123] }

[0124] For block quantization of mxfp4 e1m2, the above data structure becomes:

[0125] Struct quantBlock_fp4e1m2

[0126] {

[0127] Fp8 e8m0 s

[32] ; / / Stores s, each s is the same, corresponding to all data elements.

[0128] fp4e1m2element

[32]

[32] ; / / Stores quantized data

[0129] }

[0130] This disclosure also provides a data quantization apparatus, including a memory; and a processor connected to the memory, the memory being used to store instructions, the processor being configured to perform steps of the data quantization method as described in any embodiment of this disclosure based on the instructions stored in the memory.

[0131] like Figure 5 As shown, in one example, the data quantization device may include: a processor 510, a memory 520, a bus system 530, and a transceiver 540. The processor 510, memory 520, and transceiver 540 are connected via the bus system 530. The memory 520 stores instructions, and the processor 510 executes the instructions stored in the memory 520 to control the transceiver 540 to transmit and receive signals. Specifically, the transceiver 540, under the control of the processor 510, acquires a block of data to be quantized. The processor 510 divides the block of data to be quantized into one or more groups of data to be quantized according to a preset grouping method; performs quantization calculations on each group of data to be quantized according to a target quantization format to obtain quantized data corresponding to the data in each group of data to be quantized and quantized metadata corresponding to each group of data to be quantized; and stores the quantized metadata corresponding to the block of data to be quantized and the quantized data in a continuous data structure.

[0132] It should be understood that processor 510 can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0133] Memory 520 may include read-only memory and random access memory, and provides instructions and data to processor 510. A portion of memory 520 may also include non-volatile random access memory. For example, memory 520 may also store device type information.

[0134] In addition to a data bus, the bus system 530 may also include a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 5 The general labeled all buses as Bus System 530.

[0135] In implementation, the processing performed by the processing device can be accomplished through integrated logic circuits in the hardware of the processor 510 or through software instructions. That is, the method steps of this embodiment can be executed by a hardware processor, or by a combination of hardware and software modules within the processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other storage media. This storage medium is located in memory 520, and the processor 510 reads information from memory 520 and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, further details are omitted here.

[0136] This disclosure also provides a chip including a data quantization device as described in any embodiment of this disclosure.

[0137] The chip in this disclosure embodiment can be any kind of chip logic architecture, such as a central processing unit (CPU) chip, a graphics processing unit (GPU) chip, a field-programmable gate array (FPGA) chip, an application-specific integrated circuit (ASIC) chip, a tensor processing unit (TPU) chip, or other artificial intelligence (AI) chips.

[0138] The chip in this embodiment can be connected to memory for data interaction and instruction transmission with memory.

[0139] This disclosure also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the data quantization method as described in any embodiment of this disclosure. The method of driving floating-point arithmetic by executing executable instructions is essentially the same as the data quantization method provided in the above embodiments of this disclosure, and will not be described in detail here.

[0140] In some possible implementations, various aspects of the data quantization method provided in this disclosure may also be implemented as a program product comprising program code that, when run on a computer device, causes the computer device to perform the steps of the data quantization method according to various exemplary embodiments of this disclosure as described above. For example, the computer device may execute the data quantization method described in the embodiments of this disclosure.

[0141] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0142] It will be understood by those skilled in the art that all or some of the steps, systems, or apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0143] It should be noted that the above embodiments or implementation methods are merely exemplary and not restrictive. Therefore, this disclosure is not limited to the content specifically shown and described herein. Various modifications, substitutions, or omissions can be made to the form and details of the implementations without departing from the scope of this disclosure.

Claims

1. A data quantification method, comprising: Obtain the data block to be quantized, and divide the data block to be quantized into one or more data groups to be quantized according to a preset grouping method; According to the target quantization format, each of the data groups to be quantized is quantized to obtain the quantized data corresponding to the data in each data group to be quantized and the quantization metadata corresponding to each data group to be quantized. The quantization metadata is used to dequantize the quantized data. The quantization metadata corresponding to the data block to be quantized and the quantization data are stored as a continuous data structure.

2. The method according to claim 1, wherein, The preset grouping method is any one of the following: grouping by row, grouping by column, and grouping by block.

3. The method according to claim 1, wherein, The target quantization format is any one of the following: unsigned integer quantization, signed integer quantization, and floating-point quantization.

4. The method according to claim 3, wherein, When the target quantization format is unsigned integer quantization, the quantization metadata includes a scaling factor and a median, and the data structure includes a scaling factor array, a median array, and a quantization data matrix. The scaling factor array is used to store scaling factors corresponding to one or more of the data groups to be quantized, the median array is used to store medians corresponding to one or more of the data groups to be quantized, and the quantization data matrix is ​​used to store quantized data corresponding to the data in the data block to be quantized.

5. The method according to claim 4, wherein, When the preset grouping method is block grouping, the scaling factor array includes m identical scaling factors, and the median array includes m identical medians, where m is the number of data rows or columns of the data block to be quantized.

6. The method according to claim 3, wherein, When the target quantization format is either signed integer quantization or floating-point quantization, the quantization metadata includes a scaling factor, and the data structure includes a scaling factor array and a quantization data matrix. The scaling factor array is used to store scaling factors corresponding to one or more of the data groups to be quantized, and the quantization data matrix is ​​used to store quantized data corresponding to the data in the data blocks to be quantized.

7. A data quantization apparatus, comprising a memory; and a processor connected to the memory, the memory being configured to store instructions, the processor being configured to perform the steps of the data quantization method as claimed in any one of claims 1 to 6 based on the instructions stored in the memory.

8. A chip comprising the data quantization device as described in claim 7.

9. A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the data quantization method as described in any one of claims 1 to 6.

10. A computer program product, when run on a computer, causes the computer to perform the data quantization method as described in any one of claims 1 to 6.

Images