Data quantization circuit based on lookup table, data processing method, processor, device and medium

Abstract

The present disclosure provides a data quantization circuit based on a lookup table, a data processing method, a processor, an apparatus and a medium. The data quantization circuit comprises: a data block preprocessing unit configured to determine target input data with the largest exponential part in an input data block; an exponential alignment unit configured to align the exponential parts of a plurality of input data to an exponential value according to the exponential value of the exponential part of the target input data, to obtain a plurality of intermediate data; an index generation unit configured to extract the mantissa part of the plurality of intermediate data, and combine it with N scaling factor mantissa part candidates to obtain N first indexes corresponding to each input data; and an output encoding unit configured to access a lookup table according to the N first indexes corresponding to each input data, to determine a target scaling factor and a quantization result of the input data block. The present disclosure dynamically selects the optimal scaling factor through the lookup table, ensures high quantization accuracy, reduces implementation cost and delay, and improves hardware efficiency.

Description

Technical Field

[0001] The embodiments of this disclosure are in the field of artificial intelligence chips, specifically relating to a data quantization circuit based on a lookup table, a data processing method, a processor, an electronic device, and a storage medium. Background Technology

[0002] With the increasing inference demands of large-scale artificial intelligence (AI) models, stringent requirements have been placed on the storage capacity, memory bandwidth, and computational efficiency of AI chips. Low-precision quantization technology, which effectively alleviates these pressures, has become a core technology in current AI chip design. Specifically, low-precision data formats such as FP4 and FP8 can significantly reduce the storage requirements and memory bandwidth consumption of large models, and hardware accelerators are typically 2-8 times more efficient at computationally processing these low-precision data formats than high-precision formats such as FP16 and FP32. Therefore, low-precision quantization technology has significant advantages in balancing model performance and hardware cost and is widely used in AI inference scenarios.

[0003] However, traditional low-precision quantization techniques still face many challenges in hardware implementation. For example, real-time computation using scaling factors can affect inference speed; the data quantization process involves a loop of "dequantization-computation-requantization," which significantly increases inference latency; the representation range of low-precision data formats is limited, making it difficult to maintain the required inference accuracy of the model while ensuring the data representation range; and the selection of multiple candidate scaling factors introduces additional computational overhead, further limiting the improvement of hardware inference performance. Summary of the Invention

[0004] At least one embodiment of this disclosure provides a lookup table-based data quantization circuit. This circuit determines a target scaling factor for quantizing an input data block to a target precision and the quantization result of the input data block. The data quantization circuit includes: a data block preprocessing unit configured to determine target input data with the largest exponent portion in the input data block, wherein the input data block includes multiple input data; an exponent alignment unit configured to align the exponent portions of the multiple input data to the exponent value based on the exponent value of the exponent portion of the target input data, obtaining multiple intermediate data corresponding to the multiple input data; and an index generation unit configured to extract the mantissa portion of the multiple intermediate data and combine it with N scaling factor mantissa portion candidates to obtain N first indexes corresponding to each input data. An index is defined, where N is a positive integer determined based on the number of bits in the mantissa candidates of the scaling factor; a lookup table is configured to pre-store multiple data items and multiple index items in a one-to-one correspondence, wherein the multiple index items are composed of T input mantissa candidates and N scaling factor mantissa candidates, where T is a positive integer determined based on the number of bits in the mantissa of the input data, and each data item includes the quantization error and quantization result of data quantization using the information in the corresponding index item; an output encoding unit is configured to access the lookup table according to the N first indices corresponding to each input data, and determine the target scaling factor and the quantization result of the input data block according to the N query results corresponding to each of the multiple input data, wherein each query result includes the quantization error and quantization result corresponding to each first index.

[0005] For example, in the data quantization circuit provided in at least one embodiment of this disclosure, each index item is composed of one input mantissa candidate and one scaling factor mantissa candidate. The data item corresponding to the index item includes a first quantization error obtained by quantizing the first data using a first scaling factor candidate. The first scaling factor candidate uses one scaling factor mantissa candidate in the index item as its mantissa. The first data uses one input mantissa candidate in the index item as its mantissa. The precision format of the first scaling factor is the same as the precision format of the target scaling factor, and the precision format of the first data is the same as the precision format of the multiple input data.

[0006] For example, in the data quantization circuit provided in at least one embodiment of this disclosure, each data item further includes a first quantization result obtained by quantizing the first data using a first scaling factor, wherein the floating-point precision of the first quantization result is the target precision.

[0007] For example, in the data quantization circuit provided in at least one embodiment of this disclosure, the target precision includes at least one precision format, and the first quantization result is obtained through the following steps: converting the mantissa of the first data into first floating-point data, and converting the mantissa of the first scaling factor into second floating-point data; determining a first value according to the precision format adopted by the first quantization result, wherein at least one precision format corresponds to a different first value, and the first value is the least power of 2 greater than the maximum value in the numerical expression range corresponding to the precision format; performing a multiplication operation on the first value and the ratio of the first floating-point data to the second floating-point data to obtain a multiplication operation result; and determining the value closest to the multiplication operation result as the first quantization result within the numerical expression range corresponding to the precision format adopted by the first quantization result.

[0008] For example, in the data quantization circuit provided in at least one embodiment of this disclosure, the first quantization error is obtained through the following steps: performing an inverse quantization operation on the first quantization result based on the first numerical value and the second floating-point data to obtain the first inverse quantization result; and determining the first quantization error corresponding to the first data based on the first inverse quantization result and the mantissa portion of the first data.

[0009] For example, in the data quantization circuit provided in at least one embodiment of this disclosure, the data block preprocessing unit includes a tree comparator network, which is configured to: extract the exponential portions of multiple input data and determine the largest exponential portion among the exponential portions of the multiple input data, so as to use the input data with the largest exponential portion as the target input data.

[0010] For example, in the data quantization circuit provided in at least one embodiment of this disclosure, the exponent alignment unit includes a subtractor and a shifter. The subtractor is configured to: determine the difference between the exponent portion and the maximum exponent portion of each of the plurality of input data; the shifter is configured to: shift the mantissa portion of the plurality of input data to the right by the number of bits indicated by the difference, so as to align the exponent portions of the plurality of input data to the exponent value and obtain a plurality of intermediate data corresponding to the plurality of input data respectively.

[0011] For example, in the data quantization circuit provided in at least one embodiment of this disclosure, when the output encoding unit determines the target scaling factor based on N query results corresponding to each of the multiple input data, it includes the following operations: determining the mantissa of the target scaling factor based on the N query results corresponding to each of the multiple input data; shifting the exponent value left by p bits to obtain the exponent of the target scaling factor, wherein the maximum value in the numerical expression range corresponding to the precision format adopted by the target precision is less than 2. p And greater than 2 p-1 p is a positive integer.

[0012] For example, in the data quantization circuit provided in at least one embodiment of this disclosure, when the output encoding unit determines the mantissa of the target scaling factor based on the N query results corresponding to the multiple input data, it includes the following operations: dividing the quantization errors in the N query results corresponding to the multiple input data into N sets of quantization errors, wherein each set of quantization errors includes multiple quantization errors obtained by quantizing the multiple input data using the same scaling factor mantissa candidate; calculating the average value of the multiple quantization errors included in each set of quantization errors as the average quantization error corresponding to the scaling factor mantissa candidate; and selecting the scaling factor mantissa candidate with the smallest average quantization error among the N scaling factor mantissa candidates as the mantissa of the target scaling factor.

[0013] For example, in the data quantization circuit provided in at least one embodiment of this disclosure, when the output encoding unit performs the operation of determining the quantization result of the input data block, it includes performing the following operations: using the mantissa of the target scaling factor and the mantissa of each input data to form a second index; accessing a lookup table according to the second index corresponding to each of the multiple input data, and obtaining the second quantization result corresponding to each of the multiple input data from the multiple query results returned by the lookup table, as the quantization result of the input data block.

[0014] For example, in the data quantization circuit provided in at least one embodiment of this disclosure, the target precision includes at least one precision format, and the at least one precision format corresponds to different lookup tables. The data quantization circuit is also configured to select the corresponding lookup table according to the precision format adopted by the target precision.

[0015] At least one embodiment of this disclosure also provides a data processing method, which includes: determining target input data with the largest exponent portion in an input data block, wherein the input data block includes multiple input data; aligning the exponent portions of the multiple input data to the exponent value according to the exponent value of the exponent portion of the target input data to obtain multiple intermediate data corresponding to the multiple input data respectively; extracting the mantissa portion of the multiple intermediate data and combining it with N scaling factor mantissa portion candidates to obtain N first indices corresponding to each input data, wherein N is a positive integer and is determined based on the number of bits of the scaling factor mantissa portion candidates; and according to each input data... For each of the N first indices, a lookup table is accessed, and based on the N query results corresponding to the multiple input data, the target scaling factor and the quantization result of the input data block are determined. Each query result includes the quantization error and quantization result corresponding to each first index. The lookup table includes multiple data items and multiple index items that correspond one-to-one. The multiple index items are composed of T input mantissa candidates and N scaling factor mantissa candidates, where T is a positive integer and is determined based on the number of bits in the mantissa of the input data. Each data item includes the quantization error and quantization result of data quantization using the information in the corresponding index item.

[0016] For example, in at least one embodiment of the data processing method provided in this disclosure, determining a target scaling factor for quantizing an input data block to a target precision and a quantization result of the input data block based on N query results corresponding to each of the multiple input data includes: determining the mantissa of the target scaling factor based on the N query results corresponding to each of the multiple input data; shifting the exponent value p bits to the left to obtain the exponent of the target scaling factor, wherein the maximum value in the numerical expression range corresponding to the precision format adopted by the target precision is less than 2. p And greater than 2 p-1 p is a positive integer; the second index is formed by using the mantissa of the target scaling factor and the mantissa of each input data; and according to the second index corresponding to each of the multiple input data, the lookup table is accessed, and the second quantization result corresponding to each of the multiple input data is obtained from the multiple query results returned by the lookup table, which is used as the quantization result of the input data block.

[0017] At least one embodiment of this disclosure also provides a processor, which includes a plurality of computing units, wherein each computing unit is provided with a lookup table-based data quantization circuit provided in any embodiment of this disclosure.

[0018] At least one embodiment of this disclosure also provides an electronic device that includes a processor provided in any embodiment of this disclosure.

[0019] At least one embodiment of this disclosure also provides a computer-readable storage medium storing computer-readable instructions thereon, which, when executed by at least one processor, cause the at least one processor to perform the data processing method provided in any embodiment of this disclosure.

[0020] The data quantization circuit based on a lookup table provided in at least one embodiment of this disclosure can achieve efficient testing of multiple different scaling factors and selection of the optimal scaling factor. It uses a lookup table to implement the "quantization-dequantization-error calculation" operation, and determines the target scaling factor and the quantization result of the input data block based on the query results returned by the lookup table. The complete quantization process (including candidate scaling factor evaluation and quantization error calculation) is encoded into a unified lookup table, enabling all key operations (including candidate scaling factor evaluation) to be completed through a simple lookup table and basic logic gates. This simplifies the quantization process from a multi-step, complex calculation process to a single-cycle lookup operation, achieving a hardware-friendly quantization scheme and greatly reducing hardware complexity. This disclosure dynamically selects the optimal target scaling factor through a lookup table, ensuring high quantization accuracy. While maintaining high quantization accuracy, it significantly reduces implementation costs and latency, greatly improving hardware efficiency.

[0021] Furthermore, this disclosure innovatively provides an index structure that, through exponent alignment, allows the mantissas of all data to be directly compared and processed at the same precision. Thus, index entries only need to consider the mantissa portion, and the lookup table includes a limited number of index entries, minimizing the storage space occupied by the lookup table. Moreover, the data quantization circuit provided in this disclosure is adaptable to different precision formats and supports more candidate scaling factors (i.e., candidates for the mantissa portion of the scaling factor), improving hardware reusability and achieving better quantization precision while maintaining hardware efficiency. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments will be briefly described below. Obviously, the drawings described below only relate to some embodiments of this disclosure and are not intended to limit this disclosure.

[0023] Figure 1 A schematic diagram of the structure of a general-purpose graphics processor (GPGPU) is shown.

[0024] Figure 2 A schematic diagram of the structure of a processor provided in at least one embodiment of the present disclosure is shown.

[0025] Figure 3 A schematic diagram of a lookup table-based data quantization circuit provided in at least one embodiment of this disclosure is shown.

[0026] Figure 4A schematic diagram illustrating the process of a data quantization circuit performing data quantization according to at least one embodiment of the present disclosure is shown.

[0027] Figure 5 A schematic flowchart of a data processing method provided in at least one embodiment of this disclosure is shown.

[0028] Figure 6 A schematic diagram of an electronic device provided in at least one embodiment of the present disclosure is shown.

[0029] Figure 7 A schematic diagram of another electronic device provided in at least one embodiment of the present disclosure is shown. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0031] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms “first,” “second,” and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, the terms “an,” “a,” or “the,” and similar terms do not indicate a quantity limitation, but rather indicate the presence of at least one. The terms “including,” “comprising,” or “containing,” and similar terms mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. The terms “connected,” “linked,” or similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms “upper,” “lower,” “left,” and “right,” etc., are used only to indicate relative positional relationships, and these relative positional relationships may change accordingly when the absolute position of the described objects changes.

[0032] To keep the following description of the embodiments of this disclosure clear and concise, detailed descriptions of some known functions and known components have been omitted.

[0033] Floating-point numbers (FP) are primarily used to represent decimals and typically consist of three parts: a sign bit, an exponent, and a mantissa. The exponent part can also be called the exponent code. For example, a floating-point number V can usually be represented in the following form:

[0034]

[0035] In this context, the sign bit s can be 1 bit, determining whether the floating-point number V is negative or positive; M represents the mantissa, which can include multiple bits and is a binary fractional form, defining the precision of the floating-point number; E represents the exponent (also called the exponent value), used to weight the floating-point number, reflecting the position of the decimal point in the floating-point number V, and defining the range of values ​​for the floating-point number.

[0036] Traditional floating-point numbers typically include three formats: half-precision floating-point (FP16), single-precision floating-point (FP32), and double-precision floating-point (FP64), with different numbers of bits in their exponent and mantissa parts.

[0037] AI accelerators are widely used for training deep learning models. For convolution operations, common in deep learning models, both hardware and software designs have been specifically optimized to accelerate computation. For example, various floating-point data formats have been developed and optimized for fields such as artificial intelligence and deep learning, including BF16 (brain floating point 16, 16-bit width), BF24 (brain floating point 24, 24-bit width), and TF32 (Tensor Float 32, 19-bit width). These data formats can significantly reduce computational processing, especially the computational resources and power consumption required for matrix multiplication or convolution multiplication operations. In addition, the processor also supports some common floating-point types, such as half-precision floating-point numbers (FP16, 16-bit width) or single-precision floating-point numbers (FP32, 32-bit width).

[0038] Low-precision matrix multiplication is increasingly being used in the training and inference of large AI models due to its significant performance gains with acceptable precision loss. In GPUs (Graphics Processing Units) or GPGPUs, matrix multiplication is typically performed in hardware by tensor kernels. Low-precision tensor kernels are several times more computationally powerful than high-precision tensor kernels, resulting in higher computational efficiency. Furthermore, the data volume of a low-precision tensor is also only a few times that of a high-precision tensor, leading to higher data transfer efficiency. Therefore, the end-to-end efficiency improvement brought by low-precision tensor computation is almost several times greater. For example, the computational power of an FP4 tensor kernel can be 2 to 8 times that of FP8, 4 to 16 times that of FP16 / BF16, or even higher. Moreover, the data volume is half that of FP8 and one-quarter that of FP16 / BF16.

[0039] To reduce the loss of precision, low-precision matrix multiplication introduces a scaling factor to maximize the numerical expressive power of low-precision tensors.

[0040] For low-precision matrix multiplication, such as D = A × B + C, where A, B, C, and D are all high-precision tensors, the general matrix multiplication operation using a scaling factor can be described as follows:

[0041] D = (α⊙A') × (β⊙B') + σ⊙C' or D = (α⊙A') × (β⊙B') + σ⊙C' β)⊙(A'×B')+σ⊙C'

[0042] D→D',γ

[0043] Where ⊙ represents element-wise multiplication. σ represents the outer product, × represents matrix multiplication, α is the scaling factor for tensor A, β is the scaling factor for tensor B, σ is the scaling factor for tensor C, and γ is the scaling factor for tensor D. Quantized tensors A', B', C', and D' are the low-precision tensors obtained by floating-point quantization of tensors A, B, C, and D, respectively. For example, the data dimensions of each parameter are shown below:

[0044] A: [m,s×k], B: [s×k,n], C: [m,n], D: [m,n], α: [m,s], β: [s,n], σ: [m,r] or [r,n], γ: [m,r] or [r,n].

[0045] Where m, n, s, k, and r are all positive integers. For example, for any row in tensor A, every k tensor elements in that row share a single scaling factor parameter; for any column in tensor B, every k tensor elements in that column share a single scaling factor parameter.

[0046] Figure 1 A schematic diagram of the structure of a general-purpose graphics processor (GPGPU) is shown.

[0047] like Figure 1 As shown, a general-purpose graphics processor is actually an array of programmable multiprocessors. For example, a programmable multiprocessor can be a streaming processor cluster (SPC), such as including... Figure 1 The diagram shows streaming processor clusters 1, ..., M, where M is a positive integer greater than 1. In a general-purpose graphics processor, one streaming processor cluster processes one computational task, or multiple streaming processor clusters process one computational task. Multiple streaming processor clusters share data through a global cache or global memory.

[0048] like Figure 1 As shown, taking streaming processor cluster 1 as an example, one streaming processor cluster includes multiple computing units, such as... Figure 1The system consists of Compute Unit 1, Compute Unit 2, ..., Compute Unit N, where N is a positive integer. Each Compute Unit (CU) is used to perform arithmetic and logical operations, such as accumulation, reduction, and regular addition, subtraction, multiplication, and division.

[0049] A computing unit includes multiple cores (also called computing kernels or computing cores). Figure 1 (Not shown in the image), each computing core includes an arithmetic logic unit (ALU), a floating-point unit, etc., and is used to perform specific computing tasks.

[0050] like Figure 1 As shown, each computational unit also provides a tensor core for performing tensor-related computations, such as matrix multiplication operations using the GEMM operator. Tensors are a crucial data structure in deep learning; they are high-dimensional generalizations of scalars, vectors, and matrices. Tensor operations are commonly used in the training and inference of deep learning models, and tensor cores can accelerate matrix multiplication. Tensor cores across multiple computational units can be uniformly scheduled and controlled.

[0051] In addition, the computation unit includes a register file, shared memory, and a tensor core memory unit for storing source and destination data related to computation tasks. Shared memory within a computation unit is used to share data between cores within that unit. The tensor core memory unit is a storage resource closely associated with the tensor core, used to store intermediate data when the tensor core performs tensor operations (such as matrix multiplication), and can perform data format processing on the tensor data to be processed so that data loaded from external sources conforms to the tensor core's data format requirements.

[0052] In parallel computing, computational tasks are typically executed by multiple threads. These threads are divided into multiple thread blocks before execution in a general-purpose graphics processor (or parallel computing processor), and then dispatched via a thread block distribution module. Figure 1 (Not shown in the image) Multiple thread blocks are distributed to various computation units. All threads in a thread block must be assigned to the same computation unit for execution. Simultaneously, thread blocks are broken down into minimum execution thread bundles (or simply warps), each containing a fixed number (or less than this fixed number) of threads, for example, 32 threads. Multiple thread blocks can execute in the same computation unit or in different computation units.

[0053] In each computing unit, the thread beam scheduling / distribution module ( Figure 1(Not shown in the diagram) Thread bundles are scheduled and allocated so that multiple computing cores within the computing unit can run thread bundles. Depending on the number of computing cores in the computing unit, multiple thread bundles within a thread block can be executed concurrently or in a time-sharing manner. Multiple threads within each thread bundle execute the same instructions. Memory execution instructions are issued to shared memory within the computing unit or further issued to intermediate-level caches, global caches, or global memory for read and write operations, etc.

[0054] Currently, the scaling factor is often determined by finding the maximum value of a data set. The purpose of using a scaling factor is to scale the data to the maximum range that the target data can represent with low precision, minimizing precision loss. For example, tensors... Its FP4 quantization tensor The definition of is:

[0055]

[0056] in, ,here Q max It is the maximum range of target accuracy. It is a very small floating-point number (e.g.) =1e-8, 1e-9, ..., 1e-12, etc.) are used to prevent scenarios involving division by zero.

[0057] As mentioned earlier, the increasing inference demands of large-scale artificial intelligence models place stringent requirements on the storage capacity, memory bandwidth, and computational efficiency of AI chips. Low-precision quantization technology, which effectively alleviates these pressures, has become a core technology in current AI chip design.

[0058] In traditional quantization methods (such as Max-Normalization), a single scaling factor is typically used to map the input data to the effective representation range of the target low-precision format, thus completing the quantization conversion from high-precision to low-precision data. For example, for an input tensor X, the quantization process follows the formula: Q = round (X / (max(|X|) / Q_max)), where max(|X|) represents the data with the largest absolute value in the input tensor X, and Q_max represents the maximum value in the numerical representation range corresponding to the target low-precision format. For example, floating-point precision FP4 includes E2M1 and E1M2 precision formats. If floating-point precision FP4 uses the E2M1 precision format, then Q_max is 6; if floating-point precision FP4 uses the E1M2 precision format, then Q_max is 3.5.

[0059] However, these traditional quantization methods based on a single scaling factor have many drawbacks, which severely limit their application in high-performance AI accelerators.

[0060] Traditional quantization methods based on a single scaling factor suffer from severe loss of quantization accuracy. When the input data exhibits a non-uniform distribution (e.g., containing a small number of large values ​​and a large number of small values ​​simultaneously), a single scaling factor, dominated by the global maximum value, leads to excessive compression of smaller values. For example, for the input data set {8, 8, 6, 6, 6, 4, 4, 3, 3, 2, 2, 1, 1, 1, 1, 1}, with floating-point precision FP4 using E2M1 precision format, the numerical expression range corresponding to E2M1 precision format is [-6.0, -4.0, -3.0, -2.0, -1.5, -1.0, -0.5, 0.0, 0.5, 1.0, 1.5, 2.0, 3.0, 4.0, 6.0]. Traditional quantization methods would map the maximum value 8 in the input data set to Q_max=6, with a corresponding scaling factor of 8 / 6≈1.333. At this point, for the minimum value 1 in the input data set, it is first quantized (for example, 1 / 1.333≈0.75, and the value closest to 0.75 in the numerical expression range corresponding to the E2M1 precision format is 1). Then, the quantized result 1 is dequantized (for example, 1×1.333=1.333). The final dequantized result is 1.333, that is, the original minimum value 1 is reconstructed as 1.333, which has a significant loss of precision (relative error of 33.3%), thus affecting the accuracy of model inference.

[0061] Traditional quantization methods lack flexibility because they employ fixed mapping rules, mapping the maximum value of the input data to the maximum value within the numerical expression range corresponding to the precision format used for the target precision. This method fails to consider the distribution differences of different input data, cannot dynamically adjust the mapping relationship according to the distribution of different input data, and ignores the principle of minimizing quantization error, resulting in poor adaptability to diverse data scenarios.

[0062] Furthermore, for precision formats with limited representation ranges, such as FP4 (e.g., E2M1 with a maximum of 6 and E1M2 with a maximum of 3.5), traditional quantization methods cannot effectively balance the precision allocation between the exponent field and the fractional field (i.e., the mantissa field).

[0063] For example, to meet the accuracy requirements of some scenarios, traditional quantization methods evaluate candidate scaling factors through multiple parallel circuits. This requires the deployment of repetitive quantization circuits, which significantly increases hardware area overhead and makes it difficult to deploy efficiently in area-constrained high-performance AI accelerators.

[0064] To address the above shortcomings, at least one embodiment of this disclosure proposes a data quantization circuit, data processing method, processor, electronic device, and storage medium based on a lookup table. The data quantization circuit is used to determine a target scaling factor for quantizing an input data block to a target precision and the quantization result of the input data block. The data quantization circuit includes: a data block preprocessing unit, an exponent alignment unit, an index generation unit, a lookup table, and an output encoding unit. The data block preprocessing unit is configured to determine target input data with the largest exponent portion in the input data block, wherein the input data block includes multiple input data. The exponent alignment unit is configured to align the exponent portions of the multiple input data to the exponent value based on the exponent value of the exponent portion of the target input data, obtaining multiple intermediate data corresponding to the multiple input data. The index generation unit is configured to extract the mantissa portion of the multiple intermediate data and combine it with N scaling factor mantissa portion candidates to obtain N first indices corresponding to each input data, wherein N is a positive integer determined based on the number of bits in the scaling factor mantissa portion candidates. The lookup table is configured to pre-store multiple data items and multiple index entries in a one-to-one correspondence. Each index entry is composed of T input mantissa candidates and N scaling factor mantissa candidates, where T is a positive integer determined based on the number of bits in the mantissa of the input data. Each data item includes the quantization error and quantization result obtained by quantizing the data using the information in the corresponding index entry. The output encoding unit is configured to access the lookup table based on the N first indices corresponding to each input data item, and determine the target scaling factor and the quantization result of the input data block based on the N query results corresponding to each of the multiple input data items. Each query result includes the quantization error and quantization result corresponding to each first index.

[0065] The data quantization circuit based on a lookup table provided in at least one embodiment of this disclosure can achieve efficient testing of multiple different scaling factors and selection of the optimal scaling factor. It uses a lookup table to implement the "quantization-dequantization-error calculation" operation, and determines the target scaling factor and the quantization result of the input data block based on the query results returned by the lookup table. The complete quantization process (including candidate scaling factor evaluation and quantization error calculation) is encoded into a unified lookup table, enabling all key operations (including candidate scaling factor evaluation) to be completed through a simple lookup table and basic logic gates. This simplifies the quantization process from a multi-step, complex calculation process to a single-cycle lookup operation, achieving a hardware-friendly quantization scheme and greatly reducing hardware complexity. This disclosure dynamically selects the optimal target scaling factor through a lookup table, ensuring high quantization accuracy. While maintaining high quantization accuracy, it significantly reduces implementation costs and latency, greatly improving hardware efficiency.

[0066] This disclosure innovatively provides an index structure that, through exponent alignment, allows the mantissas of all data to be directly compared and processed with the same precision. Thus, the index entries only need to consider the mantissa portion, and the lookup table includes a finite number of index entries, minimizing the storage space occupied by the lookup table.

[0067] Furthermore, the data quantization circuit provided in this disclosure can adapt to different precision formats and supports more candidate scaling factors (i.e., candidate scaling factors in the mantissa), which improves hardware reusability and achieves better quantization accuracy while maintaining hardware efficiency.

[0068] The present disclosure will now be described through several specific embodiments. To keep the following description of the embodiments of the present disclosure clear and concise, detailed descriptions of known functions and components may be omitted. When any component of the embodiments of the present disclosure appears in more than one drawing, the component is represented by the same or similar reference numerals in each drawing.

[0069] The following detailed description, with reference to the accompanying drawings, describes some embodiments and examples of this disclosure.

[0070] The lookup table-based data quantization circuit disclosed herein can be applied to processors such as artificial intelligence processors. The processor can be any one of a graphics processing unit (GPU), a tensor processing unit (TPU), a neural network processing unit (NPU), a deep learning processing unit (DPU), an accelerated processing unit (APU), or a general-purpose graphics processing unit (GPGPU).

[0071] For example, this disclosure describes the data quantization circuit as being set in a graphics processor or a general-purpose graphics processor, but of course, this disclosure does not limit it.

[0072] Figure 2 A schematic diagram of the structure of a processor provided in at least one embodiment of the present disclosure is shown.

[0073] For example, Figure 2The second processor 200 shown can be a graphics processing unit (GPU) or a general-purpose graphics processing unit (GPGPU). The second processor 200 includes multiple streaming processor clusters (e.g., streaming processor cluster 1, ..., streaming processor cluster M, etc., where M is an integer greater than 1), and each streaming processor cluster includes multiple computing units (e.g., computing unit 1, computing unit 2, ..., computing unit N, etc., where N is an integer greater than 1). Figure 2 As shown, each computing unit is equipped with a data quantization module 201. This "data quantization module 201" is the lookup table-based data quantization circuit provided in this disclosure, and its circuit structure can be referred to Figure 3 The data quantization circuit 300 shown below is described in detail.

[0074] In addition, it should be noted that Figure 2 The components of the second processor 200 shown are merely exemplary and not limiting. Depending on the actual application requirements, the second processor 200 may also have other components (such as tensor cores, shared memory, etc.).

[0075] Figure 3 A schematic diagram of a lookup table-based data quantization circuit provided in at least one embodiment of this disclosure is shown.

[0076] like Figure 3 As shown, the lookup table-based data quantization circuit 300 includes: a data block preprocessing unit 310, an exponent alignment unit 320, an index generation unit 330, a lookup table 340, and an output encoding unit 350. The data quantization circuit 300 is configured to determine a target scaling factor for quantizing the input data block to a target precision and the quantization result of the input data block. The target precision represents the precision of the quantization result of the input data block.

[0077] For example, the data block preprocessing unit 310 can be configured to determine target input data with the largest exponential portion in the input data block, wherein the input data block includes multiple input data.

[0078] For example, the input data block can be a tensor data block, which includes tensor elements with the same floating-point precision; this disclosure does not impose any restrictions on this. For example, multiple input data are high-precision floating-point numbers, such as FP16, BF16, etc.; this disclosure does not impose any specific restrictions on this.

[0079] For example, the data block preprocessing unit 310 includes a tree-structured comparator network. A tree-structured comparator network is a parallel, hierarchical hardware comparison architecture used to efficiently find the maximum (or minimum) value among multiple input data. The tree-structured comparator network uses the input layer as leaf nodes and each comparator node as an intermediate level, converging upwards according to the branching relationship of a tree: the bottom-level comparator receives the original input data and performs comparison and exchange operations; its output serves as the input to the comparator of the next higher level. After multiple rounds of hierarchical comparisons, the final sorting result is output from the level where the root node of the tree resides.

[0080] For example, a tree comparator network is configured to extract the exponential portion of multiple input data and determine the largest exponential portion among the exponential portions of the multiple input data, so as to use the input data with the largest exponential portion as the target input data.

[0081] For example, in response to an input data block comprising M input data points, where M is a positive integer, a tree comparator network comprising ⌈log₂M⌉ stages of comparators is configured to receive the exponent portions of the M input data points in parallel and output the largest exponent portion by comparing them stage by stage. In this case, the delay of the tree comparator network is O(log₂M) to achieve fast detection of the maximum value.

[0082] For example, the exponent alignment unit 320 can be configured to align the exponent portions of multiple input data to the exponent value MAX_exp based on the exponent value MAX_exp of the exponent portion of the target input data, thereby obtaining multiple intermediate data corresponding to the multiple input data. Here, "intermediate data" can be understood as the input data with the exponent portions aligned after performing the exponent alignment operation on the multiple input data.

[0083] For example, the exponent alignment unit 320 may include a subtractor and a shifter. The subtractor is configured to determine the difference between the exponent portion and the maximum exponent portion of each of the multiple input data. The shifter is configured to right-shift the mantissa portion of the multiple input data by the number of bits indicated by the difference output by the subtractor, thereby aligning the exponent portions of the multiple input data to the exponent value MAX_exp of the exponent portion of the target input data, and obtaining multiple intermediate data corresponding to the multiple input data. The exponent portion of the multiple intermediate data is the exponent value MAX_exp.

[0084] For example, if the difference between the exponent portion and the maximum exponent portion of the input data in the input data block is 0, there is no need to move the mantissa portion of the input data.

[0085] For example, the index generation unit 330 can be configured to extract the mantissa of multiple intermediate data and combine it with N scaling factor mantissa candidates to obtain N first indices corresponding to each input data, where N is a positive integer determined based on the number of bits in the scaling factor mantissa candidates. For example, N equals 2. m 'm' represents the number of bits in the mantissa of the scaling factor. For example, taking a floating-point precision of FP8 for the scaling factor, it can specifically use the E4M3 precision format, meaning the exponent has 4 bits and the mantissa has 3 bits. Therefore, the number of candidates for the mantissa of the scaling factor can be 2. 3 =8, that is, N=8, and the specific candidates for the mantissa of the 8 scaling factors can be 000, 001, 010, 011, 100, 101, 110, and 111.

[0086] For example, lookup table 340 is configured to pre-store multiple data items and multiple index items in a one-to-one correspondence. Each index item is composed of T input mantissa candidates and N scaling factor mantissa candidates, where T is a positive integer determined based on the number of bits in the mantissa of the input data. Each data item includes the quantization error and quantization result obtained by quantizing the data using information from the corresponding index item. (See the supplement above for T.)

[0087] For example, each data item in lookup table 340 also includes a first quantization result obtained by quantizing the first data using a first scaling factor, wherein the floating-point precision of the first quantization result is a target precision. For example, the target precision is a low-precision floating-point number, such as FP4. Of course, this disclosure does not impose specific limitations on this, and the target precision is lower than the floating-point precision of the input data.

[0088] For example, each index entry in lookup table 340 consists of one input mantissa candidate and one scaling factor mantissa candidate. The data item corresponding to this index entry includes a first quantization error obtained by quantizing the first data using a first scaling factor candidate. The first scaling factor candidate uses the scaling factor mantissa candidate in this index entry as its mantissa, and the first data uses the input mantissa candidate in this index entry as its mantissa. The precision format of the first scaling factor is the same as the precision format of the target scaling factor, and the precision format of the first data is the same as the precision format of the multiple input data.

[0089] For example, the output encoding unit 350 is configured to access the lookup table 340 according to the N first indices corresponding to each input data, and determine the target scaling factor and the quantization result of the input data block according to the N query results corresponding to each of the multiple input data, wherein each query result includes the quantization error and quantization result corresponding to each first index.

[0090] It is important to note that for each input data point, there are N query results. The quantization error included in each query result is obtained by quantizing the input data based on a candidate scaling factor tail.

[0091] The data quantization circuit based on a lookup table provided in at least one embodiment of this disclosure can achieve efficient testing of multiple different scaling factors and selection of the optimal scaling factor. It uses a lookup table to implement the "quantization-dequantization-error calculation" operation, and determines the target scaling factor and the quantization result of the input data block based on the query results returned by the lookup table. The complete quantization process (including candidate scaling factor evaluation and quantization error calculation) is encoded into a unified lookup table, enabling all key operations (including candidate scaling factor evaluation) to be completed through a simple lookup table and basic logic gates. This simplifies the quantization process from a multi-step, complex calculation process to a single-cycle lookup operation, achieving a hardware-friendly quantization scheme and greatly reducing hardware complexity. This disclosure dynamically selects the optimal target scaling factor through a lookup table, ensuring high quantization accuracy. While maintaining high quantization accuracy, it significantly reduces implementation costs and latency, greatly improving hardware efficiency.

[0092] This disclosure innovatively provides an index structure that, through exponent alignment, allows the mantissas of all data to be directly compared and processed at the same precision. Thus, index entries only need to consider the mantissa portion, and the lookup table includes a limited number of index entries, minimizing the storage space occupied by the lookup table. Furthermore, the data quantization circuit provided in this disclosure is adaptable to different precision formats and supports more candidate scaling factors (i.e., candidates for the mantissa portion of the scaling factor), improving hardware reusability and achieving better quantization precision while maintaining hardware efficiency.

[0093] In at least one embodiment of this disclosure, the target precision includes at least one precision format, each precision format corresponding to a different lookup table, and the data quantization circuit is further configured to select the corresponding lookup table according to the precision format used for the target precision.

[0094] For example, when the target precision is FP4, the target precision can adopt either the E2M1 precision format (exponent part bit width is 2, mantissa part bit width is 1) or the E1M2 precision format (exponent part bit width is 1, mantissa part bit width is 2). In this case, two lookup tables can be set in the data quantization circuit provided in this disclosure, corresponding to the E2M1 precision format and the E1M2 precision format respectively.

[0095] For example, in some embodiments of this disclosure, the data quantization circuit may further include a register configured to store a predetermined value that indicates a lookup table corresponding to the precision format used for the target precision. For example, when the predetermined value is "0", the lookup table corresponding to the E2M1 precision format is selected; when the predetermined value is "1", the lookup table corresponding to the E1M2 precision format is selected.

[0096] The data quantization circuit provided in at least one embodiment of this disclosure can dynamically select the corresponding lookup table according to the precision format adopted by the target precision, thereby realizing hardware reuse of multiple precision formats and avoiding circuit reconstruction caused by changes in precision format.

[0097] In one embodiment, if the precision format of the first scaling factor and the precision format of the target scaling factor are both E4M3 (E4 indicates that the bit width of the exponent part is 4 bits, and M3 indicates that the bit width of the mantissa part is 3 bits), and the floating-point precision of the first data and multiple input data is BF16, specifically, for example, E7M8 (E4 indicates that the bit width of the exponent part is 7 bits, and M3 indicates that the bit width of the mantissa part is 8 bits), then the bit width of the mantissa part candidate of the scaling factor is 3 bits, the bit width of the mantissa part of the input data is 8 bits, and the bit width of the input mantissa part candidate is also 8 bits. Therefore, the index entries of the lookup table are 11 bits, the lookup table includes 2048 index entries, and these index entries are composed of 256 (2 8 ) input mantissa candidates (e.g., 00000000, 00000001, ..., 11111110 and 11111111) and 8 (2 3 It is composed of a combination of scaling factor mantissa candidates (e.g., 0000, 0001, ..., 1110 and 1111).

[0098] For example, an index entry in the lookup table, index_entry1, is composed of an input mantissa candidate X1_mantissa (e.g., 00000000) and a scaling factor mantissa candidate SF1_mantissa (e.g., 0000). The corresponding data entry, data_entry1, includes the quantization error SE1 and quantization result Q1 obtained by quantizing data X1 using scaling factor candidate SF1, where scaling factor candidate SF1 uses scaling factor mantissa candidate SF1_mantissa from index_entry1 as its mantissa, and data X1 uses input mantissa candidate X1_mantissa from index_entry1 as its mantissa.

[0099] Specifically, the quantification result Q1 is obtained through the following steps S1-S4:

[0100] S1: Convert the mantissa of data X1, X1_mantissa, to the first floating-point data, float_X1_mantissa, and convert the mantissa of scaling factor candidate SF1, SF1_mantissa, to the second floating-point data, float_SF1_mantissa.

[0101] S2: Determine the first value q based on the precision format used in the quantization result Q1. max .

[0102] For example, different precision formats correspond to different first values, where the first value is the least power of 2 greater than the maximum value in the range corresponding to the precision format. For instance, when the quantization result Q1 uses the precision format E2M1, the range corresponding to E2M1 is [-6.0, -4.0, -3.0, -2.0, -1.5, -1.0, -0.5, 0.0, 0.5, 1.0, 1.5, 2.0, 3.0, 4.0, 6.0], and the first value q... max For 8 (2) 3 > 6) When the quantization result Q1 adopts the precision format E1M2, the numerical expression range corresponding to E1M2 is [-3.5, -3.0, -2.5, -2.0, -1.5, -1.0, -0.5, 0.0, 0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 3.5], and the first value q max 4 (2) 2 > 3.5). In this embodiment, the first value q max The determination method can achieve better quantization accuracy.

[0103] In addition, the first value q max Alternatively, it can be determined based on the exponent portion of the target scaling factor. For example, when shifting the exponent value of the exponent portion of the target input data in the input data block p bits to the left as the exponent portion of the target scaling factor, the first value q... max Equal to 2 p Among them, the maximum value in the numerical expression range corresponding to the precision format used for the target precision is less than 2. p And greater than 2 p-1 p is a positive integer.

[0104] S3: Perform a multiplication operation on the first value and the ratio of the first floating-point data to the second floating-point data to obtain the multiplication result (q). max × float_X1_mantissa / float_SF1_mantissa).

[0105] S4: Within the numerical expression range corresponding to the precision format adopted by the quantization result Q1, determine the result of the multiplication operation (q). max The value closest to (× float_X1_mantissa / float_SF1_mantissa) is taken as the quantization result Q1.

[0106] Specifically, the quantization error SE1 is obtained through the following steps S5-S6:

[0107] S5: Based on the first value q max The second floating-point data float_SF1_mantissa is used to perform a dequantization operation on the quantization result Q1 to obtain the dequantization result De_Q1.

[0108] For example, the dequantization result De_Q1 = Q1 × float_SF1_mantissa / q max .

[0109] S6: Based on the dequantization result De_Q1 and the first floating-point data float_X1_mantissa, determine the quantization error SE1 corresponding to data X1.

[0110] For example, the dequantization result De_Q1 is first converted from floating-point data to fixed-point data fixed_De_Q1. Then, the squared error between the fixed-point dequantization result fixed_De_Q1 and the mantissa of data X1 is calculated, and the calculated squared error is used as the quantization error SE1 corresponding to data X1. Of course, the method of calculating the quantization error in this example is merely illustrative, and other methods can also be used to calculate the quantization error in the embodiments of this disclosure.

[0111] After executing steps S1-S6, the obtained quantization result Q1 and quantization error SE1 are stored in the data item data_entry1, thus completing the construction process of the index item index_entry1 and its corresponding data item data_entry1. Following this, the remaining 2047 index items and their corresponding data items in the lookup table are constructed in a similar manner, which will not be elaborated upon here.

[0112] For example, the obtained quantization error can be log-encoded and stored in the corresponding data item in a lookup table to reduce storage requirements.

[0113] For example, a piecewise linear approximation can be performed on the lookup table, dividing the original nonlinear mapping function into multiple intervals, and using a linear function to approximate the exact value in each interval. As a result, only the endpoints or slope parameters of each segment need to be stored, rather than the complete mapping table entries, which significantly reduces the number of entries required by the lookup table (i.e., the number of index entries and data entries), reducing hardware storage overhead while ensuring quantization accuracy.

[0114] In at least one embodiment of this disclosure, when the output encoding unit determines the target scaling factor based on N query results corresponding to each of the multiple input data, it may perform the following operations: determine the mantissa portion of the target scaling factor based on the N query results corresponding to each of the multiple input data; shift the exponent value of the exponent portion of the target input data left by p bits to obtain the exponent portion of the target scaling factor, wherein the maximum value in the numerical expression range corresponding to the precision format adopted by the target precision is less than 2. p And greater than 2 p-1 p is a positive integer.

[0115] For example, when the target precision uses the E2M1 precision format, the maximum value in its numerical expression range is 6. In this case, the exponent value of the exponent part of the target input data is shifted left by 3 bits to obtain the exponent part of the target scaling factor. When the target precision uses the E1M2 precision format, the maximum value in its numerical expression range is 3.5. In this case, the exponent value of the exponent part of the target input data is shifted left by 2 bits to obtain the exponent part of the target scaling factor. Subsequently, the output encoding unit is configured to concatenate the exponent part of the determined target scaling factor with the mantissa part determined later to form the complete target scaling factor.

[0116] In the data quantization circuit provided in the above embodiment, since the exponent alignment unit pre-performs exponent alignment operation on multiple input data, the output encoding unit only needs to determine the mantissa of the target scaling factor from N scaling factor mantissa candidates to obtain the complete target scaling factor, thereby simplifying the determination process of the target scaling factor and reducing computational complexity and hardware overhead.

[0117] For example, when the output encoding unit determines the mantissa of the target scaling factor based on the N query results corresponding to the multiple input data, it can perform the following operations: divide the quantization errors in the N query results corresponding to the multiple input data into N sets of quantization errors, where each set of quantization errors includes multiple quantization errors obtained by quantizing multiple input data using the same scaling factor mantissa candidate; calculate the average of the multiple quantization errors included in each set of quantization errors as the average quantization error corresponding to the scaling factor candidate; and select the scaling factor mantissa candidate with the smallest average quantization error among the N scaling factor mantissa candidates as the mantissa of the target scaling factor.

[0118] The following example describes the process of constructing the first index and accessing the lookup table based on the first index, with the input data block consisting of 16 floating-point numbers with a precision of BF16 (precision format of E8M7), the candidate bits (or bit width) of the scaling factor mantissa being 3 bits, and the target precision being FP4 (using E2M1 precision format).

[0119] For example, based on the number of candidate bits in the mantissa portion of the scaling factor (i.e., 3 bits), set 8 (N=2). 3 If there are 100 candidate scaling factor mantissas (e.g., 000, 001, 010, 011, 100, 101, 110, and 111), then each input data corresponds to 8 different first indices. The mantissa of the input data is 7 bits. The mantissa of the intermediate data obtained after exponent alignment and shift operations is also 7 bits (the leading 1 before the decimal point is omitted). The first index, which is composed of the candidate scaling factor mantissas and the mantissa of the input data, is 11 bits long and is denoted as index[10:0].

[0120] For example, index[10:8] can be used to store candidates for the mantissa portion of the scaling factor, and index[7:0] can be used to store the mantissa portion of the intermediate data; or index[2:0] can be used to store candidates for the mantissa portion of the scaling factor, and index[10:3] can be used to store the mantissa portion of the intermediate data. This disclosure does not restrict how the number of bits of the first index is specifically allocated.

[0121] For example, the index generation unit generates eight first indexes for each input data in sequence, generating a total of 128 first indexes. Simultaneously, the output encoding unit uses these 128 first indexes to access the lookup table, obtaining the query result corresponding to each first index (i.e., the data item corresponding to the first index in the lookup table), resulting in a total of 128 query results. Each query result includes the quantization error and quantization result corresponding to each first index.

[0122] Subsequently, the output encoding unit is configured to: divide the 128 quantization errors into 8 sets of quantization errors, where each set includes 16 quantization errors obtained by quantizing data using the same scaling factor mantissa candidate. For example, one set uses 000 as the scaling factor mantissa candidate and another set uses 001 as the scaling factor mantissa candidate. The average value of the 16 quantization errors in each set is calculated and used as the average quantization error corresponding to the scaling factor candidate, thus obtaining the average quantization error corresponding to each of the 8 scaling factor mantissa candidates. The scaling factor mantissa candidate with the smallest average quantization error among the 8 scaling factor mantissa candidates is taken as the mantissa of the target scaling factor. According to the E2M1 precision format used for the target precision, the exponent value of the exponent part of the target input data is shifted left by 3 bits to obtain the exponent part of the target scaling factor. The exponent part and mantissa part of the determined target scaling factor are concatenated to form the complete target scaling factor.

[0123] For example, the lookup table returns query results that also include the quantization results of the corresponding input data. The output encoding unit, after determining the mantissa of the target scaling factor, is configured to determine the quantization results of the input data block.

[0124] In at least one embodiment of this disclosure, when the output encoding unit performs the determination of the quantization result of the input data block, it may include performing the following operations: using the mantissa of the target scaling factor and the mantissa of each input data to form a second index, wherein one input data corresponds to one second index; accessing a lookup table according to the second index corresponding to each of the multiple input data, obtaining the second quantization result corresponding to each of the multiple input data (i.e., the second quantization result corresponding to each of the multiple second indices) from the multiple query results returned by the lookup table, and using it as the quantization result of the input data block.

[0125] In the data quantization circuit provided in at least one embodiment of this disclosure, the output encoding unit can directly obtain the quantization results of the target scaling factor and the input data block by accessing a lookup table, thereby unifying the selection of the scaling factor and the processing of the input data into a single-cycle lookup operation, thereby reducing the computational overhead and power consumption of the circuit and reducing the circuit complexity.

[0126] For example, Figure 4 A schematic diagram illustrating the process of a data quantization circuit performing data quantization according to at least one embodiment of the present disclosure is shown. The data quantization circuit includes: a data block preprocessing unit, an exponent alignment unit, an index generation unit, a lookup table, and an output encoding unit.

[0127] exist Figure 4In the example, the input data block includes 16 input data, and the description is based on the following example: the floating-point precision of multiple input data in the input data block is BF16 (precision format is E8M7), the floating-point precision of the scaling factor is FP8 (precision format is E4M3), and the target precision is FP4 (precision formats include E2M1 and E1M2). Of course, this disclosure is not limited to this. For other floating-point precisions or precision formats, or other quantities of input data included in the input data block, adaptive modifications can be made based on this embodiment, which will not be elaborated in detail here.

[0128] like Figure 4 As shown, the data block preprocessing unit is configured to receive an input data block and determine the maximum value X_max (i.e., the target input data with the largest exponent portion) among 16 BF16 input data, the target input data including the mantissa portion M_mant and the exponent portion M_exp.

[0129] The output encoding unit is configured to: determine whether the target precision adopts the E2M1 precision format; if the target precision adopts the E2M1 precision format, then the exponent part M_exp of the target input data is shifted left by 3 bits as the exponent part SF_exp of the scaling factor; if the target precision adopts the E1M2 precision format, then the exponent part M_exp of the target input data is shifted left by 2 bits as the exponent part SF_exp of the scaling factor.

[0130] exist Figure 4 In the example shown, 8 (N=2) is set based on the number of candidate bits in the mantissa of the scaling factor (i.e., 3 bits). 3 8 scaling factor mantissa candidates SF_mantissa (i.e., 000, 001, 010, 011, 100, 101, 110 and 111), and iterate through the 8 scaling factor mantissa candidates SF_mantissa.

[0131] Considering that some SF_mantissa values ​​are small, the corresponding scaling factors for these SF_mantissas will also be small, which can easily lead to quantization results exceeding the target precision range during data quantization. Therefore, during the process of traversing the eight SF_mantissas (hereinafter referred to as the "first traversal process"), the output encoding unit is also configured to: determine whether the current SF_mantissa is less than or equal to 2; if the current SF_mantissa is less than or equal to 2, set the exponent part corresponding to the current SF_mantissa to SF_exp+1; if the current SF_mantissa is greater than 2, set the exponent part corresponding to the current SF_mantissa to SF_exp. The output encoding unit adjusts the exponent part corresponding to the SF_mantissa according to its size to ensure that the quantization result obtained using the scaling factor corresponding to the SF_mantissa is within the target precision range.

[0132] For the current SF_mantissa, for example, SF_mantissa=000, 16 BF16 input data are traversed. During the process of traversing the 16 BF16 input data (hereinafter also referred to as the "second traversal process"), the exponent alignment unit is configured to align the exponent part X_mant_i of the current input data to the exponent value of the exponent part M_exp of the target input data, thereby obtaining the mantissa part X*_mant_i of the intermediate data corresponding to the current input data.

[0133] The index generation unit is configured to combine the mantissa of the intermediate data corresponding to the current input data (e.g., X*_mant_0) with the current SF_mantissa to obtain the first index index_0[10:0] corresponding to the current input data, where index_0[10:8] = SF_mantissa and index_0[7:0] = X*_mant_0. The output encoding unit is configured to access the lookup table (LUT) based on the first index index_0[10:0] of the current input data to obtain the quantization error (e.g., SE_0) of quantizing the current input data X*_mant_0 using a first scaling factor, where the first scaling factor uses the current SF_mantissa as the mantissa. Then, the above process is performed on the mantissa of the intermediate data corresponding to the next input data (e.g., X*_mant_1), including constructing a first index index_1 corresponding to X*_mant_1, where index_1[10:8] = SF_mantissa and index_1[7:0] = X*_mant_1. The first index index_1[10:0] is used to access the lookup table to obtain the quantization error (e.g., SE_1) of quantizing the current input data X*_mant_1 using the first scaling factor. The above process is repeated until the second traversal process is completed. The quantization error (SE_0-SE_15) of quantizing 16 input data using the current SF_mantissa (e.g., 000) as the mantissa of the scaling factor is obtained through the above second traversal process.

[0134] For the current SF_mantissa, after completing the second traversal process, the output encoding unit calculates the average quantization error (MSE = (∑SE_i) / 16) corresponding to the current SF_mantissa based on the quantization errors (SE_0-SE_15) corresponding to the 16 input data. Then, the output encoding unit determines whether the first traversal process has been completed. If it is determined that the first traversal process has not been completed, the output encoding unit continues to process the next SF_mantissa (e.g., 001) and obtains the average quantization error corresponding to the next SF_mantissa in a similar process until the first traversal process has been completed, obtaining the average quantization errors corresponding to each of the 8 SF_mantissas.

[0135] After determining that the first traversal process is complete (i.e., the average quantization error corresponding to each of the 8 SF_mantissas has been obtained), the output encoding unit is configured to select the SF*_mantissa with the smallest average quantization error from the 8 SF_mantissas as the mantissa of the target scaling factor, and concatenate the SF*_mantissa and its corresponding SF_exp to obtain the complete target scaling factor.

[0136] The index generation unit is configured to combine SF*_mantissa and the mantissa portions of the intermediate data corresponding to each of the 16 input data (e.g., X*_mant_0-X*_mant_15) to obtain 16 second indices index*[10:0]. The output encoding unit sequentially accesses a lookup table based on the 16 second indices index*[10:0] to obtain the quantization results (i.e., the quantization results of the input data blocks) corresponding to each of the 16 input data returned by the lookup table.

[0137] Finally, the output encoding unit outputs the quantization result of the input data block and the target scaling factor used, and ends the data quantization process.

[0138] Compared with the lookup table-based data quantization circuit provided according to at least one embodiment of the present disclosure (such as...) Figure 3 Correspondingly, at least one embodiment of this disclosure also provides a data processing method, which possesses the corresponding principles and beneficial effects of the data quantization circuit.

[0139] Figure 5 A schematic flowchart of a data processing method provided in at least one embodiment of this disclosure is shown.

[0140] For example, such as Figure 5 As shown, the data processing method includes steps S510-S540.

[0141] S510: Determine the target input data in the input data block that has the largest exponential portion, wherein the input data block includes multiple input data.

[0142] S520: Based on the exponent value of the exponent part of the target input data, align the exponent parts of multiple input data to the exponent value to obtain multiple intermediate data corresponding to the multiple input data respectively.

[0143] S530: Extract the mantissa of multiple intermediate data and combine it with N scaling factor mantissa candidates to obtain N first indices corresponding to each input data, where N is a positive integer and is determined based on the number of bits of the scaling factor mantissa candidates.

[0144] S540: Based on the N first indices corresponding to each input data, access the lookup table, and based on the N query results corresponding to each of the multiple input data, determine the target scaling factor and the quantization result of the input data block for quantizing to the target precision. Each query result includes the quantization error and quantization result corresponding to each first index. The lookup table includes multiple data items and multiple index items that correspond one-to-one. The multiple index items are composed of T input mantissa candidates and N scaling factor mantissa candidates, where T is a positive integer and is determined based on the number of bits in the mantissa of the input data. Each data item includes the quantization error and quantization result of quantizing the data using the information in the corresponding index item.

[0145] For example, when determining the target scaling factor and the quantization result of the input data block to a target precision based on N query results corresponding to each of the multiple input data, the data processing method includes: determining the mantissa of the target scaling factor based on the N query results corresponding to each of the multiple input data; shifting the exponent value p bits to the left to obtain the exponent of the target scaling factor, wherein the maximum value in the numerical expression range corresponding to the precision format used for the target precision is less than 2. p And greater than 2 p-1 p is a positive integer; the second index is formed by using the mantissa of the target scaling factor and the mantissa of each input data; and according to the second index corresponding to each of the multiple input data, the lookup table is accessed, and the second quantization result corresponding to each of the multiple input data is obtained from the multiple query results returned by the lookup table, which is used as the quantization result of the input data block.

[0146] The data processing method provided in at least one embodiment of this disclosure can achieve highly efficient testing of multiple different scaling factors and selection of the optimal scaling factor. It uses a lookup table to implement the "quantization-dequantization-error calculation" operation, and determines the target scaling factor and the quantization result of the input data block based on the query results returned by the lookup table. The complete quantization process (including candidate scaling factor evaluation and quantization error calculation) is encoded into a unified lookup table, enabling all key operations (including candidate scaling factor evaluation) to be completed through a simple lookup table and basic logic gates. This simplifies the quantization process from a multi-step, complex calculation process to a single-cycle lookup operation, achieving a hardware-friendly quantization scheme and greatly reducing hardware complexity. By dynamically selecting the optimal target scaling factor through the lookup table, high quantization accuracy is ensured, achieving a significant reduction in implementation cost and latency while maintaining high quantization accuracy, thus greatly improving hardware efficiency.

[0147] The data processing method provided in at least one embodiment of this disclosure innovatively offers an index structure. By aligning the exponents, the mantissas of all data can be directly compared and processed at the same precision. Thus, the index entries only need to consider the mantissa portion, and the lookup table includes a limited number of index entries, minimizing the storage space occupied by the lookup table. Moreover, this data processing method can adapt to different precision formats and supports more candidate scaling factors (i.e., candidates for the mantissa portion of the scaling factor), improving hardware reusability and achieving better quantization precision while maintaining hardware efficiency.

[0148] At least one embodiment of this disclosure also provides a computer-readable storage medium storing computer-readable instructions thereon, which, when executed by at least one processor, cause the at least one processor to perform the data processing method provided in any embodiment of this disclosure.

[0149] At least one embodiment of this disclosure also provides a processor (for example, see...). Figure 2 The processor shown includes multiple computing units, each of which is provided with a lookup table-based data quantization circuit according to any embodiment of the present disclosure. For a more detailed description of the data quantization circuit disposed in the computing unit, please refer to the relevant description of the data quantization circuit 300 as described above; repeated details will not be repeated here.

[0150] In at least one embodiment, a dedicated hardware module, namely a data quantization circuit, is added to the processor. This data quantization circuit can efficiently test multiple different scaling factors and select the optimal scaling factor. It uses a lookup table to implement the "quantization-dequantization-error calculation" operation, and determines the target scaling factor and the quantization result of the input data block based on the lookup table results. The complete quantization process (including candidate scaling factor evaluation and quantization error calculation) is encoded into a unified lookup table, enabling all key operations (including candidate scaling factor evaluation) to be completed through a simple lookup table and basic logic gates. This simplifies the quantization process from a multi-step, complex calculation process to a single-cycle lookup operation, achieving a hardware-friendly quantization scheme and greatly reducing hardware complexity. By dynamically selecting the optimal target scaling factor through the lookup table, high quantization accuracy is ensured. This achieves high quantization accuracy while significantly reducing implementation costs and latency, and greatly improving hardware efficiency.

[0151] Furthermore, this data quantization circuit incorporates an innovative index structure. Through exponent alignment, the mantissas of all data can be directly compared and processed at the same precision. Thus, index entries only need to consider the mantissa portion, and the lookup table includes a limited number of index entries, minimizing the storage space occupied by the lookup table. Moreover, this data quantization circuit is adaptable to different precision formats and supports more candidate scaling factors (i.e., candidates for the mantissa portion of the scaling factor), improving hardware reusability and achieving superior quantization precision while maintaining hardware efficiency.

[0152] Figure 6 A schematic diagram of an electronic device provided in at least one embodiment of the present disclosure is shown.

[0153] For example, such as Figure 6 As shown, the first electronic device 600 includes a first processor 610. For example, the first processor 610 may employ... Figure 2 The second processor 200 shown is implemented in the following structure. For example, the first electronic device 600 can be any electronic device including those with computing capabilities, such as a laptop, tablet, desktop computer, web server, etc., and the embodiments disclosed herein are not limited thereto.

[0154] For example, the electronic device may also include other forms of processing units with data processing capabilities and / or instruction execution capabilities, such as a central processing unit (CPU) or a digital signal processor (DSP), as well as storage units. The electronic device may also have an operating system and application programming interfaces (e.g., OpenGL (Open Graphics Library), Metal, etc.). For example, the electronic device may also include output components such as display components, such as liquid crystal displays (LCDs), organic light-emitting diode (OLED) displays, quantum dot light-emitting diode (QLED) displays, etc. The embodiments disclosed herein are not limited in this regard.

[0155] It should be noted that, for clarity and brevity, this disclosure does not show all the constituent units of the first electronic device 600. To achieve the necessary functions of the first electronic device 600, those skilled in the art can provide or set other constituent units (not shown) according to specific needs, and this disclosure does not limit this.

[0156] The following is for reference. Figure 7It shows a specific structural schematic diagram of a second electronic device (e.g., a terminal device or a server) 700 suitable for implementing a processor including embodiments of the present disclosure.

[0157] The terminal devices in this disclosure can include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. For example, the electronic device can be in the form of a server, used for various application scenarios such as deep learning and artificial intelligence, scientific computing, graphics rendering and video editing, virtual reality and game development, and cloud services. For example, the electronic device can be a dedicated server such as a data center or cloud computing center that is deployed with tasks such as deep learning training, large-scale data analysis, and high-performance computing.

[0158] Figure 7 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0159] like Figure 7 As shown, the second electronic device 700 may include a processing device 701, such as the aforementioned second processor 200 or first processor 610, which can perform various appropriate actions and processes according to non-transitory computer-readable instructions stored in memory to achieve various functions. The processing device 701 may also include a central processing unit (CPU), a tensor processor (TPU), or other devices with instruction optimization capabilities and / or program execution capabilities. The CPU may be based on x86, ARM, RISC-V architectures, etc. The GPU may be directly integrated into the SOC, directly integrated onto the motherboard, or built into the northbridge chip of the motherboard.

[0160] like Figure 7As shown, for example, the memory may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) 703 and / or cache memory, etc., for example, computer-readable instructions may be loaded from storage device 708 into random access memory (RAM) 703 to execute computer-readable instructions. Non-volatile memory may include, for example, read-only memory (ROM) 702, hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB memory, flash memory, etc. Various applications and various data, such as style images, and various data used and / or generated by the applications, may also be stored in the computer-readable storage medium.

[0161] For example, the processing device 701, the read-only memory (ROM) 702, and the random access memory (RAM) 703 are interconnected via a bus 704. The input / output (I / O) interface 705 is also connected to the bus 704.

[0162] Typically, the following devices can be connected to the input / output (I / O) interface 705: input devices 706 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 707 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 708 including, for example, magnetic tape, hard disk, flash memory, etc.; and communication devices 709. Communication device 709 allows the second electronic device 700 to communicate wirelessly or wiredly with other electronic devices to exchange data. Although Figure 7 A second electronic device 700 with various devices is shown, but it should be understood that it is not required to implement or possess all of the devices shown, and the second electronic device 700 may alternatively implement or possess more or fewer devices. For example, the processing device 701 may control other components in the second electronic device 700 to perform desired functions.

[0163] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0164] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0165] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0166] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: acquire at least two Internet Protocol (IP) addresses; send a node evaluation request including the at least two IP addresses to a node evaluation device, wherein the node evaluation device selects an IP address from the at least two IP addresses and returns it; and receive the IP address returned by the node evaluation device; wherein the acquired IP address indicates an edge node in a content delivery network.

[0167] Alternatively, the aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: receive a node evaluation request including at least two Internet Protocol (IP) addresses; select an IP address from the at least two IP addresses; and return the selected IP address; wherein the received IP address indicates an edge node in the content delivery network.

[0168] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0169] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0170] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0171] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0172] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

[0173] The following points should be noted regarding this disclosure:

[0174] (1) The accompanying drawings of the embodiments of this disclosure only involve the structures involved in the embodiments of this disclosure. Other structures can be referred to the general design.

[0175] (2) Where there is no conflict, the embodiments of this disclosure and the features in the embodiments can be combined with each other to obtain new embodiments.

[0176] The above description is only a specific embodiment of this disclosure, but the protection scope of this disclosure is not limited thereto. The protection scope of this disclosure should be determined by the protection scope of the claims.

Claims

1. A data quantization circuit based on a lookup table, characterized in that, The data quantization circuit is configured to determine a target scaling factor for quantizing the input data block to a target precision and the quantization result of the input data block. The data quantization circuit includes: A data block preprocessing unit is configured to determine target input data with the largest exponential portion in the input data block, wherein the input data block includes multiple input data. An exponent alignment unit is configured to align the exponent portions of the plurality of input data to the exponent value based on the exponent portion of the target input data, thereby obtaining a plurality of intermediate data corresponding to the plurality of input data respectively; The index generation unit is configured to extract the mantissa portion of the plurality of intermediate data and combine it with N scaling factor mantissa candidates to obtain N first indices corresponding to each input data, wherein N is a positive integer and is determined based on the number of bits of the scaling factor mantissa candidates; The lookup table is configured to pre-store multiple data items and multiple index items that correspond one-to-one. The multiple index items are composed of T input mantissa candidates and N scaling factor mantissa candidates, where T is a positive integer and is determined based on the number of mantissa bits of the input data. Each data item includes the quantization error and quantization result of data quantization using the information in the corresponding index item. The output encoding unit is configured to access the lookup table according to N first indices corresponding to each input data, and determine the target scaling factor and the quantization result of the input data block according to the N query results corresponding to each of the multiple input data, wherein each query result includes the quantization error and quantization result corresponding to each first index.

2. The data quantization circuit as described in claim 1, characterized in that, Each index entry consists of one input mantissa candidate and one scaling factor mantissa candidate. The data item corresponding to the index item includes a first quantization error obtained by quantizing the first data using a first scaling factor candidate. The first scaling factor candidate uses the mantissa candidate of the one scaling factor in the index item as its mantissa. The first data uses the mantissa candidate of the one input in the index item as its mantissa. The precision format of the first scaling factor is the same as the precision format of the target scaling factor. The precision format of the first data is the same as the precision format of the plurality of input data.

3. The data quantization circuit as described in claim 2, characterized in that, Each data item also includes a first quantization result obtained by quantizing the first data using the first scaling factor, wherein the floating-point precision of the first quantization result is the target precision.

4. The data quantization circuit as described in claim 3, characterized in that, The target precision includes at least one precision format, and the first quantization result is obtained through the following steps: The mantissa portion of the first data is converted into first floating-point data, and the mantissa portion of the first scaling factor is converted into second floating-point data; Based on the precision format adopted by the first quantization result, a first value is determined, wherein the at least one precision format corresponds to a different first value, and the first value is the smallest power of 2 greater than the maximum value in the value expression range corresponding to the precision format. Perform a multiplication operation on the first value and the ratio of the first floating-point data to the second floating-point data to obtain the multiplication result; Within the numerical expression range corresponding to the precision format adopted by the first quantization result, the value closest to the multiplication result is determined as the first quantization result.

5. The data quantization circuit as described in claim 4, characterized in that, The first quantization error is obtained through the following steps: Based on the first value and the second floating-point data, perform a dequantization operation on the first quantization result to obtain a first dequantization result; Based on the first inverse quantization result and the mantissa portion of the first data, the first quantization error corresponding to the first data is determined.

6. The data quantization circuit as described in claim 1, characterized in that, The data block preprocessing unit includes a tree comparator network configured to: extract the exponential portion of the plurality of input data and determine the largest exponential portion among the exponential portions of the plurality of input data, so as to use the input data having the largest exponential portion as the target input data.

7. The data quantization circuit as described in claim 6, characterized in that, The exponent alignment unit includes a subtractor and a shifter. The subtractor is configured to: determine the difference between the exponential portion of each of the plurality of input data and the maximum exponential portion; The shifter is configured to: shift the mantissa portion of the plurality of input data to the right by the number of bits indicated by the difference, based on the difference, so as to align the exponent portion of the plurality of input data to the exponent value, and obtain a plurality of intermediate data corresponding to the plurality of input data respectively.

8. The data quantization circuit as described in claim 1, characterized in that, When the output encoding unit determines the target scaling factor based on the N query results corresponding to the plurality of input data, it includes the following operations: Based on the N query results corresponding to each of the multiple input data, determine the mantissa portion of the target scaling factor; The exponent value is shifted left by p bits to obtain the exponent part of the target scaling factor, wherein the maximum value in the numerical expression range corresponding to the precision format used for the target precision is less than 2. p And greater than 2 p-1 p is a positive integer.

9. The data quantization circuit as described in claim 8, characterized in that, When the output encoding unit determines the mantissa portion of the target scaling factor based on the N query results corresponding to the plurality of input data, it includes the following operations: The quantization errors in the N query results corresponding to the multiple input data are divided into N sets of quantization errors. Each set of quantization errors includes multiple quantization errors obtained by quantizing the multiple input data using the same scaling factor mantissa candidate. The average value of multiple quantization errors included in each set of quantization errors is calculated as the average quantization error corresponding to the candidate tail portion of the scaling factor. The candidate scaling factor with the smallest average quantization error among the N scaling factor mantissa candidates is taken as the mantissa of the target scaling factor.

10. The data quantization circuit as described in claim 1, characterized in that, When the output encoding unit performs the quantization result determination of the input data block, it includes the following operations: The second index is formed by combining the mantissa of the target scaling factor and the mantissa of each input data. Based on the second index corresponding to each of the multiple input data, the lookup table is accessed, and the second quantization result corresponding to each of the multiple input data is obtained from the multiple query results returned by the lookup table, which is used as the quantization result of the input data block.

11. The data quantization circuit as described in claim 1, characterized in that, The target precision includes at least one precision format, and each precision format corresponds to a different lookup table. The data quantization circuit is also configured to select the corresponding lookup table based on the precision format used for the target precision.

12. A data processing method, characterized in that, The data processing method includes: Identify the target input data with the largest exponential portion in the input data block, wherein the input data block includes multiple input data; Based on the exponent value of the exponent portion of the target input data, the exponent portions of the multiple input data are aligned to the exponent value to obtain multiple intermediate data corresponding to the multiple input data respectively; Extract the mantissa portion of the multiple intermediate data and combine it with N scaling factor mantissa candidates to obtain N first indices corresponding to each input data, where N is a positive integer and is determined based on the number of bits of the scaling factor mantissa candidates; Based on the N first indices corresponding to each input data, a lookup table is accessed, and based on the N query results corresponding to each of the multiple input data, a target scaling factor for quantizing the input data block to the target precision and a quantization result for the input data block are determined. Each query result includes the quantization error and quantization result corresponding to each first index. The lookup table includes multiple data items and multiple index items that correspond one-to-one. The multiple index items are composed of T input mantissa candidates and the N scaling factor mantissa candidates, where T is a positive integer and is determined based on the number of bits in the mantissa of the input data. Each data item includes the quantization error and quantization result for data quantization using the information in the corresponding index item.

13. The data processing method as described in claim 12, characterized in that, The step of determining the target scaling factor for quantizing the input data block to the target precision and the quantization result of the input data block based on the N query results corresponding to each of the multiple input data includes: Based on the N query results corresponding to each of the multiple input data, determine the mantissa portion of the target scaling factor; The exponent value is shifted left by p bits to obtain the exponent part of the target scaling factor, wherein the maximum value in the numerical expression range corresponding to the precision format used for the target precision is less than 2. p And greater than 2 p-1 p is a positive integer; The second index is constructed using the mantissa of the target scaling factor and the mantissa of each input data; and Based on the second index corresponding to each of the multiple input data, the lookup table is accessed, and the second quantization result corresponding to each of the multiple input data is obtained from the multiple query results returned by the lookup table, which is used as the quantization result of the input data block.

14. A processor, characterized in that, The processor includes multiple computing units, wherein each computing unit is provided with a data quantization circuit based on a lookup table according to any one of claims 1-11.

15. An electronic device, characterized in that, The electronic device includes the processor as described in claim 14.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions that, when executed by at least one processor, cause the at least one processor to perform the data processing method as described in claim 12 or 13.

Images