Coding method based on quantization and artificial intelligence chip

Abstract

The application relates to the field of artificial intelligence, and provides a coding and decoding method based on logarithmic quantization and an artificial intelligence chip, wherein the coding method comprises the following steps: based on the logarithmic values of elements in a data block, counting logarithmic value distribution information of the data block; based on the logarithmic value distribution information, dividing the logarithmic values of the elements in the data block to obtain at least one boundary point, and the boundary point is used for dividing different density regions; based on the boundary point, determining the density region to which the logarithmic value of each element belongs, and performing logarithmic quantization coding on each element based on the density region to which the logarithmic value of each element belongs. The coding and decoding method and the artificial intelligence chip provided by the application realize independent logarithmic quantization coding of each density region, and a relatively reasonable quantization resolution can be matched in each density region, thereby improving the reliability of logarithmic quantization coding, reducing information loss caused by logarithmic quantization coding, and improving the feasibility and effectiveness of logarithmic quantization coding in a model deployment and reasoning scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a coding and decoding method based on logarithmic quantization and an artificial intelligence chip. BACKGROUND

[0002] With the rapid development of artificial intelligence technology, logarithmic quantization technology has emerged and is widely used in large language models (LLM) to optimize their deployment and inference efficiency.

[0003] However, in the application of logarithmic quantization technology, the logarithmic function changes the distribution characteristics of the data, causing insufficient quantization resolution and outlier sensitivity problems, resulting in information loss in the quantization process and affecting the perception ability of the large language model. SUMMARY

[0004] The present application provides a coding and decoding method based on logarithmic quantization and an artificial intelligence chip to solve the problem of information loss caused by logarithmic quantization in related technologies.

[0005] The present application provides a coding method based on logarithmic quantization, comprising:

[0006] Based on the logarithmic values of each element in the data block, the logarithmic value distribution information of the data block is counted;

[0007] Based on the logarithmic value distribution information, the logarithmic values of each element in the data block are divided to obtain at least one boundary point, and the boundary point is used to divide different density regions;

[0008] Based on the boundary point, the density region to which the logarithmic value of each element belongs is determined, and each element is logarithmically quantized based on the density region to which the logarithmic value of each element belongs.

[0009] According to the coding method based on logarithmic quantization provided by the present application, the logarithmic values of each element in the data block are counted to obtain the logarithmic value distribution information of the data block, comprising:

[0010] A plurality of barrels are constructed;

[0011] The logarithmic values of each element in the data block are linearly normalized to obtain the normalized logarithmic values of each element;

[0012] Based on the normalized logarithmic values of each element and the number of barrels, the logarithmic values of each element are assigned to the corresponding barrels;

[0013] Based on the logarithmic values contained in each barrel, the logarithmic value distribution information of the data block is determined.

[0014] According to the application, a coding method based on pair quantization is provided, which comprises the following steps:

[0015] Mapping the normalized logarithmic values of the elements again to obtain logarithmic mapping values of the elements;

[0016] Based on the logarithmic mapping values of the elements and the number of the buckets, the logarithmic values of the elements are allocated to the corresponding buckets.

[0017] According to the application, a coding method based on pair quantization is provided, which comprises the following steps:

[0018] Determining the target data amount of a plurality of density regions;

[0019] For a density region, based on the order of the minimum logarithmic values of the buckets from high to low, the buckets are traversed and the logarithmic values in the traversed buckets are divided into a density region until the data amount of the density region is greater than or equal to the target data amount, and the remaining buckets are traversed for the next density region;

[0020] Based on the minimum logarithmic values of the density regions, the boundary points are determined.

[0021] According to the application, a coding method based on pair quantization is provided, which comprises the following steps:

[0022] For each element, based on the maximum logarithmic value and the minimum logarithmic value of the density region to which the logarithmic value of the element belongs, the logarithmic value of the element is encoded to obtain the quantization code of the element;

[0023] Based on the sign of the element, a sign code is determined, based on the density region to which the element belongs, a region code is determined, and the sign code, the region code and the quantization code are spliced to obtain the pair quantization code of the element.

[0024] The application further provides a decoding method based on pair quantization, which comprises the following steps:

[0025] Obtaining the pair quantization code of each element in a data block;

[0026] Based on the pair quantization code of each element, the density region to which the logarithmic value of the element belongs is determined, the density region is divided based on a boundary point, and the boundary point is obtained by dividing the logarithmic value of each element in the data block based on the logarithmic value distribution information of the data block;

[0027] Decoding the quantization coding of the pair of each element based on the density region to which the logarithm value of the pair of each element belongs, to obtain each element in the data block.

[0028] According to the decoding method based on pair quantization provided by the application, the decoding method based on the quantization coding of the pair of each element, determining the density region to which the logarithm value of the pair of each element belongs, comprises:

[0029] Determining the density region to which the logarithm value of the pair of each element belongs based on the region coding in the quantization coding of the pair of each element;

[0030] The decoding method based on the quantization coding of the pair of each element based on the density region to which the logarithm value of the pair of each element belongs, to obtain each element in the data block, comprises:

[0031] Decoding the quantization coding of the pair of each element based on the maximum and minimum logarithm value of the density region to which the logarithm value of the pair of each element belongs, to obtain the logarithm value of the pair of each element;

[0032] Determining the sign of each element based on the sign coding in the quantization coding of the pair of each element;

[0033] Restoring each element based on the sign and the logarithm value of each element.

[0034] The application further provides an artificial intelligence chip, comprising an execution unit, a quantization unit and a storage unit.

[0035] The execution unit is used to read a data block or pair quantization coding from the storage unit and send it to the quantization unit;

[0036] The quantization unit is used to execute the encoding method based on pair quantization as described above based on the received data block, and store the obtained pair quantization coding in the storage unit, or execute the decoding method based on pair quantization as described above based on the received pair quantization coding, and store the obtained data block in the storage unit.

[0037] The application further provides an electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to realize the encoding method or decoding method based on pair quantization as described above.

[0038] The application further provides a non-transitory computer readable storage medium, which stores a computer program executable by a processor to realize the encoding method or decoding method based on pair quantization as described above.

[0039] The application further provides a computer program product comprising a computer program which, when executed by a processor, implements the encoding method or decoding method based on logarithmic quantization as described above.

[0040] The log quantization-based coding method and the artificial intelligence chip provided by the application divide the log values of elements in a data block based on log value distribution information, obtain boundary points for dividing different density regions, determine the density regions to which the log values of the elements belong based on the boundary points, and perform log quantization coding on the elements based on the density regions to which the log values of the elements belong, so that the log quantization coding of each density region is independent, each density region does not interfere with each other, a relatively reasonable quantization resolution can be matched in each density region, and the influence of outliers on most elements is avoided, thereby improving the reliability of the log quantization coding, reducing the information loss caused by the log quantization coding, and improving the feasibility and effectiveness of the log quantization coding in the model deployment and inference scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the application or the related art, the following will briefly introduce the drawings needed to be used in the embodiments or the related art description. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0042] Figure 1 is a log distribution histogram in the related art.

[0043] Figure 2 is a flowchart of the log quantization-based encoding method provided by the application.

[0044] Figure 3 is a schematic diagram of the density region division based on boundary points provided by the application.

[0045] Figure 4 is a flowchart of the log quantization-based decoding method provided by the application.

[0046] Figure 5 is one of the structural schematic diagrams of the artificial intelligence chip provided by the application.

[0047] Figure 6 is another structural schematic diagram of the artificial intelligence chip provided by the application.

[0048] Figure 7 is a structural schematic diagram of the log quantization-based encoding device provided by the application.

[0049] Figure 8It is a structural schematic diagram of a decoding device based on quantization provided by the application.

[0050] Figure 9 It is a structural schematic diagram of an electronic device provided by the application. DETAILED DESCRIPTION

[0051] To make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below in combination with the drawings in the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0052] Since large language models usually have a large number of parameters and computing requirements, how to improve their storage and inference efficiency while ensuring model accuracy has become the focus of the industry. In this context, quantization technology as an effective data compression method has been widely studied and applied.

[0053] Currently, the application of quantization technology in large language models mainly reflects in two aspects:

[0054] The first is the storage compression of large model weights. By adopting a low-bit quantization scheme, such as quantizing the traditional half-precision floating-point number bf16 to a 4-bit integer, the storage volume of the model can be reduced by several times. Storage compression of large model weights can exchange significant storage space advantages within an acceptable accuracy loss range, thereby effectively alleviating the memory bottleneck faced by large language models in deployment, making it possible to run large models on consumer-grade graphics cards or edge computing devices with limited memory.

[0055] The second is the quantization of KV (Key-Value) cache during inference. To address the context bottleneck problem faced by large language models when processing long texts, i.e., the KV cache of the attention mechanism in large language models grows linearly with the sequence length, resulting in a surge in memory overhead, real-time quantization of the KV cache can significantly reduce the memory usage peak during long sequence inference, thereby enabling large language models to support and process longer context information.

[0056] The principle of quantization technology is to map floating-point values to the logarithmic domain for quantization, and the core steps are as follows:

[0057] First, input the data block, which is denoted as where m denotes the size of the data block, for example m = 128.

[0058] Secondly, the statistical calculations cover the scope of the data blocks for quantification.

[0059]

[0060]

[0061] in, and These are the minimum and maximum values ​​of the logarithmic absolute values ​​of each element in the data block, respectively. and These are the minimum and maximum logarithmic values ​​of the data block, respectively.

[0062] Next, quantitative calculations are performed:

[0063]

[0064] In the formula, Indicates the quantization step size; This represents the number of bits encoded after quantization.

[0065] Then, encoding is performed:

[0066]

[0067] in, That is, targeting elements Encode the quantified result. This is the rounding function.

[0068] Finally, decode:

[0069]

[0070] in, For the decoding result, A binary function, used to represent The symbol.

[0071] However, inherent limitations in the application of quantitative techniques restrict further improvements in their quantification effectiveness. Specifically, the use of logarithmic functions in quantitative techniques, due to the inherent characteristics of logarithmic functions, tends to transform data that approximates a normal distribution into a left-skewed (or negatively skewed) distribution. For example, Figure 1 It is a logarithmic distribution histogram in related technologies. Figure 1 The horizontal axis represents the logarithm, and the vertical axis represents the frequency. For example... Figure 1 As shown, the logarithmic distribution exhibits a left-skewed distribution.

[0072] This data distribution shift directly leads to two problems:

[0073] One problem is that the quantization resolution in the high-density area is insufficient. In a left-skewed distribution, the main part of the data is concentrated in a certain interval, forming a high-density area, and a small number of values in the tail of the distribution present extremely small extreme values. When performing quantization calculation, the minimum value of the quantization range is determined by these extremely small values, which leads to the calculation of the quantization step overly large. When encoding the majority of data in the high-density area based on the overly large quantization step , it leads to the inability to finely distinguish the majority of data in the high-density area, insufficient resolution, and thus serious information loss.

[0074] Another problem is that it is overly sensitive to outliers. In a left-skewed distribution, although there are only a few outliers, these outliers can significantly lower the minimum value , thereby greatly affecting the determination of the entire quantization range. This causes the vast majority of non-outlier data values to be passively concentrated at the upper end of the quantization range, resulting in the main part of the data that needs to be finely quantized not being able to obtain sufficient quantization resources, and a small number of outliers being allocated too many quantization resources, and the allocation of quantization resources being extremely uneven.

[0075] Therefore, how to overcome the problem of left-skewed distribution of data caused by the logarithmic function transformation, and thereby solve the defects of insufficient quantization resolution and sensitivity to outliers, to improve the overall performance and precision of the quantization technology, is an important technical problem to be solved in the field at present.

[0076] To solve the above problems, embodiments of the present application provide an encoding method and a decoding method based on quantization. The encoding method and the decoding method based on quantization can be applied to artificial intelligence chips. Here, the artificial intelligence chip can be a graphics processing unit (GPU), a general-purpose computing on graphics processing units (GPGPU), a tensor processing unit (TPU), etc.

[0077] ​The encoding method and the decoding method based on pair quantization can be applied to an artificial intelligence chip to realize encoding and decoding of weights and activation values of an artificial intelligence model, thereby realizing compression and inference acceleration of the artificial intelligence model. For example, the encoding method and the decoding method based on pair quantization can be applied to the field of image processing, specifically, can be applied to image processing tasks such as target detection and image segmentation, and can be applied to encoding and decoding of weights and activation values of models such as target detection and image segmentation, thereby realizing compression and inference acceleration of the models such as target detection and image segmentation; for another example, the encoding method and the decoding method based on pair quantization can be applied to the field of natural language processing, specifically, can be applied to natural language processing tasks such as text generation and semantic understanding, and can be applied to encoding and decoding of weights and activation values of models such as text generation and semantic understanding, thereby realizing compression and inference acceleration of the models such as text generation and semantic understanding.

[0078] Figure 2 is a flowchart of the encoding method based on pair quantization provided by the present application, as shown in Figure 2 , the encoding method comprises:

[0079] In step 210, based on the pair values of each element in the data block, the pair value distribution information of the data block is counted.

[0080] Here, the data block is a data block that needs to be quantized by pairs to realize compression. The data block can be a weight in an artificial intelligence model, or an activation value generated in the inference process of the artificial intelligence model. For example, when the encoding method based on pair quantization is applied to an image segmentation model, the data block can be a weight of the image segmentation model, for example, a weight of a convolution layer in the image segmentation model; or the data block can be an activation value in the image segmentation model, for example, an image feature output by a convolution layer in the image segmentation model. For another example, when the encoding method based on pair quantization is applied to a text generation model, the data block can be a weight of an attention module in the text generation model; or the data block can be an activation value in the text generation model, for example, a text feature output by the attention module in the text generation model, which is not limited in the present application.

[0081] An element is a single numerical value used to constitute a data block. For example, in a data block , there are m elements, that is, to are elements in the data block.

[0082] For each element in the data block, the pair value of the element can be calculated. Here, specifically, the logarithmic value of the absolute value of the element can be calculated, for example, for the element , the logarithmic value of the element can be represented as , .

[0083] After obtaining the logarithmic value of each element in the data block, the logarithmic value distribution information of the data block can be statistically analyzed. This logarithmic value distribution information reflects the data distribution of the logarithmic values ​​of each element in the data block; for example, it can reflect the distribution density of the logarithmic values ​​of each element. The logarithmic value distribution information can be represented in the form of a non-linear histogram, a density plot, or other forms; this embodiment of the invention does not specifically limit its representation.

[0084] Step 220: Based on the logarithmic value distribution information, divide the logarithmic values ​​of each element in the data block to obtain at least one boundary point, which is used to divide regions of different densities.

[0085] Specifically, after obtaining the logarithmic value distribution information of the data block, density regions can be divided based on the logarithmic value of each element in the data block, thereby obtaining multiple density regions. The logarithmic values ​​of elements within a density region exhibit similar distribution densities, while the logarithmic values ​​of elements within different density regions exhibit different distribution densities.

[0086] Understandably, during the process of dividing density regions, the boundary points used to divide these regions can be obtained. Here, the boundary points are the thresholds for dividing density regions. The boundary points can be the maximum or minimum value of the logarithmic values ​​of all elements in the density region, thus serving as the boundaries that separate the density region from other density regions.

[0087] In this embodiment of the invention, the number of boundary points can be one or more. For example, with one boundary point, two density regions can be obtained, corresponding to, for example... Figure 1 In the logarithmic distribution histogram, the boundary points can be... Figure 1 The illustrated left-skewed distribution is divided into two density regions: a high-density region and a low-density region. The high-density region refers to the main data portion of the left-skewed distribution, while the low-density region refers to the left-side tail region. Alternatively, with two boundary points, three density regions can be obtained: a high-density region, a medium-density region, and a low-density region. This embodiment of the invention does not impose specific limitations on these regions.

[0088] Step 230: Determine the density region to which the logarithm of each element belongs based on the boundary points, and perform logarithmic encoding on each element based on the density region to which the logarithm of each element belongs.

[0089] Specifically, after the boundary point is obtained, the logarithm values of the elements in the data block can be assigned to the density region to which the elements belong by comparing the boundary point with the logarithm values of the elements in the data block. For example, in the case where there is one boundary point, the logarithm values of the elements in the data block can be compared with the boundary point, and the logarithm values of the elements greater than or equal to the boundary point are placed in the high-density region, and the logarithm values of the elements less than the boundary point are placed in the low-density region.

[0090] After the belonging relationship between the density region and the logarithm values of the elements in the data block is determined, the logarithm values of the elements in each density region can be quantized independently for each density region.

[0091] For example, in the case where the high-density region and the low-density region are divided by one boundary point, the logarithm values of the elements belonging to the high-density region can be determined, and the maximum logarithm value and the minimum logarithm value of the high-density region are obtained therefrom, and the quantization step of the high-density region is determined based on the maximum logarithm value and the minimum logarithm value, so that the logarithm values of the elements belonging to the high-density region are quantized. In addition, the logarithm values of the elements belonging to the low-density region can be determined, and the maximum logarithm value and the minimum logarithm value of the low-density region are obtained therefrom, and the quantization step of the low-density region is determined based on the maximum logarithm value and the minimum logarithm value, so that the logarithm values of the elements belonging to the low-density region are quantized.

[0092] It can be understood that in the process of quantizing the logarithm values of the elements in each density region, the parameters used for quantization are irrelevant to other density regions, and each density region does not interfere with each other.

[0093] Figure 3 is a schematic diagram of density region division based on a boundary point provided by the present application, as shown in Figure 3 , a boundary point pivot y is used to divide the density region, and the part with a logarithm value greater than or equal to pivot y is taken as the high-density region, and the part with a logarithm value less than pivot y is taken as the low-density region. Thus, compared with the quantization scheme in the related art, when the logarithm values of the elements in the high-density region are quantized, the minimum logarithm value of the high-density region is no longer the outlier of the left-skewed distribution, but the boundary point pivot y , and pivot y is taken as the minimum logarithm value, and the quantization step of the high-density region calculated therefrom is , the value is smaller, thereby being able to provide a higher-fineness distinction for the high-density area, and the quantization resolution of the high-density area is greatly increased. In addition, the outlier in the left-skewed distribution no longer affects the minimum logarithmic value of the high-density area, and the quantization of the number of pairs for the high-density area is no longer sensitive to outliers.

[0094] After the number-of-pair quantization encoding is independently performed for each density area, the number-of-pair quantization encoding of each element in the data block is obtained. It can be understood that the number of bits occupied by the number-of-pair quantization encoding is less than that of the element itself, and thus the number-of-pair quantization encoding of each element in the data block can be stored to replace the data block, and the number-of-pair quantization encoding is transmitted in subsequent applications, thereby greatly reducing the data size from the perspective of data storage and data transmission, and reducing the requirement for storage space. When the data block needs to be applied for calculation in the subsequent, the number-of-pair quantization encoding can be decoded to restore the data block, and the restored data block is applied for calculation.

[0095] For example, in the case that the data block is the weight of an image segmentation model, the number-of-pair quantization encoding of the weight can be stored, and when image segmentation is needed, the stored number-of-pair quantization encoding is read and decoded, thereby obtaining the weight of the image segmentation model to realize model inference. For example, in the case that the data block is the activation value of an image segmentation model, the activation value can be encoded after the activation value is obtained to obtain the number-of-pair quantization encoding of the activation value and be transmitted, and the number-of-pair quantization encoding is decoded to obtain the activation value at the lower-level computing unit and be used for subsequent inference steps.

[0096] In the method provided in the embodiments of the present application, based on the logarithmic value distribution information, the logarithmic values of the elements in the data block are divided to obtain boundary points for dividing different density areas, and the density area to which each element belongs is determined based on the boundary points, and each element is number-of-pair quantization encoded based on the density area to which the element belongs, thereby realizing number-of-pair quantization encoding of each density area independently, each density area does not interfere with each other, a relatively reasonable quantization resolution can be matched in each density area, and the influence of outliers on most elements is avoided, thereby improving the reliability of the number-of-pair quantization encoding, reducing the information loss caused by the number-of-pair quantization encoding, and improving the feasibility and effectiveness of the number-of-pair quantization encoding in the model deployment and inference scenarios.

[0097] Based on the above embodiments, in step 210, the logarithmic value distribution information of the data block is counted based on the logarithmic values of the elements in the data block, including:

[0098] A plurality of buckets are constructed;

[0099] The logarithmic values of the elements in the data block are linearly normalized to obtain the normalized logarithmic values of the elements;

[0100] distributing the logarithm values of the elements to corresponding bins based on the normalized logarithm values of the elements and the number of the bins;

[0101] determining the logarithm value distribution information of the data block based on the logarithm values contained in each bin.

[0102] Specifically, for the logarithm values of the elements in the data block, the statistics of the data distribution information can be achieved by means of a plurality of bins.

[0103] Here, the bin is an interval for grouping the logarithm values of the elements in the data block, and the number of the bins can be determined based on the data size of the data block. It can be understood that the larger the data size of the data block is, the more the number of the bins is. For example, for a data block containing 128 elements, 16 bins can be set.

[0104] In addition, for the logarithm values of the elements in the data block, the logarithm values of the elements can be linearly normalized based on the maximum value and the minimum value in the logarithm values of all the elements, and the logarithm values of the elements obtained by the linear normalization operation are recorded as the normalized logarithm values of the elements. It can be understood that by linearly normalizing the logarithm values of the elements, the logarithm values of the elements are uniformly mapped to the standard interval [0, 1], and the normalized logarithm values of the elements obtained thereby can reflect the relative position of the logarithm values of the elements in the logarithm values of all the elements in the data block, and can also be understood as a linear position in the data interval defined by the maximum value and the minimum value of the logarithm values of all the elements in the data block.

[0105] After obtaining the normalized logarithm value of each element, the logarithm values of the elements can be distributed to the corresponding bins based on the normalized logarithm values of the elements and the number of the bins. For example, for any element, the normalized logarithm value of the element can be multiplied by the number of the bins, thereby obtaining the index position of the bin corresponding to the logarithm value of the element. Thus, the logarithm values of each element in the data block can be distributed to the corresponding bins. It can be understood that based on the normalized logarithm values of the elements and the number of the bins, the distribution of the logarithm values of the elements can be achieved, and thus after the distribution, the range of the logarithm values contained in each bin is equal, and the width of each bin itself changes regularly to adapt to the actual distribution characteristics of the logarithm values.

[0106] Thereafter, for each bin, the logarithm value distribution information of the data block can be determined based on the logarithm values contained in each bin. For example, in the case that the range of the logarithm values contained in each bin is the same, the number of the logarithm values contained in each bin can reflect the density of the logarithm values of the elements in the data block in different logarithm value ranges.

[0107] Based on any of the above embodiments, in step 210, the log values of the elements are assigned to corresponding buckets based on the normalized log values of the elements and the number of the buckets, including:

[0108] performing secondary mapping on the normalized log values of the elements to obtain log mapping values of the elements;

[0109] assigning the log values of the elements to corresponding buckets based on the log mapping values of the elements and the number of the buckets.

[0110] Specifically, after obtaining the normalized log values of the elements in the data block, secondary mapping can also be performed on the normalized log values of the elements. Here, the secondary mapping is to square the normalized log values, and the square of the normalized log value can be taken as the log mapping value. It can be understood that the normalized log value is a value in the range of [0, 1], and the secondary mapping on the normalized log value makes the log mapping value have stronger distinguishability compared with other log mapping values, compared with the normalized log value having stronger distinguishability compared with other normalized log values.

[0111] On this basis, the log mapping value of the element can be multiplied by the number of the buckets to obtain the index position of the bucket corresponding to the log value of the element, and then the log value of the element is assigned to the corresponding bucket.

[0112] For example, K buckets can be initialized, and the counters of the K buckets can be initialized to 0. Here, K can be 16, or 32, 48, or other values. In addition, the minimum log value of each bucket is initialized to +∞. B B B

[0113] Subsequently, for the log value of each element , the following steps are performed:

[0114] Step a. calculating the normalized log value of the element :

[0115]

[0116] wherein . and are the minimum log value and the maximum log value of the data block, respectively.

[0117] Step b. performing secondary mapping on the normalized log value to obtain the log mapping value :​​​​​​​

[0118]

[0119] Step c. Determine the element Corresponding bucket index :

[0120]

[0121] Wherein, Indicates rounding down.

[0122] Step d. Assign the logarithmic value of the element to the corresponding bucket index After updating the count of the bucket index +1.

[0123] Step e. Update the minimum logarithmic value of the corresponding bucket of the bucket index :

[0124]

[0125] Thus, the logarithmic values of all elements in the data block can be assigned to the above-mentioned B buckets, and the logarithmic value distribution information of the data block is obtained.

[0126] Based on any of the above embodiments, in step 220, based on the logarithmic value distribution information, dividing the logarithmic values of each element in the data block to obtain at least one boundary point, comprising:

[0127] Determining the target data amount of a plurality of density regions;

[0128] For a density region, based on the order from high to low of the minimum logarithmic value of each bucket, traversing each bucket and dividing the logarithmic values in the traversed bucket into a density region, until the data amount of the density region is greater than or equal to the target data amount, and returning to traverse the remaining buckets for the next density region;

[0129] Based on the minimum logarithmic value of each density region, determining the boundary point.

[0130] ​​​In particular, the number of density regions to be divided can be determined in advance, and the amount of data each density region is expected to contain, i.e., the target data amount of each density region, can be determined in advance. For example, the number of density regions to be divided can be determined in advance as 2, where the target data amount of one density region is 80% or 90% of the total data amount of the data block, and the target data amount of the other density data is 20% or 10% of the total data amount of the data block, etc. By selecting a reasonable target data amount for the density region, it can be ensured that the high-density region of actual interest is sufficiently covered.

[0131] For each density region, the density regions can be arranged in descending order of the logarithmic values of the required division, and the density region with the largest logarithmic value of the required division can be determined first.

[0132] In addition, for each bucket, the buckets can be arranged in descending order of the minimum logarithmic value of each bucket.

[0133] First, for the density region with the largest logarithmic value of the required division, the buckets can be traversed in descending order of the minimum logarithmic value of each bucket, the logarithmic values in the traversed buckets can be divided into the density region, and after each traversal of a bucket is completed, the data amount of the density region can be updated, it can be determined whether the data amount of the density region is greater than or equal to the target data amount, if the data amount is less than the target data amount, the bucket traversal can be continued, the logarithmic values in the next bucket can be divided into the density region, and the data amount of the density region can be updated again to determine whether the data amount of the density region is greater than or equal to the target data amount; if it is greater than or equal to the target data amount, the bucket traversal can be stopped, and it can be determined that the data division for the density region is completed.

[0134] Subsequently, for the next density region, the buckets that have not been divided into the previous density region can be traversed, the logarithmic values in the traversed buckets can be divided into the density region, and after each traversal of a bucket is completed, the data amount of the density region can be updated, it can be determined whether the data amount of the density region is greater than or equal to the target data amount, if the data amount is less than the target data amount, the bucket traversal can be continued, the logarithmic values in the next bucket can be divided into the density region, and the data amount of the density region can be updated again to determine whether the data amount of the density region is greater than or equal to the target data amount; if it is greater than or equal to the target data amount, the bucket traversal can be stopped, and it can be determined that the data division for the density region is completed. In this way, the logarithmic values in all buckets can be divided into the density regions.

[0135] After the division of the logarithmic values in the buckets into the density regions is completed, the minimum logarithmic value of each density region can be obtained, and thus the boundary point can be obtained.

[0136] For example, for the case of planning to divide into two density regions, i.e., a high-density region and a low-density region, the target data amount of the high-density region can be set as 80% of the total data amount of the data block, and the target data amount of the low-density region can be set as 20% of the total data amount of the data block. .

[0137] Secondly, the data volume counter can be initialized .

[0138] Then, the bucket traversal can be performed from high bucket to low bucket, where the high bucket to low bucket is determined by the bucket index, for example, can be determined by the bucket index to 0, and the specific operations include:

[0139] Step a. cumulative count: , where is the bucket index traversed;

[0140] Step b. if , the boundary point is determined and the traversal is ended.

[0141] If the boundary point is not determined after traversing all the buckets, the boundary point is determined.

[0142] In the embodiments of the present application, the boundary point is determined by the bucket traversal, so as to realize the division of different density regions. Compared with the scheme of directly traversing all the elements in the data block to realize the arrangement of the logarithmic values from high to low and then determine the boundary point, the bucket traversal based scheme requires less operation and has lower complexity, which is helpful to realize efficient quantization of the logarithmic values.

[0143] Based on any of the above embodiments, in step 230, the quantization encoding of each element based on the density region to which the logarithmic value of the element belongs includes:

[0144] For each element, the logarithmic value of the element is encoded based on the maximum logarithmic value and the minimum logarithmic value of the density region to which the logarithmic value of the element belongs, to obtain the quantization encoding of the element.

[0145] The sign encoding is determined based on the sign of the element, the region encoding is determined based on the density region to which the element belongs, and the sign encoding, the region encoding and the quantization encoding are spliced to obtain the quantization encoding of the element.

[0146] Specifically, the logarithmic encoding of each element in a data block can include three parts: sign encoding, region encoding, and quantization encoding. Sign encoding represents the sign of the element, specifically whether it is positive or negative. Region encoding represents the density region to which the element's logarithmic value belongs, specifically an identifier for that density region. Quantization encoding is the encoding obtained by quantizing the element's logarithmic value. Specifically, the quantization operation can be based on the maximum and minimum logarithmic values ​​of the density region to which the element's logarithmic value belongs, determining the quantization step size within that density region, and then quantizing the element's logarithmic value based on that step size and the minimum logarithmic value.

[0147] The resulting logarithmic encoding of the elements can be n The encoding consists of 1 bit, specifically represented as 1 bit of symbol encoding, 1 bit of region encoding, and... n -2-bit quantization encoding concatenation encoding.

[0148] For example, for each non-zero element in the data block :

[0149] First, calculate the elements. logarithm .

[0150] Secondly, assuming there are two density regions, namely a high-density region and a low-density region, it can be determined as follows: Density region to which it belongs:

[0151] like ,but It belongs to a high-density area; if ,but It belongs to a low-density area.

[0152] Subsequently, it can be based on The maximum and minimum logarithms of the density region, for Encode to obtain the element Quantization encoding:

[0153] Among them, targeting In the case of high-density regions, the maximum logarithm value of high-density regions is The smallest logarithm is The logarithmic range of the high-density region is Therefore, in response to The quantization code obtained after encoding can be represented as:

[0154]

[0155] against In the case of low-density areas, the maximum logarithm of low-density areas is The smallest logarithm is The logarithmic range of the low-density region is Therefore, in response to The quantization code obtained after encoding can be represented as:

[0156]

[0157] Therefore, it is possible to generate n Logarithmic encoding of bits:

[0158]

[0159] In this context, region code = 0 indicates a low-density region, and region code = 1 indicates a high-density region.

[0160] Figure 4 This is a flowchart illustrating the decoding method based on quantization provided by the present invention, as shown below. Figure 4 As shown, the decoding method includes:

[0161] Step 410: Obtain the logarithmic encoding of each element in the data block.

[0162] Here, a data block refers to a data block compressed through quantization encoding. A data block can be a weight in an artificial intelligence model, or an activation value generated by the model during inference. For example, when applying a quantization-based encoding method to an image segmentation model, a data block can be a weight of the image segmentation model, such as the weight of a convolutional layer; or a data block can be an activation value in the image segmentation model, such as image features output by a convolutional layer. As another example, when applying a quantization-based encoding method to a text generation model, a data block can be a weight of the attention module in the text generation model; or a data block can be an activation value in the text generation model, such as text features output by the attention module. This embodiment of the invention does not specifically limit this.

[0163] An element is a single numerical value used to make up a data block, for example, in a data block. In, there exists m 1 element, that is, in the data block to All of these are elements within the data block.

[0164] After the encoding operation based on the pair quantization is performed on the data block, the pair quantization encoding of each element in the data block is obtained. Compared with the element itself, the pair quantization encoding occupies fewer bits, and thus the pair quantization encoding of each element in the data block can be stored to replace the data block, and the pair quantization encoding is transmitted in subsequent applications, so that the data size is greatly reduced from the perspective of data storage and data transmission, and the requirement for storage space is reduced.

[0165] In the embodiment of the present application, in order to apply the data block to calculation, the pair quantization encoding of each element in the data block that is pre-encoded and stored can be read or loaded.

[0166] For example, in the case that the data block is the weight of an image segmentation model, the pair quantization encoding of the weight can be stored, and when image segmentation is needed, the pre-stored pair quantization encoding is read. For example, in the case that the data block is the activation value of an image segmentation model, the upper-level calculation unit can encode the activation value to obtain the pair quantization encoding of the activation value after obtaining the activation value and transmit it, so that the pair quantization encoding transmitted by the upper-level calculation unit can be received.

[0167] In step 420, based on the pair quantization encoding of each element, a density region to which the pair value of each element belongs is determined, and the density region is divided based on a boundary point, and the boundary point is obtained based on the pair value distribution information of the data block.

[0168] In step 430, based on the density region to which the pair value of each element belongs, the pair quantization encoding of each element is decoded to obtain each element in the data block.

[0169] Specifically, after obtaining the pair quantization encoding of each element in the data block, the pair quantization encoding of each element can be decoded. Here, the process of the pair quantization encoding can be the inverse operation of the pair quantization encoding of the element.

[0170] In the process of the pair quantization encoding of the element, the pair value of the element is first determined, and the density region to which the pair value of the element belongs is allocated, and then the pair quantization encoding of the element is performed based on the density region to which the pair value of the element belongs. Here, the division of the density region can be realized based on the boundary point, and the boundary point can be determined by the pair value distribution information of the data block.

[0171] Therefore, in the process of decoding the pair quantization encoding, the density region to which the pair value of the element belongs needs to be determined from the pair quantization encoding. For example, the region encoding representing the density region to which the pair value of the element belongs can be determined from the pair quantization encoding, so that the density region to which the pair value of the element belongs is determined.

[0172] On this basis, the pair quantization coding can be decoded based on the density region to which the logarithmic value belongs.

[0173] In the method provided in the embodiments of the present application, the logarithmic values of the elements in the data block are divided based on the logarithmic value distribution information, the boundary points for dividing different density regions are obtained, the density region to which the logarithmic value of each element belongs is determined based on the boundary points, and each element is pair quantization coded based on the density region to which the logarithmic value of each element belongs. In this way, the pair quantization coding of each density region is realized independently, the density regions do not interfere with each other, a relatively reasonable quantization resolution can be matched in each density region, and the influence of outliers on most elements is avoided. On this basis, the pair quantization coding and decoding can reduce the information loss caused by pair quantization and improve the feasibility and effectiveness of pair quantization coding and decoding in model deployment and inference scenarios.

[0174] Based on any of the above embodiments, in step 420, the density region to which the logarithmic value of each element belongs is determined based on the pair quantization coding of each element, including:

[0175] The density region to which the logarithmic value of each element belongs is determined based on the region coding in the pair quantization coding of each element.

[0176] Specifically, the sign coding is used to represent the sign of the element, specifically indicating whether the element is positive or negative; the region coding is used to represent the density region to which the logarithmic value of the element belongs, which can be the identifier of the density region; the quantization coding of the element can be the coding obtained by quantizing the logarithmic value of the element, and the specific quantization operation can be to determine the step size for quantization in the density region to which the logarithmic value of the element belongs based on the maximum logarithmic value and the minimum logarithmic value of the density region, and to quantize the logarithmic value of the element based on the step size and the minimum logarithmic value. The pair quantization coding of the element formed based on this can be n 1-bit sign coding, 1-bit region coding, and n 2-bit quantization coding.

[0177] In this way, the region coding can be obtained from the pair quantization coding, and the density region to which the logarithmic value of the element belongs can be determined based on the region coding.

[0178] For example, n The pair quantization coding of 2 bits is specifically The second bit in the pair quantization coding can be taken as the region coding, and when the region coding = 0, it is determined that the pair value of the corresponding element in the pair quantization coding belongs to the low-density region, and when the region coding = 1, it is determined that the pair value belongs to the high-density region.

[0179] Correspondingly, in step 430, the pair quantization coding of each element is decoded based on the density region to which the pair value of the element belongs, to obtain each element in the data block, including:

[0180] The quantization coding in the pair quantization coding of each element is decoded based on the maximum pair value and the minimum pair value of the density region to which the pair value of the element belongs, to obtain the pair value of the element;

[0181] The sign of each element is determined based on the sign coding in the pair quantization coding of the element;

[0182] The element is restored based on the sign and the pair value of the element.

[0183] Specifically, the sign coding can be obtained from the pair quantization coding, and then the sign of the pair value of the element is determined based on the sign coding. For example, in the pair quantization coding of 2 bits, n The first bit in the pair quantization coding can be taken as the sign coding, and when the sign coding = 0, it is determined that the pair value of the corresponding element in the pair quantization coding is positive, and when the sign coding = 1, it is determined that the pair value is negative.

[0184] In addition, the quantization coding can be obtained from the pair quantization coding, and then the absolute value of the pair value of the element is decoded based on the quantization coding. For example, in the pair quantization coding of 2 bits, n The 3rd to 4th bits in the pair quantization coding can be taken as the quantization coding. n

[0185] In addition, after determining the density region to which the pair value belongs based on the region coding, the maximum pair value and the minimum pair value of the density region stored in advance can be obtained, and then the quantization coding is decoded based on the maximum pair value and the minimum pair value.

[0186] For example, for the case that the pair value belongs to the high-density region, the maximum pair value of the high-density region is , the minimum pair value is , the pair value range of the high-density region is , and the specific decoding method can be represented by the following formula:

[0187]

[0188] wherein, is the quantization coding,​ is the absolute value of the logarithm value obtained by decoding. is the absolute value of the logarithm value obtained by decoding.

[0189] For the case that the logarithm value belongs to a low-density region, the maximum logarithm value of the low-density region is , the minimum logarithm value is , and the logarithm range of the low-density region is . The specific decoding method can be expressed as the following formula:

[0190]

[0191] wherein, is the quantization coding, is the absolute value of the logarithm value obtained by decoding.

[0192] After obtaining the sign and the logarithm value, the elements in the data block can be restored. Specifically, it can be expressed as the following steps:

[0193] First, the absolute value of the element is calculated ;

[0194] Then, when the sign coding = 1, the element is determined as ; when the sign coding = 0, the element is determined as .

[0195] Based on any of the above embodiments, Figure 5 is one of the structural diagrams of the artificial intelligence chip provided by the present application, as shown in Figure 5 , the present application provides an artificial intelligence chip 500 for executing the above encoding method or decoding method based on logarithm quantization. The artificial intelligence chip 500 includes an execution unit 510, a quantization unit 520 and a storage unit 530;

[0196] The execution unit 510 is used to read the data block or the logarithm quantization coding from the storage unit 530 and send it to the quantization unit 520;

[0197] The quantization unit 520 is used to execute the encoding method based on logarithm quantization based on the received data block, and store the obtained logarithm quantization coding in the storage unit 530, or execute the decoding method based on logarithm quantization based on the received logarithm quantization coding, and store the obtained data block in the storage unit 530.

[0198] ​Specifically, the Execution Unit (EU) 510 is the source of computing power in the artificial intelligence chip. When encoding of a data block is required, the Execution Unit 510 can read the data block to be encoded from the storage unit 530 and send the data block to the quantization unit 520. Here, the quantization unit 520 is a unit in the artificial intelligence chip set up to realize encoding and decoding based on quantization. The quantization unit has the ability to encode data blocks and decode the encoded data.

[0199] After receiving the data block to be encoded from the execution unit 510, the quantization unit 520 executes the logarithmic encoding method provided in the above embodiments on the data block, thereby encoding the data block and obtaining the logarithmic encoding of the data block. After completing the logarithmic encoding method, the quantization unit 520 stores the encoded logarithmic encoding into the storage unit 530. In addition, the quantization unit 520 can also store information such as the boundary points used in the logarithmic encoding method, the maximum logarithmic value of the density region divided based on the boundary points, and the minimum logarithmic value into the storage unit 530 for use when decoding the logarithmic encoding.

[0200] The execution unit 510 can also read the logarithmic encoding to be decoded from the storage unit 530 and send it to the quantization unit 520 when decoding of the logarithmic encoding is required. After receiving the logarithmic encoding to be decoded from the execution unit 510, the quantization unit 520 can execute the logarithmic encoding method provided in the above embodiments for the logarithmic encoding, thereby decoding the logarithmic encoding and obtaining a data block. Furthermore, the execution unit 510 can also read information such as the boundary points obtained from decoding, the maximum logarithmic value, and the minimum logarithmic value of the density regions divided based on the boundary points, from the storage unit 530, and send this information to the quantization unit 520 for use during decoding. After the quantization unit 520 decodes the data block, it can store the data block in the storage unit 530 for subsequent computational applications.

[0201] In the artificial intelligence chip provided in the embodiments of the present application, a quantization unit is added to implement a coding and decoding method based on logarithmic quantization. In the coding and decoding method, based on logarithmic value distribution information, logarithmic values of each element in a data block are divided to obtain boundary points for dividing different density regions, and based on the boundary points, a density region to which each element belongs is determined, and each element is coded by logarithmic quantization based on the density region to which each element belongs. In this way, the logarithmic quantization coding of each density region is independent, each density region does not interfere with each other, a relatively reasonable quantization resolution can be matched in each density region, and the influence of outliers on most elements is avoided, thereby improving the reliability of the logarithmic quantization coding, reducing the information loss caused by the logarithmic quantization coding, and improving the feasibility and effectiveness of the logarithmic quantization coding in the model deployment and inference scenarios.

[0202] Based on any of the above embodiments, Figure 6 is a second structural schematic diagram of the artificial intelligence chip provided in the present application, as Figure 6 shown, the artificial intelligence chip 500 includes an execution unit 510, a quantization unit 520, a storage unit, a control unit 540, a memory access unit 550, an L2 Cache (level 2 cache) 560, and a global memory 570.

[0203] The storage unit includes a register 531 and a shared memory 532; the quantization unit 520 includes a logarithm calculation unit 521, a maximum and minimum value lookup unit 522, a non-linear histogram processing unit 523, a coding unit 524, and a decoding unit 525. The memory access unit 550 includes an address calculation unit 551 and an L1 Cache (level 1 cache) 552.

[0204] After receiving a quantization instruction, the control unit 540 can schedule the execution unit 510 to send quantization parameters and data to be processed to the quantization unit 520. The quantization instruction here is a coding instruction based on logarithmic quantization, or a decoding instruction based on logarithmic quantization. The control unit 540 can determine the source of the quantization parameters and the data to be processed, and the output address of the output data obtained by executing the quantization instruction according to the information carried in the quantization instruction. The quantization parameters, the data to be processed, and the output data can be stored in the register 531 or the shared memory 532.

[0205] For the case where the quantization instruction is a coding instruction based on logarithmic quantization, the data to be processed sent by the execution unit 510 to the quantization unit 520 is a data block to be coded, and the quantization parameters can include the data block size (for example, the data block size is m, m = 128), the data type (for example, it can be bf16, fp16, fp32, etc.), the bit width of the output logarithmic quantization coding n, the output address, etc. After receiving the data block, the quantization unit 520 can execute the encoding method based on the logarithm quantization:

[0206] In the quantization unit 520, first, the logarithm values of the elements in the data block can be calculated by the logarithm calculation unit 521, and the maximum and minimum values of the logarithm values of the elements in the data block can be analyzed by the maximum and minimum value lookup unit 522. On this basis, the logarithm distribution information of the data block can be determined by the non-linear histogram processing unit 523 in combination with the logarithm values of the elements in the data block and the maximum and minimum values, so as to determine the boundary points to divide the logarithm values of the elements into different density regions. Then, the elements can be encoded by the encoding unit 524 based on the density regions to which the logarithm values of the elements belong, so as to obtain the logarithm quantization encoding of the elements in the data block, the maximum and minimum logarithm values of the density regions, the boundary points, etc. as the output data, and output the output data to the register 531 or the shared memory 532 pointed to by the output address. The logarithm quantization encoding in the output data can be specifically m bit data. n

[0207] At this point, the encoding instruction based on the logarithm quantization is executed, and a completion signal is returned to the control unit 540.

[0208] For the case that the quantization instruction is a decoding instruction based on the logarithm quantization, the data to be processed sent by the execution unit 510 to the quantization unit 520 is the logarithm quantization encoding, and the quantization parameters can include the data block size, the data type, the bit width of the logarithm quantization encoding n , the maximum and minimum logarithm values of the density regions, the boundary points, the output address, etc. After receiving the logarithm quantization encoding and the quantization parameters such as the boundary points of the density regions, the quantization unit 520 can execute the decoding method based on the logarithm quantization:

[0209] In the quantization unit 520, the decoding unit 525 can decode the logarithm quantization encoding based on the quantization parameters, so as to decode the elements in the data block as the output data, and output the output data to the register 531 or the shared memory 532 pointed to by the output address. The output data is m an element value with a high bit width.

[0210] At this point, the decoding instruction based on the logarithm quantization is executed, and a completion signal is returned to the control unit 540.

[0211] ​The pair quantization-based encoding device and decoding device provided by the present application are described below, and the pair quantization-based encoding device and decoding device described below can be mutually corresponding with the pair quantization-based encoding method and decoding method described above.

[0212] Figure 7 is a structural schematic diagram of the pair quantization-based encoding device provided by the present application, as Figure 7 indicated, the device comprises:

[0213] The statistical module 710 is configured to statistically obtain the logarithm value distribution information of the data block based on the logarithm values of the elements in the data block.

[0214] The division module 720 is configured to divide the logarithm values of the elements in the data block based on the logarithm value distribution information to obtain at least one boundary point, and the boundary point is used to divide different density regions.

[0215] The encoding module 730 is configured to determine the density region to which the logarithm value of each element belongs based on the boundary point, and perform pair quantization encoding on each element based on the density region to which the logarithm value of each element belongs.

[0216] In the device provided by the embodiment of the present application, the logarithm values of the elements in the data block are divided based on the logarithm value distribution information to obtain the boundary point used to divide different density regions, and the density region to which the logarithm value of each element belongs is determined based on the boundary point, and each element is pair quantization encoded based on the density region to which the logarithm value of each element belongs, thereby realizing pair quantization encoding of each density region independently, each density region does not interfere with each other, a relatively reasonable quantization resolution can be matched in each density region, and the influence of outliers on most elements is avoided, thereby improving the reliability of pair quantization encoding, reducing the information loss caused by pair quantization encoding, and improving the feasibility and effectiveness of pair quantization encoding and decoding in model deployment and inference scenarios.

[0217] According to any one of the above embodiments, the statistical module is specifically configured to:

[0218] Construct a plurality of barrels;

[0219] Linearly normalize the logarithm values of the elements in the data block to obtain normalized logarithm values of the elements;

[0220] Distribute the logarithm values of the elements to corresponding barrels based on the normalized logarithm values of the elements and the number of barrels;

[0221] Determine the logarithm value distribution information of the data block based on the logarithm values contained in each barrel.

[0222] According to any one of the above embodiments, the statistical module is specifically configured to:

[0223] mapping the normalized logarithmic values of the elements to obtain logarithmic mapping values of the elements;

[0224] allocating the logarithmic values of the elements to corresponding buckets based on the logarithmic mapping values of the elements and the number of the buckets.

[0225] According to any one of the above embodiments, the dividing module is specifically configured to:

[0226] determine target data amounts of a plurality of density regions;

[0227] for a density region, traverse the buckets in a descending order of the minimum logarithmic values of the buckets, and divide the logarithmic values in the traversed buckets into the density region until a data amount of the density region is greater than or equal to the target data amount, and return to traverse the remaining buckets for a next density region;

[0228] determine the boundary points based on the minimum logarithmic values of the density regions.

[0229] According to any one of the above embodiments, the encoding module is specifically configured to:

[0230] for each element, encode a logarithmic value of the element based on a maximum logarithmic value and a minimum logarithmic value of a density region to which the logarithmic value of the element belongs, to obtain a quantized encoding of the element;

[0231] determine a symbol encoding based on a symbol of the element, determine a region encoding based on a density region to which the element belongs, and splice the symbol encoding, the region encoding and the quantized encoding to obtain a pair quantized encoding of the element.

[0232] Figure 8 is a structural schematic diagram of a decoding device based on pair quantization provided by the present application, as shown in Figure 8 The device comprises:

[0233] an obtaining module 810, configured to obtain pair quantized encodings of elements in a data block;

[0234] a partitioning module 820, configured to determine density regions to which logarithmic values of the elements belong based on the pair quantized encodings of the elements, the density regions being divided based on boundary points, the boundary points being obtained by dividing logarithmic values of the elements in the data block based on logarithmic value distribution information of the data block;

[0235] a decoding module 830, configured to decode the pair quantized encodings of the elements based on the density regions to which the logarithmic values of the elements belong, to obtain the elements in the data block.

[0236] In the device provided in the embodiments of the present application, based on the logarithm value distribution information, the logarithm values of the elements in the data block are divided to obtain boundary points for dividing different density regions, and the density region to which each element belongs is determined based on the boundary points, and each element is quantized and encoded based on the density region to which the element belongs, so that the quantization coding of each density region is independent, each density region does not interfere with each other, a relatively reasonable quantization resolution can be matched in each density region, and the influence of outliers on most elements is avoided, and on this basis, the quantization coding and decoding can reduce the information loss caused by quantization, and improve the feasibility and effectiveness of the quantization coding and decoding in the model deployment and inference scenarios.

[0237] Based on any of the above embodiments, the partition module is specifically used for:

[0238] Based on the region coding in the quantization coding of the elements, the density region to which each element belongs is determined;

[0239] The decoding module is specifically used for:

[0240] Based on the maximum and minimum logarithm values of the density region to which each element belongs, the quantization coding in the quantization coding of the elements is decoded to obtain the logarithm values of the elements;

[0241] Based on the sign coding in the quantization coding of the elements, the sign of each element is determined;

[0242] Based on the sign and logarithm value of each element, the element is restored.

[0243] Figure 9 An example of an entity structure diagram of an electronic device is shown in Figure 9 The electronic device can include a processor 910, a communications interface 920, a memory 930 and a communications bus 940, wherein the processor 910, the communications interface 920 and the memory 930 communicate with each other through the communications bus 940. The processor 910 can call the logical instructions in the memory 930 to execute the quantization-based coding method, which includes:

[0244] Based on the logarithm values of the elements in the data block, the logarithm value distribution information of the data block is counted;

[0245] Based on the logarithm value distribution information, the logarithm values of the elements in the data block are divided to obtain at least one boundary point, and the boundary point is used to divide different density regions;

[0246] Determine a density region to which the logarithm value of each element belongs based on the boundary point, and quantize and encode each element based on the density region to which the logarithm value of each element belongs.

[0247] Alternatively, the processor 910 can invoke the logical instructions in the memory 930 to perform a decoding method based on quantization, which includes:

[0248] Obtain the quantization encoding of each element in the data block;

[0249] Determine a density region to which the logarithm value of each element belongs based on the quantization encoding of each element, the density region being divided based on a boundary point, the boundary point being obtained based on the logarithm value distribution information of the data block and the logarithm value of each element in the data block;

[0250] Decode the quantization encoding of each element based on the density region to which the logarithm value of each element belongs, and obtain each element in the data block.

[0251] In addition, the logical instructions in the memory 930 described above can be implemented in the form of a software function unit and sold or used as a stand-alone product, which can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application or the parts that essentially contribute to the related art or parts of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, and various storage medium that can store program codes.

[0252] On the other hand, the present application also provides a computer program product, which includes a computer program, the computer program can be stored on a non-transitory computer readable storage medium, and the computer program can be executed by a processor to enable a computer to perform the encoding method based on quantization provided by the above-mentioned methods, which includes:

[0253] Based on the logarithm value of each element in the data block, statistics the logarithm value distribution information of the data block;

[0254] Based on the logarithm value distribution information, divide the logarithm value of each element in the data block to obtain at least one boundary point, the boundary point being used to divide different density regions;

[0255] determining, based on the log value of each element, a density region to which the log value of each element belongs, the density region being divided based on a boundary point, the boundary point being obtained based on log value distribution information of the data block;

[0256] Alternatively, the computer can execute a decoding method based on the log quantization provided by the above-mentioned methods, the method comprising:

[0257] obtaining log quantization encoding of each element in a data block;

[0258] determining, based on the log quantization encoding of each element, a density region to which the log value of each element belongs, the density region being divided based on a boundary point, the boundary point being obtained based on log value distribution information of the data block;

[0259] decoding the log quantization encoding of each element based on the density region to which the log value of each element belongs, to obtain each element in the data block.

[0260] In another aspect, the present application also provides a non-transitory computer readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement an encoding method based on log quantization provided by the above-mentioned methods, the method comprising:

[0261] based on the log value of each element in a data block, obtaining log value distribution information of the data block;

[0262] based on the log value distribution information, dividing the log value of each element in the data block to obtain at least one boundary point, the boundary point being used to divide different density regions;

[0263] determining, based on the log value of each element, a density region to which the log value of each element belongs, the density region being divided based on a boundary point, the boundary point being obtained based on log value distribution information of the data block;

[0264] Alternatively, the computer program is executed by a processor to implement a decoding method based on log quantization provided by the above-mentioned methods, the method comprising:

[0265] obtaining log quantization encoding of each element in a data block;

[0266] determining, based on the log quantization encoding of each element, a density region to which the log value of each element belongs, the density region being divided based on a boundary point, the boundary point being obtained based on log value distribution information of the data block;

[0267] decoding the log quantization encoding of each element based on the density region to which the log value of each element belongs, to obtain each element in the data block.

[0268] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment scheme according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0269] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software and the necessary general hardware platform, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0270] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A coding method based on quantization, characterized in that, include: Based on the logarithm of each element in the data block, the logarithm distribution information of the data block is statistically analyzed. The logarithm distribution information is used to reflect the distribution density of the logarithm of each element in the data block. Based on the logarithmic value distribution information, the logarithmic values ​​of each element in the data block are divided to obtain at least one boundary point, which is used to divide regions of different densities. Based on the boundary points, the density region to which the logarithm of each element belongs is determined, and each element is logarithmically encoded based on the density region to which the logarithm of each element belongs.

2. The encoding method based on quantization according to claim 1, characterized in that, The step of calculating the logarithmic distribution information of the data block based on the logarithmic values ​​of each element in the data block includes: Build multiple buckets; The logarithmic values ​​of each element in the data block are linearly normalized to obtain the normalized logarithmic values ​​of each element. Based on the normalized logarithmic value of each element and the number of buckets, the logarithmic value of each element is assigned to the corresponding bucket. Based on the logarithmic values ​​contained in each bucket, the logarithmic value distribution information of the data block is determined.

3. The encoding method based on quantization according to claim 2, characterized in that, The step of allocating the logarithm of each element to the corresponding bucket based on the normalized logarithm of each element and the number of buckets includes: A second mapping is performed on the normalized logarithmic values ​​of each element to obtain the logarithmic mapped values ​​of each element; Based on the logarithmic mapping value of each element and the number of buckets, the logarithmic value of each element is assigned to the corresponding bucket.

4. The encoding method based on quantization according to claim 2, characterized in that, The step of dividing the logarithmic values ​​of each element in the data block based on the logarithmic distribution information to obtain at least one boundary point includes: Determine the target data volume for multiple density regions; For a density region, based on the minimum logarithm of each bucket in descending order, traverse each bucket and assign the logarithm of the traversed bucket to a density region, until the data volume of the density region is greater than or equal to the target data volume, then return to traverse the remaining buckets for the next density region. The boundary points are determined based on the minimum logarithmic value of each density region.

5. The encoding method based on quantization according to any one of claims 1 to 4, characterized in that, The step of performing logarithmic encoding on each element based on the density region to which the logarithm of each element belongs includes: For each element, the logarithm of the element is encoded based on the maximum and minimum logarithm of the density region to which the logarithm of the element belongs, thus obtaining the quantization code of the element. The symbol encoding is determined based on the symbol of the element, the region encoding is determined based on the density region to which the element belongs, and the symbol encoding, the region encoding and the quantization encoding are concatenated to obtain the logarithmic quantization encoding of the element.

6. A decoding method based on quantization, characterized in that, include: Obtain the logarithmic encoding of each element in the data block; Based on the logarithmic encoding of each element, the density region to which the logarithm of each element belongs is determined. The density region is divided based on boundary points, which are obtained by dividing the logarithm of each element in the data block based on the logarithm distribution information of the data block. Based on the density region to which the logarithmic value of each element belongs, the logarithmic encoding of each element is decoded to obtain each element in the data block.

7. The decoding method based on quantization according to claim 6, characterized in that, The step of determining the density region to which the logarithm of each element belongs based on the logarithmic encoding of each element includes: Based on the region encoding in the logarithmic encoding of each element, the density region to which the logarithm of each element belongs is determined. The step of decoding the logarithmic encoding of each element based on the density region to which the logarithm of each element belongs to obtain each element in the data block, including: Based on the maximum and minimum logarithms of the density region to which the logarithms of each element belong, the quantization code in the logarithmic encoding of each element is decoded to obtain the logarithms of each element. The symbol of each element is determined based on the symbol encoding in the logarithmic encoding of each element; Based on the sign and logarithmic value of each element, the elements are reconstructed.

8. An artificial intelligence chip, characterized in that, It includes an execution unit, a quantization unit, and a storage unit; The execution unit is used to read data blocks from the storage unit or to quantize and encode them and send them to the quantization unit; The quantization unit is used to perform the logarithmic encoding method as described in any one of claims 1 to 5 based on the received data block, and store the encoded logarithmic code into the storage unit; or, based on the received logarithmic code, to perform the logarithmic decoding method as described in claim 6 or 7, and store the decoded data block into the storage unit.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the encoding method based on quantization as described in any one of claims 1 to 5, or the decoding method based on quantization as described in claim 6 or 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the encoding method based on quantization as described in any one of claims 1 to 5, or the decoding method based on quantization as described in claim 6 or 7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the encoding method based on quantization as described in any one of claims 1 to 5, or the decoding method based on quantization as described in claim 6 or 7.

Images